WO2019183843A1 - 基于帧间配准和自适应步长的红外图像非均匀性校正方法 - Google Patents

基于帧间配准和自适应步长的红外图像非均匀性校正方法 Download PDF

Info

Publication number
WO2019183843A1
WO2019183843A1 PCT/CN2018/080925 CN2018080925W WO2019183843A1 WO 2019183843 A1 WO2019183843 A1 WO 2019183843A1 CN 2018080925 W CN2018080925 W CN 2018080925W WO 2019183843 A1 WO2019183843 A1 WO 2019183843A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
uniformity
infrared image
pixel
original infrared
Prior art date
Application number
PCT/CN2018/080925
Other languages
English (en)
French (fr)
Inventor
周慧鑫
赵东
钱润达
郭立新
贾秀萍
周峻
黄楙森
秦翰林
姚博
于跃
李欢
宋江鲁奇
王炳健
成宽洪
杜娟
宋尚真
Original Assignee
西安电子科技大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西安电子科技大学 filed Critical 西安电子科技大学
Priority to US16/980,794 priority Critical patent/US11195254B2/en
Priority to PCT/CN2018/080925 priority patent/WO2019183843A1/zh
Publication of WO2019183843A1 publication Critical patent/WO2019183843A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/38—Registration of image sequences
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image

Definitions

  • the invention belongs to the field of infrared image processing, and in particular relates to an infrared image non-uniformity correction method based on inter-frame registration and adaptive step size.
  • the infrared detectors Due to the influence of the fabrication process and materials, the infrared detectors produce different outputs even under the same incident radiation conditions, that is, response non-uniformity; in addition, the charge transfer efficiency of different pixels is inconsistent, IRFPA is blind.
  • the influence of the element, the influence of the infrared optical system, the non-uniformity brought by the signal amplification, the 1/f noise, the non-uniformity caused by the A/D conversion, and the external temperature are all causes of non-uniformity; Images have lower resolution, lower signal-to-noise ratio, and poor contrast, so non-uniformity correction must be performed before use to improve the quality of infrared images.
  • scene-based algorithms use scene information to update correction parameters without stopping the operation of the infrared detector.
  • scene-based algorithms have become the main research object; typical scene-based algorithms include time-domain high-pass filtering, neural network, constant statistics, Kalman filtering, and inter-frame registration.
  • the inter-frame registration method has a fast convergence rate and is completely dependent on the scene, and has a certain non-uniformity correction effect.
  • the inter-frame registration method is not ideal in the case of non-uniformity.
  • the main object of the present invention is to provide an infrared image non-uniformity correction method based on inter-frame registration and adaptive step size.
  • Embodiments of the present invention provide an infrared image non-uniformity correction method based on inter-frame registration and adaptive step size, which is: establishing a linear response model of an infrared image pixel, and obtaining a correction formula by inverse transform; respectively determining The relative displacement of the original infrared image with non-uniformity of the nth frame and the n-1th frame, and the spatial variance and time variance of each pixel of the original infrared image with non-uniformity of the nth frame, and according to The spatial variance and the time variance determine the adaptive iterative step size of each pixel of the original infrared image with non-uniformity in the nth frame; the error function of the original infrared image with non-uniformity according to the n-1th frame
  • the adaptive iterative step size of the i-th row and the j-th column of the original infrared image with non-uniformity of the nth frame determines each of the
  • the gain correction coefficient, the offset correction coefficient, and the correction formula of each pixel in the overlapping region of the original infrared image with non-uniformity according to the nth frame and the n-1th frame are Non-uniformity correction is performed for each pixel of the overlap region of the original infrared image with non-uniformity of the n-frame and the n-1th frame, and then the method further comprises: determining whether the image of the n-th frame is non-non-uniform The last frame of the original infrared image sequence of uniformity, if so, completes the non-uniformity correction; if not, continues to perform non-uniformity correction on subsequent frame images.
  • y n (i,j) g n (i,j)x n (i,j)+o n (i,j)
  • g n (i, j) and o n (i, j) respectively represent the gain coefficient and the offset coefficient of the pixel of the i-th row and the j-th column in the infrared image of the nth frame
  • x n (i, j) represents The true input gray value of the i-th row and the j-th column in the n-th frame infrared image
  • y n (i, j) represents the output gray value of the i-th row and the j-th column in the n-th frame infrared image including non-uniformity
  • x n (i,j) w n (i,j)y n (i,j)+b n (i,j)
  • a gain correction coefficient of a pixel of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the nth frame and the n-1th frame An offset correction coefficient of a cell of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the nth frame and the n-1th frame.
  • the determining the relative displacement of the original infrared image with non-uniformity in the nth frame and the n-1th frame is specifically implemented by the following steps:
  • the n-th frame infrared image is obtained by relative displacement of the output gray value y n-1 (i, j) of the i-th row and the j-th column in the n- 1th frame infrared image.
  • d x and d y represent relative displacements in the horizontal direction and the vertical direction of y n (i, j) and y n-1 (i, j), respectively;
  • FFT -1 represents the inverse Fourier transform and Re represents the real part operation.
  • the spatial variance and the time variance of each pixel of the original infrared image with non-uniformity in the nth frame are determined, and the nth frame is determined to have non-uniformity according to the spatial variance and the time variance.
  • the adaptive iteration step size of each pixel of the original infrared image is specifically achieved by the following steps:
  • D represents a variance operation and m represents a positive integer less than n;
  • the error function of the original infrared image with non-uniformity according to the n-1th frame and the adaptive iteration step of the ith row and the jth column of the original infrared image with non-uniformity of the nth frame Determining the gain correction coefficient and the offset correction coefficient of each pixel in the overlapping region of the original infrared image with the non-uniformity of the nth frame and the n-1th frame, specifically by the following steps:
  • e n (i,j) (w n (id x ,jd y )y n-1 (id x ,jd y )+b n (id x ,jd y ))-(w n (i,j)y n (i,j)+b n (i,j))
  • the error function e n-1 (i, j) of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the n-1th frame and the n-2th frame is determined;
  • w n (i,j) w n-1 (i,j)+step n (i,j)e n-1 (i,j)y n-1 (i,j)(overlapped area)
  • w n-1 (i, j) represents the gain correction coefficient of the pixel of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the n-1th frame and the n-2th frame
  • the overlapped area indicates an overlapping area of the original infrared image with non-uniformity of the nth frame and the n-1th frame;
  • b n (i,j) b n-1 (i,j)+step n (i,j)e n-1 (i,j)(overlapped area)
  • b n-1 (i, j) represents the offset correction coefficient of the pixel of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the n-1th frame and the n-2th frame .
  • the invention can adaptively adjust according to the spatial characteristics and temporal characteristics of the infrared image, and has a faster convergence speed and a better correction effect.
  • Figure 1 is a flow chart of the present invention
  • 3 is a 500th frame image corrected by an inter-frame registration and adaptive iterative step non-uniformity correction algorithm according to the present invention
  • 4 is a difference image of a 500th frame image of an original image sequence with non-uniformity corrected by an inter-frame registration and adaptive iterative step size non-uniformity correction algorithm according to the present invention
  • Figure 5 is a graph of roughness in the present invention.
  • Embodiments of the present invention provide a method for correcting infrared image non-uniformity based on inter-frame registration and adaptive step size. As shown in FIG. 1 , the method is:
  • Step 1 Enter all images of the original infrared image sequence
  • FIG. 2 is an original infrared image with non-uniformity in the 500th frame according to an embodiment of the present invention
  • the original image sequence with non-uniformity has a total of 500 frames, and the image size of each frame is 320 ⁇ . 256 pixels; as can be seen from Figure 2, the original image has significant fixed pattern noise and the image quality is severely affected.
  • Step 2 Establish a linear response model of the infrared image pixel, and obtain a correction formula by inverse transformation
  • y n (i,j) g n (i,j)x n (i,j)+o n (i,j)
  • g n (i, j) and o n (i, j) respectively represent the gain coefficient and the offset coefficient of the pixel of the i-th row and the j-th column in the infrared image of the nth frame
  • x n (i, j) represents The true input gray value of the i-th row and the j-th column in the n-th frame infrared image
  • y n (i, j) represents the output gray value of the i-th row and the j-th column in the n-th frame infrared image including non-uniformity
  • x n (i,j) w n (i,j)y n (i,j)+b n (i,j)
  • a gain correction coefficient of a pixel of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the nth frame and the n-1th frame An offset correction coefficient of a cell of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the nth frame and the n-1th frame.
  • Step 3 Calculate the relative displacement of the original infrared image with non-uniformity in the nth frame and the n-1th frame;
  • the n-th frame infrared image is obtained by relative displacement of the output gray value y n-1 (i, j) of the i-th row and the j-th column in the n- 1th frame infrared image.
  • y n (i,j) y n-1 (id x ,jd y );
  • d x and d y represent relative displacements in the horizontal direction and the vertical direction of y n (i, j) and y n-1 (i, j), respectively;
  • Y n (u, v) and Y n-1 (u, v) denote images y n (i, j) and y n-1 (i, j, respectively)
  • Fourier transform u and v represent the coordinates of the Fourier domain, respectively;
  • FFT -1 represents the inverse Fourier transform and Re represents the real part operation.
  • Step 4 Calculate the spatial variance and time variance of each pixel of the original infrared image with non-uniformity in the nth frame, and obtain the original infrared image with the non-uniformity of the nth frame according to the spatial variance and the time variance Adaptive iterative step size for each cell;
  • D represents a variance operation
  • m represents a positive integer less than n, and the value of m is set to 10;
  • the adaptive iterative step size step n (i, j) of the i-th row and the j-th column of the original infrared image with non-uniformity of the nth frame is obtained according to the following equation in combination with the spatial variance and the time variance:
  • Step 5 Determine the nth according to the error function of the original infrared image with non-uniformity in the n-1th frame and the adaptive iteration step of the i-th row and the j-th column of the original infrared image with non-uniformity in the nth frame Gain correction coefficient and offset correction coefficient for each pixel in the overlap region of the original infrared image with non-uniformity of the frame and the n-1th frame;
  • e n (i,j) (w n (id x ,jd y )y n-1 (id x ,jd y )+b n (id x ,jd y ))-(w n (i,j)y n (i,j)+b n (i,j))
  • the error function e n-1 (i, j) of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the n-1th frame and the n-2th frame is determined;
  • w n (i,j) w n-1 (i,j)+step n (i,j)e n-1 (i,j)y n-1 (i,j)(overlapped area)
  • w n-1 (i, j) represents the gain correction coefficient of the pixel of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the n-1th frame and the n-2th frame
  • the overlapped area indicates an overlapping area of the original infrared image with non-uniformity of the nth frame and the n-1th frame;
  • b n (i,j) b n-1 (i,j)+step n (i,j)e n-1 (i,j)(overlapped area)
  • b n-1 (i, j) represents the offset correction coefficient of the pixel of the i-th row and the j-th column in the overlapping region of the original infrared image with non-uniformity of the n-1th frame and the n-2th frame
  • Step 6 Using the gain correction coefficient, offset correction coefficient, and correction formula of the original infrared image with non-uniformity of the nth frame to overlap the original infrared image with non-uniformity of the nth frame and the n-1th frame The pixels are non-uniformly corrected.
  • FIG. 3 is a result of correction of the 500th frame image of the original image sequence with non-uniformity in the embodiment of the present invention.
  • the electric poles above and below the corrected image are changed. It is clearly visible that the strip-shaped fixed pattern noise is almost invisible;
  • FIG. 4 is a calibration result of the 500th frame image of the original image sequence with non-uniformity and the original with non-uniformity in the embodiment of the present invention.
  • the difference image of the image of the 500th frame of the image sequence shows that the difference image contains fixed pattern noise.
  • FIG. 5 is a roughness curve diagram of an embodiment of the present invention.
  • the correction result obtained by the non-uniformity correction algorithm based on inter-frame registration and adaptive iterative step has lower roughness than the IRLMS method. It is shown that the non-uniformity correction algorithm based on inter-frame registration and adaptive iterative step size can reduce the non-uniformity of the original infrared image more effectively.
  • the infrared image sequence non-uniformity correction method based on inter-frame registration and adaptive step size proposed by the invention first calculates the normalized cross-power spectrum of the adjacent two frames of infrared images, and then uses the normalized cross-power spectrum obtained. Find the relative displacement of the infrared images of two adjacent frames, and then find the spatial variance and time variance of each pixel, and then calculate the adaptive step size of each pixel by using the spatial variance and the time variance, and update it using the iterative step size. Gain correction coefficient and offset correction coefficient, and finally non-uniformity correction of overlapping regions of adjacent two frames of infrared images.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

一种基于帧间配准和自适应步长的红外图像非均匀性校正方法，首先计算第n帧和第n-1帧带有非均匀性的原始红外图像的归一化互功率谱，然后求得第n帧和第n-1帧带有非均匀性的原始红外图像的水平相对位移和垂直相对位移；再求出第n帧带有非均匀性的原始红外图像的每一个像元的空间方差和时间方差，利用得到的空间方差和时间方差计算第n帧带有非均匀性的原始红外图像的每一个像元的自适应迭代步长，使用迭代步长更新增益校正系数和偏置校正系数；最后对第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域的像元进行非均匀性校正。

Description

基于帧间配准和自适应步长的红外图像非均匀性校正方法 技术领域
本发明属于红外图像处理领域，具体涉及一种基于帧间配准和自适应步长的红外图像非均匀性校正方法。
背景技术
由于受制作工艺和材料的影响，红外探测器即使在相同的入射辐射的条件下也会产生不同的输出，即响应非均匀性；除此之外，不同像元的电荷传输效率不一致、IRFPA盲元的影响、红外光学系统的影响、信号放大带来的非均匀性、1/f噪声、A/D转换时带来的非均匀性和外界温度等因素都是产生非均匀性的原因；红外图像具有分辨率较低、信噪比较低和对比度较差的特点，因此在使用前必须进行非均匀性校正以提高红外图像的质量。
目前，红外图像的非均匀性校正方法主要分为基于定标的算法和基于场景的算法两大类；场景类算法是利用场景信息来更新校正参数的，不需要暂停红外探测器的工作来定标，所以近年来场景类算法成为了主要的研究对象；典型的场景类算法包括时域高通滤波法、神经网络法、恒定统计法、卡尔曼滤波法以及帧间配准法。
帧间配准法收敛速度快，完全依赖于场景，具有一定的非均匀性校正效果；但是，帧间配准法在非均匀性较强的情况下校正效果不够理想。
发明内容
有鉴于此，本发明的主要目的在于提供一种基于帧间配准和自适应步长的红外图像非均匀性校正方法。
为达到上述目的，本发明的技术方案是这样实现的：
本发明实施例提供一种基于帧间配准和自适应步长的红外图像非均匀性校正方法，该方法为：建立红外图像像元的线性响应模型，并通过反变换获得校正公式；分别确定第n帧和第n-1帧带有非均匀性的原始红外图像的相对位移、以及第n帧带有非均匀性的原始红外图像的每个像元的空间方差和时间方差，并根据所述空间方差和时间方差确定第n帧带有非均匀性的原始红外图像的每个像元的自适应迭代步长；根据第n-1帧带有非均匀性的原始红外图像的误差函数和第n帧带有非均匀性的原始红外图像的第i行第j列的自适应迭代步长确定第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的增益校正系数和偏置校正系数；最后，根据所述第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的增益校正系数、偏置校正系数和校正公式对第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域的每个像元进行非均匀性校正。
上述方案中，所述根据所述第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的增益校正系数、偏置校正系数和校正公式对第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域的每个像元进行非均匀性校正，之后，该方法还包括，判断所述第n帧图像是否为带有非均匀性的原始红外图像序列的最后一帧图 像，如果是则完成非均匀性校正；如果不是则继续对后续帧图像进行非均匀性校正。
上述方案中，所述建立红外图像像元的线性响应模型，并通过反变换获得校正公式，具体通过以下步骤实现：
(101)根据下式，建立红外图像的像元的线性响应模型：
y n(i,j)＝g n(i,j)x n(i,j)+o n(i,j)
其中，g n(i,j)和o n(i,j)分别表示第n帧红外图像中第i行第j列的像元的增益系数和偏置系数，x n(i,j)表示第n帧红外图像中第i行第j列的真实输入灰度值，y n(i,j)表示第n帧红外图像中第i行第j列的包含非均匀性的输出灰度值；
(102)根据下式，通过反变换表示x n(i,j)：
x n(i,j)＝w n(i,j)y n(i,j)+b n(i,j)
其中，
Figure PCTCN2018080925-appb-000001
表示第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的增益校正系数，
Figure PCTCN2018080925-appb-000002
表示第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的偏置校正系数。
上述方案中，所述确定第n帧和第n-1帧带有非均匀性的原始红外图像的相对位移，具体通过以下步骤实现：
(201)根据下式，由第n-1帧红外图像中第i行第j列的包含非均匀性的输出灰度值y n-1(i,j)经相对位移得到第n帧红外图像第i行第j列的包含非均匀性的输出灰度值y n(i,j)：
y n(i,j)＝y n-1(i-d x,j-d y)
其中，d x和d y分别表示y n(i,j)和y n-1(i,j)的水平方向和垂直方向的相对位移；
(202)根据下式，使用傅里叶变换，计算y n(i,j)和y n-1(i,j)之间的归一化互功率谱：
Figure PCTCN2018080925-appb-000003
其中，
Figure PCTCN2018080925-appb-000004
表示归一化互功率谱，*表示复共轭，Y n(u,v)和Y n-1(u,v)分别表示y n(i,j)和y n-1(i,j)的傅里叶变换，u和v分别表示傅里叶域的坐标；
(203)根据下式，计算y n(i,j)和y n-1(i,j)的水平方向和垂直方向的相对位移：
Figure PCTCN2018080925-appb-000005
其中，FFT -1表示傅里叶逆变换，Re表示取实部操作，
Figure PCTCN2018080925-appb-000006
表示将傅里叶逆变换的结果取实部后所得到的矩阵中的最大值所在的行和列。
上述方案中，所述确定第n帧带有非均匀性的原始红外图像的每个像元的空间方差和时间方差，并根据所述空间方差和时间方差确定第n帧带有非均匀性的原始红外图像的每个像元的自适应迭代步长，具体通过以下步骤实现：
(301)确定以第n帧带有非均匀性的原始红外图像第i行第j列的像元为中心的3×3模板内的空间方差
Figure PCTCN2018080925-appb-000007
(302)根据下式，确定第i行第j列的像元从第n-m帧到第n帧带有非均匀性的原始红外图像的时间方差
Figure PCTCN2018080925-appb-000008
Figure PCTCN2018080925-appb-000009
其中，D表示方差运算，m表示小于n的正整数；
(303)根据下式，结合所述空间方差和时间方差求得第n帧带有非均匀性的原始红外图像的第i行第j列的自适应迭代步长step n(i,j)：
Figure PCTCN2018080925-appb-000010
其中，a表示固定常数。
上述方案中，所述根据第n-1帧带有非均匀性的原始红外图像的误差函数和第n帧带有非均匀性的原始红外图像的第i行第j列的自适应迭代步长确定第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的增益校正系数和偏置校正系数，具体通过以下步骤实现：
(401)根据下式，确定第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的误差函数e n(i,j)：
e n(i,j)＝(w n(i-d x,j-d y)y n-1(i-d x,j-d y)+b n(i-d x,j-d y))-(w n(i,j)y n(i,j)+b n(i,j))
同理确定第n-1帧和第n-2帧带有非均匀性的原始红外图像的重叠区域中的第i行第j列的误差函数e n-1(i,j)；
(402)根据下式，结合step n(i,j)，e n-1(i,j)和y n-1(i,j)确定w n(i,j)：
w n(i,j)＝w n-1(i,j)+step n(i,j)e n-1(i,j)y n-1(i,j)(overlapped area)
其中，w n-1(i,j)表示第n-1帧和第n-2帧带有非均匀性的原始红外 图像的重叠区域中第i行第j列的像元的增益校正系数，overlapped area表示第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域；
(403)根据下式，结合step n(i,j)和e n-1(i,j)确定b n(i,j)：
b n(i,j)＝b n-1(i,j)+step n(i,j)e n-1(i,j)(overlapped  area)
其中，b n-1(i,j)表示第n-1帧和第n-2帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的偏置校正系数。
与现有技术相比，本发明的有益效果：
本发明能够根据红外图像的空间特性和时间特性来自适应地进行调整，具有较快的收敛速度和更好的校正效果。
附图说明
图1为本发明的流程图；
图2为本发明中带有非均匀性的原始图像序列的第500帧图像；
图3为本发明中经过基于帧间配准和自适应迭代步长非均匀性校正算法校正后的第500帧图像；
图4为本发明中经过基于帧间配准和自适应迭代步长非均匀性校正算法校正后的第500帧图像与带有非均匀性的原始图像序列的第500帧图像的差值图像；
图5为本发明中粗糙度曲线图。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描 述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
本发明实施例提供一种基于帧间配准和自适应步长的红外图像非均匀性校正方法，如图1所示该方法为：
步骤1：输入原始红外图像序列的所有图像；
具体地，图2为本发明的实施例中，第500帧带有非均匀性的原始红外图像；带有非均匀性的原始图像序列共有500帧图像，并且每一帧图像大小都为320×256像素；从图2可以看出，原始图像带有明显的固定图案噪声，图像质量受到严重的影响。
步骤2：建立红外图像像元的线性响应模型，并通过反变换获得校正公式；
具体通过以下步骤实现：
(201)根据下式，建立红外图像的像元的线性响应模型：
y n(i,j)＝g n(i,j)x n(i,j)+o n(i,j)
其中，g n(i,j)和o n(i,j)分别表示第n帧红外图像中第i行第j列的像元的增益系数和偏置系数，x n(i,j)表示第n帧红外图像中第i行第j列的真实输入灰度值，y n(i,j)表示第n帧红外图像中第i行第j列的包含非均匀性的输出灰度值；
(202)根据下式，通过反变换表示x n(i,j)：
x n(i,j)＝w n(i,j)y n(i,j)+b n(i,j)
其中，
Figure PCTCN2018080925-appb-000011
表示第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的增益校正系数，
Figure PCTCN2018080925-appb-000012
表示第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的偏置校正系数。
步骤3：计算第n帧和第n-1帧带有非均匀性的原始红外图像的相对位移；
具体通过以下步骤实现：
(301)根据下式，由第n-1帧红外图像中第i行第j列的包含非均匀性的输出灰度值y n-1(i,j)经相对位移得到第n帧红外图像第i行第j列的包含非均匀性的输出灰度值y n(i,j)：
y n(i,j)＝y n-1(i-d x,j-d y)；
其中，d x和d y分别表示y n(i,j)和y n-1(i,j)的水平方向和垂直方向的相对位移；
(302)根据下式，使用傅里叶变换，计算y n(i,j)和y n-1(i,j)之间的归一化互功率谱：
Figure PCTCN2018080925-appb-000013
其中，
Figure PCTCN2018080925-appb-000014
表示归一化互功率谱，*表示复共轭，Y n(u,v)和Y n-1(u,v)分别表示图像y n(i,j)和y n-1(i,j)的傅里叶变换，u和v分别表示傅里叶域的坐标；
(303)根据下式，计算y n(i,j)和y n-1(i,j)的水平方向和垂直方向的相对位移：
Figure PCTCN2018080925-appb-000015
其中，FFT -1表示傅里叶逆变换，Re表示取实部操作，
Figure PCTCN2018080925-appb-000016
表示将傅里叶逆变换的结果取实部后所得到的矩阵中的最大值所在的行和列。
步骤4：计算第n帧带有非均匀性的原始红外图像的每个像元的空间方差和时间方差，并根据空间方差和时间方差求得第n帧带有非均匀性的原始红外图像的每个像元的自适应迭代步长；
具体通过以下步骤实现：
(401)确定以第n帧带有非均匀性的原始红外图像第i行第j列的像元为中心的3×3模板内的空间方差
Figure PCTCN2018080925-appb-000017
(402)根据下式，确定第i行第j列的像元从第n-m帧到第n帧带有非均匀性的原始红外图像的时间方差
Figure PCTCN2018080925-appb-000018
Figure PCTCN2018080925-appb-000019
其中，D表示方差运算，m表示小于n的正整数，m的值设定为10；
(403)根据下式，结合所述空间方差和时间方差求得第n帧带有非均匀性的原始红外图像的第i行第j列的自适应迭代步长step n(i,j)：
Figure PCTCN2018080925-appb-000020
其中，a表示固定常数，a的值设定为0.07。
步骤5：根据第n-1帧带有非均匀性的原始红外图像的误差函数和第n帧带有非均匀性的原始红外图像的第i行第j列的自适应迭代步长确定第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域 中的每个像元的增益校正系数和偏置校正系数；
具体通过以下步骤实现：
(501)根据下式，确定第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的误差函数e n(i,j)：
e n(i,j)＝(w n(i-d x,j-d y)y n-1(i-d x,j-d y)+b n(i-d x,j-d y))-(w n(i,j)y n(i,j)+b n(i,j))
同理确定第n-1帧和第n-2帧带有非均匀性的原始红外图像的重叠区域中的第i行第j列的误差函数e n-1(i,j)；
(502)根据下式，结合step n(i,j)，e n-1(i,j)和y n-1(i,j)确定w n(i,j)：
w n(i,j)＝w n-1(i,j)+step n(i,j)e n-1(i,j)y n-1(i,j)(overlapped area)
其中，w n-1(i,j)表示第n-1帧和第n-2帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的增益校正系数，overlapped area表示第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域；
(503)根据下式，结合step n(i,j)和e n-1(i,j)确定b n(i,j)：
b n(i,j)＝b n-1(i,j)+step n(i,j)e n-1(i,j)(overlapped  area)
其中，b n-1(i,j)表示第n-1帧和第n-2帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的偏置校正系数；第一帧带有非均匀性的原始红外图像的偏置校正系数全部设置为0，b n(i,j)＝0。
步骤6：利用第n帧带有非均匀性的原始红外图像的增益校正系数、偏置校正系数和校正公式对第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域的像元进行非均匀性校正。
具体地，图3为本发明的实施例中，带有非均匀性的原始图像序 列的第500帧图像的校正结果，从图3可以看出，校正后的图像上方和右下角的电线杆变得清晰可见，几乎已看不见条带状的固定图案噪声；图4为本发明实施例中，带有非均匀性的原始图像序列的第500帧图像的校正结果和带有非均匀性的原始图像序列的第500帧图像的差值图像，可以看出，差值图像中包含了固定图案噪声。图5为本发明实施例中，粗糙度曲线图，可以看出，与IRLMS法相比，基于帧间配准和自适应迭代步长的非均匀性校正算法得到的校正结果具有更低的粗糙度，表明基于帧间配准和自适应迭代步长的非均匀性校正算法可以更有效地降低原始红外图像的非均匀性。
本发明提出的基于帧间配准和自适应步长的红外图像序列非均匀性校正方法首先计算出相邻两帧红外图像的归一化互功率谱，再利用得到的归一化互功率谱求出相邻两帧红外图像的相对位移，再求出每个像元的空间方差和时间方差，然后利用空间方差和时间方差计算出每个像元的自适应步长，使用迭代步长更新增益校正系数和偏置校正系数，最后对相邻两帧红外图像的重叠区域进行非均匀性校正。
以上所述，仅为本发明的较佳实施例而已，并非用于限定本发明的保护范围。

Claims (6)

  1. 一种基于帧间配准和自适应步长的红外图像非均匀性校正方法，其特征在于，该方法为：建立红外图像像元的线性响应模型，并通过反变换获得校正公式；分别确定第n帧和第n-1帧带有非均匀性的原始红外图像的相对位移、以及第n帧带有非均匀性的原始红外图像的每个像元的空间方差和时间方差，并根据所述空间方差和时间方差确定第n帧带有非均匀性的原始红外图像的每个像元的自适应迭代步长；根据第n-1帧带有非均匀性的原始红外图像的误差函数和第n帧带有非均匀性的原始红外图像的第i行第j列的自适应迭代步长确定第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的增益校正系数和偏置校正系数；最后，根据所述第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的增益校正系数、偏置校正系数和校正公式对第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域的每个像元进行非均匀性校正。
  2. 根据权利要求1所述的基于帧间配准和自适应步长的红外图像非均匀性校正方法，其特征在于，所述根据所述第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的增益校正系数、偏置校正系数和校正公式对第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域的每个像元进行非均匀性校正，之后，该方法还包括，判断所述第n帧图像是否为带有非均匀性的原始红外图像序列的最后一帧图像，如果是则完成非均匀性校正；如果不是则继续对后续帧图像进行非均匀性校正。
  3. 根据权利要求1或2所述的基于帧间配准和自适应步长的红外图像非均匀性校正方法，其特征在于，所述建立红外图像像元的线性响应模型，并通过反变换获得校正公式，具体通过以下步骤实现：
    (101)根据下式，建立红外图像的像元的线性响应模型：
    y n(i,j)＝g n(i,j)x n(i,j)+o n(i,j)
    其中，g n(i,j)和o n(i,j)分别表示第n帧红外图像中第i行第j列的像元的增益系数和偏置系数，x n(i,j)表示第n帧红外图像中第i行第j列的真实输入灰度值，y n(i,j)表示第n帧红外图像中第i行第j列的包含非均匀性的输出灰度值；
    (102)根据下式，通过反变换表示x n(i,j)：
    x n(i,j)＝w n(i,j)y n(i,j)+b n(i,j)
    其中，
    Figure PCTCN2018080925-appb-100001
    表示第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的增益校正系数，
    Figure PCTCN2018080925-appb-100002
    表示第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的偏置校正系数。
  4. 根据权利要求3所述的基于帧间配准和自适应步长的红外图像非均匀性校正方法，其特征在于，所述确定第n帧和第n-1帧带有非均匀性的原始红外图像的相对位移，具体通过以下步骤实现：
    (201)根据下式，由第n-1帧红外图像中第i行第j列的包含非均匀性的输出灰度值y n-1(i,j)经相对位移得到第n帧红外图像第i行第j列的包含非均匀性的输出灰度值y n(i,j)：
    y n(i,j)＝y n-1(i-d x,j-d y)
    其中，d x和d y分别表示y n(i,j)和y n-1(i,j)的水平方向和垂直方向的相对位移；
    (202)根据下式，使用傅里叶变换，计算y n(i,j)和y n-1(i,j)之间的归一化互功率谱：
    Figure PCTCN2018080925-appb-100003
    其中，
    Figure PCTCN2018080925-appb-100004
    表示归一化互功率谱，*表示复共轭，Y n(u,v)和Y n-1(u,v)分别表示y n(i,j)和y n-1(i,j)的傅里叶变换，u和v分别表示傅里叶域的坐标；
    (203)根据下式，计算y n(i,j)和y n-1(i,j)的水平方向和垂直方向的相对位移：
    Figure PCTCN2018080925-appb-100005
    其中，FFT -1表示傅里叶逆变换，Re表示取实部操作，
    Figure PCTCN2018080925-appb-100006
    表示将傅里叶逆变换的结果取实部后所得到的矩阵中的最大值所在的行和列。
  5. 根据权利要求4所述的基于帧间配准和自适应步长的红外图像非均匀性校正方法，其特征在于，所述确定第n帧带有非均匀性的原始红外图像的每个像元的空间方差和时间方差，并根据所述空间方差和时间方差确定第n帧带有非均匀性的原始红外图像的每个像元的自适应迭代步长，具体通过以下步骤实现：
    (301)确定以第n帧带有非均匀性的原始红外图像第i行第j 列的像元为中心的3×3模板内的空间方差
    Figure PCTCN2018080925-appb-100007
    (302)根据下式，确定第i行第j列的像元从第n-m帧到第n帧带有非均匀性的原始红外图像的时间方差
    Figure PCTCN2018080925-appb-100008
    Figure PCTCN2018080925-appb-100009
    其中，D表示方差运算，m表示小于n的正整数；
    (303)根据下式，结合所述空间方差和时间方差求得第n帧带有非均匀性的原始红外图像的第i行第j列的自适应迭代步长step n(i,j)：
    Figure PCTCN2018080925-appb-100010
    其中，a表示固定常数。
  6. 根据权利要求5所述的基于帧间配准和自适应步长的红外图像非均匀性校正方法，其特征在于，所述根据第n-1帧带有非均匀性的原始红外图像的误差函数和第n帧带有非均匀性的原始红外图像的第i行第j列的自适应迭代步长确定第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的增益校正系数和偏置校正系数，具体通过以下步骤实现：
    (401)根据下式，确定第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域中的每个像元的误差函数e n(i,j)：e n(i,j)＝(w n(i-d x,j-d y)y n-1(i-d x,j-d y)+b n(i-d x,j-d y))-(w n(i,j)y n(i,j)+b n(i,j))
    同理确定第n-1帧和第n-2帧带有非均匀性的原始红外图像的重叠区域中的第i行第j列的误差函数e n-1(i,j)；
    (402)根据下式，结合step n(i,j)，e n-1(i,j)和y n-1(i,j)确定w n(i,j)：
    w n(i,j)＝w n-1(i,j)+step n(i,j)e n-1(i,j)y n-1(i,j)(overlapped area)
    其中，w n-1(i,j)表示第n-1帧和第n-2帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的增益校正系数，overlapped area表示第n帧和第n-1帧带有非均匀性的原始红外图像的重叠区域；
    (403)根据下式，结合step n(i,j)和e n-1(i,j)确定b n(i,j)：
    b n(i,j)＝b n-1(i,j)+step n(i,j)e n-1(i,j)(overlapped area)
    其中，b n-1(i,j)表示第n-1帧和第n-2帧带有非均匀性的原始红外图像的重叠区域中第i行第j列的像元的偏置校正系数。
PCT/CN2018/080925 2018-03-28 2018-03-28 基于帧间配准和自适应步长的红外图像非均匀性校正方法 WO2019183843A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US16/980,794 US11195254B2 (en) 2018-03-28 2018-03-28 Interframe registration and adaptive step size-based non-uniformity correction method for infrared image
PCT/CN2018/080925 WO2019183843A1 (zh) 2018-03-28 2018-03-28 基于帧间配准和自适应步长的红外图像非均匀性校正方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/080925 WO2019183843A1 (zh) 2018-03-28 2018-03-28 基于帧间配准和自适应步长的红外图像非均匀性校正方法

Publications (1)

Publication Number Publication Date
WO2019183843A1 true WO2019183843A1 (zh) 2019-10-03

Family

ID=68062050

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/080925 WO2019183843A1 (zh) 2018-03-28 2018-03-28 基于帧间配准和自适应步长的红外图像非均匀性校正方法

Country Status (2)

Country Link
US (1) US11195254B2 (zh)
WO (1) WO2019183843A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112945897A (zh) * 2021-01-26 2021-06-11 广东省科学院智能制造研究所 一种连续太赫兹图像非均匀性校正方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113709392B (zh) * 2021-09-02 2024-02-06 深圳市汇顶科技股份有限公司 图像传感器的校正方法、装置及电子设备
US20230192414A1 (en) * 2021-12-16 2023-06-22 Fmh Conveyors Llc Systems and methods for package size detection
CN116205825B (zh) * 2023-05-06 2023-07-21 北京师范大学 一种基于时空三维滤波的红外视频非均匀性校正方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102289788A (zh) * 2011-06-17 2011-12-21 中国电子科技集团公司第二十八研究所 多通道红外探测器中条纹非均匀性实时校正方法
CN102538973A (zh) * 2011-12-31 2012-07-04 南京理工大学 一种快速收敛的基于场景非均匀性校正方法
CN104268870A (zh) * 2014-09-24 2015-01-07 北京津同利华科技有限公司 基于小波变换直方图的短波红外焦平面非均匀性校正算法
US20170178307A1 (en) * 2015-10-19 2017-06-22 Shanghai United Imaging Healthcare Co., Ltd. System and method for image registration in medical imaging system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5471240A (en) * 1993-11-15 1995-11-28 Hughes Aircraft Company Nonuniformity correction of an imaging sensor using region-based correction terms
CN101776487B (zh) * 2009-12-31 2011-05-18 华中科技大学 一种红外焦平面非均匀性指纹提取及图像校正方法
CN103049879B (zh) * 2012-12-13 2016-06-01 中国航空工业集团公司洛阳电光设备研究所 一种基于fpga的红外图像预处理方法
CA3005280A1 (en) * 2015-11-18 2017-05-26 Lightlab Imaging, Inc. X-ray image feature detection and registration systems and methods
CN106934771B (zh) * 2017-02-16 2020-01-21 武汉镭英科技有限公司 一种基于局部相关性的红外图像条纹噪声去除方法
EP3785220A1 (en) * 2018-04-27 2021-03-03 Aselsan Elektronik Sanayi ve Ticaret Anonim Sirketi A method for confident registration-based non-uniformity correction using spatio-temporal update mask

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102289788A (zh) * 2011-06-17 2011-12-21 中国电子科技集团公司第二十八研究所 多通道红外探测器中条纹非均匀性实时校正方法
CN102538973A (zh) * 2011-12-31 2012-07-04 南京理工大学 一种快速收敛的基于场景非均匀性校正方法
CN104268870A (zh) * 2014-09-24 2015-01-07 北京津同利华科技有限公司 基于小波变换直方图的短波红外焦平面非均匀性校正算法
US20170178307A1 (en) * 2015-10-19 2017-06-22 Shanghai United Imaging Healthcare Co., Ltd. System and method for image registration in medical imaging system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112945897A (zh) * 2021-01-26 2021-06-11 广东省科学院智能制造研究所 一种连续太赫兹图像非均匀性校正方法
CN112945897B (zh) * 2021-01-26 2023-04-07 广东省科学院智能制造研究所 一种连续太赫兹图像非均匀性校正方法

Also Published As

Publication number Publication date
US20200410637A1 (en) 2020-12-31
US11195254B2 (en) 2021-12-07

Similar Documents

Publication Publication Date Title
WO2019183843A1 (zh) 基于帧间配准和自适应步长的红外图像非均匀性校正方法
CN108665425A (zh) 基于帧间配准和自适应步长的红外图像非均匀性校正方法
WO2020224518A1 (zh) 基于局部中值直方图的自适应红外图像去条纹算法
CN108846805B (zh) 一种基于场景自适应的红外热图像两点非均匀校正方法
CN102778296B (zh) 基于总变分的自适应红外焦平面非均匀性校正方法
CN107255521B (zh) 一种红外图像非均匀性校正方法及系统
CN100535618C (zh) 一种红外焦平面阵列非均匀性自适应校正方法
CN105318971B (zh) 对红外视频序列采用图像配准的自适应非均匀性校正方法
CN109741267A (zh) 基于三边滤波和神经网络的红外图像非均匀性校正方法
WO2019127059A1 (zh) 基于引导滤波和高通滤波的红外图像非均匀性校正方法
Zeng et al. Adaptive image-registration-based nonuniformity correction algorithm with ghost artifacts eliminating for infrared focal plane arrays
WO2017107343A1 (zh) 一种反差约束的气动热辐射校正方法
CN110782403A (zh) 一种红外图像非均匀性校正方法
CN111145115A (zh) 一种基于场景适应的非均匀校正方法及其硬件实现装置
CN106570829B (zh) 一种红外图像非均匀性校正方法
CN105869129A (zh) 针对非均匀校正后的热红外图像剩余非均匀噪声去除方法
CN103868601B (zh) Irfpa探测器非均匀响应的双边全变分正则化校正方法
CN104599261A (zh) 短波红外焦平面的均值直方图神经网络非均匀性校正算法
CN111932478A (zh) 一种非制冷红外焦平面的自适应非均匀校正方法
CN108225570B (zh) 一种短波红外焦平面自适应非均匀校正算法
Liu et al. A time-domain projection-based registration-scene-based nonuniformity correction technology and its detailed hardware realization
CN106780358B (zh) 基于全变分模型的irfpa非均匀性神经网络校正方法
Ren et al. Efficient single image stripe nonuniformity correction method for infrared focal plane arrays
Zhou et al. Local spatial correlation-based stripe non-uniformity correction algorithm for single infrared images
Mou et al. Single frame infrared image adaptive correction algorithm based on residual network

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18911507

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18911507

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 18911507

Country of ref document: EP

Kind code of ref document: A1