WO2021057455A1 - 用于红外图像序列的背景运动估计方法、装置及存储介质 - Google Patents

用于红外图像序列的背景运动估计方法、装置及存储介质 Download PDF

Info

Publication number
WO2021057455A1
WO2021057455A1 PCT/CN2020/113690 CN2020113690W WO2021057455A1 WO 2021057455 A1 WO2021057455 A1 WO 2021057455A1 CN 2020113690 W CN2020113690 W CN 2020113690W WO 2021057455 A1 WO2021057455 A1 WO 2021057455A1
Authority
WO
WIPO (PCT)
Prior art keywords
sub
window
displacement
background motion
image
Prior art date
Application number
PCT/CN2020/113690
Other languages
English (en)
French (fr)
Inventor
裴继红
廖凯文
谢维信
杨烜
Original Assignee
深圳大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳大学 filed Critical 深圳大学
Priority to US17/278,972 priority Critical patent/US11669978B2/en
Publication of WO2021057455A1 publication Critical patent/WO2021057455A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/223—Analysis of motion using block-matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing

Definitions

  • This application belongs to the field of image processing technology, and in particular relates to a background motion estimation method, device and storage medium for infrared image sequences.
  • Infrared target detection technology has very important applications in infrared warning, automatic navigation and other technical fields, especially in the detection of weak and small targets in infrared images. Due to the long imaging distance of the infrared imaging system, the imaging resolution is low, and the targets often appear as several There are bright spots of several to ten pixels, and there is no obvious texture feature, and there is a lot of noise interference, which brings great difficulties to the detection of infrared dim and small targets. Therefore, when the background is relatively static, the background subtraction method is often used to detect the target, but when the camera tracks the target and causes the background to move, background motion estimation is required to perform corresponding motion compensation to keep the background relatively still. Therefore, in the detection of weak and small infrared targets with complex backgrounds, background motion estimation is one of the more important parts, which has a greater impact on the accuracy of target detection.
  • the background motion estimation method based on image block matching is to select a sub-image at a fixed position of the current frame image as a matching template, and then put the window where these templates are located in the next frame of the image sequence to search and slide, and measure the similarity Function, calculate the similarity relationship between the template and the sub-image covered by the corresponding template window, find the best matching position of the template image in the next frame of image, and calculate the displacement of the background motion. Under normal circumstances, this method can achieve better background motion estimation results. However, when the matching template image selects an area with a relatively uniform gray distribution in the image, since the image blocks in this area have a high degree of similarity with the template image, the accuracy of the background motion estimation will be greatly reduced.
  • some people select an image template for motion estimation each time, and then use templates in different regions to estimate N times, and use a statistical average of N times as the final background motion estimation result.
  • the estimated result will be inaccurate.
  • the target area due to the unpredictability of target motion, in the selected template sub-image, if the target area is included, the matching position of the template image in the next frame of image will be affected by the target motion. Influence, make the accuracy of background motion estimation decrease.
  • the background motion estimation method based on block matching
  • the template image is selected in the gray-scale flat area, or the template image contains moving targets, it will cause the background motion estimation result to be inaccurate.
  • This application provides a background motion estimation method for infrared image sequences to solve the problem of the background motion estimation method based on image block matching in the prior art when the template image is in the gray-level flat area and the template image block contains the moving target area.
  • the first aspect of the present application provides a background motion estimation method for infrared image sequences, the method includes:
  • the area to be matched is divided into T sub-areas with the same shape and size, and M groups of sub-window sets with the same shape are generated in the T sub-areas of the area to be matched by a random method, wherein each group of sub-window sets Each is composed of K sub-windows, the T, K, and M are all positive integers, and K is greater than T; each sub-window is completely located in a sub-region and each sub-region contains at least one sub-window;
  • the potential function voting method is used to determine the background motion displacement between the two adjacent frames of images in the image sequence.
  • the shape of the sub-region is a band or a rectangle
  • the shape of the sub-window is a band or a rectangle.
  • the generating M groups of sub-window sets in the T sub-regions of the region to be matched by a random method includes:
  • the parameters of the sub-area include the point coordinates of the upper-left corner of the sub-area and the height and width values of the sub-area; the point coordinates of the upper-left corner of the sub-area are based on the The upper left corner of the area is the origin, the downward direction is the x-axis, and the right direction is determined by the two-dimensional rectangular coordinate system established by the y-axis;
  • a random generation formula is used to calculate the coordinates of the upper left corner of each sub-window
  • the set of M groups of sub-windows is obtained according to the coordinates of the upper left corner of each sub-window and the height and width values of the sub-windows.
  • the random generation formula is expressed as:
  • x mk is the coordinate of the k-th sub-window in the m-th group of sub-windows on the x-axis
  • y mk is the coordinate of the k-th sub-window in the m-th group of sub-windows on the y-axis
  • Uniform(0,1) Is a function to randomly generate uniformly distributed random numbers in the interval [0,1]
  • Round(.) is a rounding function
  • x t is the coordinate of the upper left corner of the t-th sub-region in the x-axis direction
  • y t is the t-th sub-region
  • H x is the height value of the sub-area
  • H y is the width value of the sub-area
  • h x is the height value of the sub-window
  • h y is the width value of the sub-window.
  • the calculation of M possible background motion displacements of two adjacent frames of images by using a random sampling multi-window synchronization mutual information matching method according to the M sets of sub-windows includes the following steps:
  • the calculation formula for calculating the mutual information of the first image and the second image is:
  • MI(A m ,B m (dx,dy)) H(A m )+H(B m (dx,dy))-H(A m ,B m (dx,dy))
  • the first image A m, B m (dx, dy) to the second image of the first image displacement amount A m obtained by the (dx, dy) moves MI (A m, B m ( dx, dy)) is the mutual information of the first image and the second image
  • H(.) is the image entropy function
  • H(.,.) is the image joint entropy function.
  • the determining the background motion displacement between the two adjacent frames of images in the image sequence by using a potential function voting method according to the M background motion displacements includes:
  • a radial basis function is selected as the kernel function, and the potential function value of each displacement in the background motion displacement of the M two adjacent frames of image is calculated; the largest potential function value among the potential function values of each displacement The corresponding displacement is the background motion displacement between the two adjacent frames of images.
  • the formula for calculating the value of the potential function is:
  • P(dx m , dy m ) is the m-th sub-window set
  • the potential function value of the corresponding displacement dx i is the displacement in the x-axis direction of the displacement corresponding to the i-th sub-window set
  • dy i is the displacement in the y-axis direction of the displacement corresponding to the i-th sub-window set
  • a second aspect of the present application provides a background motion estimation device for infrared image sequences, the device includes:
  • the motion characteristics acquisition module is used to acquire the motion characteristics of the camera in the scene
  • a maximum displacement estimation module configured to estimate, according to the motion characteristics, the maximum possible displacement of the camera's field of view in two adjacent frames of the infrared image sequence captured by the camera;
  • a to-be-matched area determination module configured to determine a to-be-matched area of the two adjacent frames of images based on the maximum displacement, where the to-be-matched area is the same background area in the two adjacent frames of images;
  • the sub-window set generating module is used to divide the to-be-matched area into T sub-areas and randomly generate M groups of sub-window sets with the same shape among the T sub-areas of the to-be-matched area, wherein each group of sub-windows
  • the set consists of K sub-windows, the T, K, and M are all positive integers, and K is greater than T; each sub-window is completely located in a sub-region and each sub-region contains at least one sub-window;
  • a sub-window set displacement calculation module configured to calculate the M possible background motion displacements of two adjacent frames of images by using a random sampling multi-window synchronization mutual information matching method according to the M sets of sub-windows;
  • the background motion displacement calculation module is configured to determine the background motion displacement between the two adjacent frames of images in the image sequence by using the potential function voting method according to the M background motion displacements.
  • the third aspect of the present application also provides a readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed, each step described in the first aspect is executed.
  • This application provides an infrared image sequence Background motion estimation method. This method determines the area to be matched according to the maximum displacement of two adjacent frames of images in the background motion, divides the area to be matched into a certain number of sub-areas, and generates a certain number of sub-areas in these sub-areas using a random method.
  • the method of this application is based on image block matching and does not rely on the extraction of feature points. Therefore, it has high robustness for the background motion estimation of infrared image sequences with fewer feature points and interference by noise, and can effectively overcome the problem of
  • the template image selected in the block matching algorithm causes the background estimation result to be unreliable in the flat area, and the background motion estimation result is interfered by the moving target when the template image block contains the moving target area.
  • FIG. 1 is a schematic flowchart of a background motion estimation method for an infrared image sequence provided by an embodiment of the application;
  • FIG. 2 is a schematic flow chart of calculating M possible background motion displacements of two adjacent frames of images by using a random sampling multi-window synchronization mutual information matching method provided by an embodiment of the application;
  • FIG. 3 is a schematic diagram of a region to be matched of two adjacent frames of images provided by an embodiment of the application;
  • FIG. 4 shows several examples of dividing the area to be matched into T sub-areas according to an embodiment of the application
  • Fig. 5 is a schematic diagram of a two-dimensional rectangular coordinate system provided by an embodiment of the application.
  • FIG. 6 is a schematic structural diagram of a background motion estimation device for infrared image sequences provided by an embodiment of the application.
  • the first aspect of the embodiments of the present application provides a background motion estimation method for an infrared image sequence.
  • FIG. 1 it is a schematic flowchart of the background motion estimation method for an infrared image sequence provided by an embodiment of the application. include:
  • Step 101 Obtain the motion characteristics of the camera in the scene where it is located;
  • the rotation or shaking of the infrared camera causes the background image to move during imaging, and a reliable estimation of the movement amount of the imaging background image in this scene is required.
  • the estimation method provided by the embodiment of the present application first obtains the motion characteristics of the scene in which the camera is actually shooting.
  • the motion characteristics here refer to the various possible situations of the camera's displacement in the scene in which it is located within a unit time.
  • the movement characteristics include the possible movement direction and movement speed of the camera in the scene in which it is located.
  • Step 102 Estimate the maximum possible displacement of the camera's field of view in two adjacent frames of the infrared image sequence taken by the camera according to the motion characteristics;
  • the maximum possible displacement of the camera's field of view in two adjacent frames of the infrared image sequence captured by the camera is estimated.
  • the two frames of images are F 0 and F 1 , respectively, where F 0 is the previous frame of image and F 1 is the next frame of image.
  • Step 103 Use the maximum displacement to determine the regions to be matched in two adjacent frames of images, where the to-be-matched regions are the same background areas in the two adjacent frames of images;
  • the to-be-matched regions R 0 of the two frames of images F 0 and F 1 are determined.
  • FIG. A schematic diagram of the area to be matched in a frame of image.
  • F 0 is the previous frame of image
  • F 1 is the next frame of image
  • R 0 is the two adjacent frames of F 0 and F 1 to be matched according to the maximum displacement. area.
  • Step 104 Divide the to-be-matched area into T sub-areas of the same shape and size, and generate M groups of sub-window sets with the same shape in the T sub-areas of the to-be-matched area by a random method, wherein each set of sub-windows is Composed of K sub-windows, T, K, M are all positive integers, and K is greater than T; each sub-window is completely located in a sub-region and each sub-region contains at least one sub-window;
  • the region R 0 to be matched determined in step 103 is divided into T sub-regions, where the division is uniformly divided, and the T sub-regions have the same shape and size.
  • FIG. 4 there are several examples for dividing the area to be matched into T sub-areas according to the embodiments of this application. According to the example in Fig. 4, the area to be matched is evenly divided into three sub-regions R 1 , R 2 , and R 3 . A random method is used to generate M sets of sub-windows in these T sub-regions.
  • Each sub-window set is composed of K sub-windows with the same shape, each sub-window is completely located in a sub-region, and each sub-region contains at least one sub-window.
  • T, K, and M are all positive integers and K is greater than T.
  • Step 105 According to the set of M groups of sub-windows, use a random sampling multi-window synchronization mutual information matching method to calculate M background motion displacements of two adjacent frames of images;
  • a random sampling multi-window synchronization mutual information matching method is used to calculate the background motion displacement corresponding to each sub-window set, and M background motion displacements are obtained.
  • Step 106 According to the M background motion displacements, the potential function voting method is used to determine the background motion displacement between two adjacent frames of images in the image sequence.
  • the potential function voting method is used to solve the potential function value corresponding to each displacement, and the value of the potential function is used to determine the value of the two adjacent frames in the image sequence.
  • the area to be matched is determined according to the maximum displacement of two adjacent frames of images in the background motion, and the area to be matched is divided into a certain number of sub-areas, and a certain number of sub-window sets are generated in a random method in these sub-areas.
  • the information matching method calculates the background motion displacement corresponding to these sub-window sets, and then uses the potential function voting method to calculate the background motion displacement between two adjacent frames of images according to the background motion displacement corresponding to these sub-window sets.
  • the method of this application is based on image block matching and does not rely on the extraction of feature points.
  • the template image selected in the block matching algorithm causes the background estimation result to be unreliable in the flat area, and the background motion estimation result is interfered by the moving target when the template image block contains the moving target area.
  • the shape of the T sub-regions referred to in FIG. 1 is a strip or a rectangle.
  • the shape of the T sub-regions is a band shape or a rectangular shape, and these sub-regions have the same width and height.
  • the sub-windows in the randomly generated sub-window set are also band-shaped or rectangular, and these sub-windows also have the same width and height.
  • using a random method to generate M sets of sub-windows in the T sub-regions of the region to be matched includes:
  • the parameters of the sub-area include the coordinates of the upper-left corner of the sub-area and the height and width of the sub-area; the coordinates of the upper-left corner of the sub-area are based on the origin of the upper left corner of the area to be matched, and the downward direction is The x-axis, the right direction is determined by the two-dimensional rectangular coordinate system established by the y-axis;
  • the coordinates of the upper left corner of each sub-window are calculated using a random generation formula
  • the upper left corner of the area to be matched is used as the origin, the downward direction is the x-axis, and the right direction is the y-axis to establish a two-dimensional rectangular coordinate system.
  • FIG. 5 it is a schematic diagram of a two-dimensional rectangular coordinate system provided by an embodiment of this application.
  • R t is any sub-region
  • W mk is any sub-window
  • (x t , y t ) is the coordinates of the upper left corner of the sub-region R t
  • (H x , H y ) is the height and sum of the sub-region R t width.
  • (x mk , y mk ) are the coordinates of the upper left corner of the sub-window W mk
  • (h x , h y ) are the height and width of the sub-window W mk.
  • the calculation area R 0 is divided into T sub-areas of the same size with height and width H x and H y respectively: R 1 , R 2 ,...R t ,..., R T , each sub-area is determined by four parameters :
  • (x t , y t ) are the coordinates of the upper left corner of the sub-region R t
  • (H x , Hy ) are the height and width of the sub-region R t.
  • Each sub-window is determined by four parameters:
  • (x mk , y mk ) are the coordinates of the upper left corner of the sub-window W mk
  • (h x , h y ) are the height and width of the sub-window W mk
  • x mk is the coordinate of the k-th sub-window in the m-th group of sub-windows in the x-axis direction
  • y mk is the coordinate of the k-th sub-window in the m-th group of sub-windows in the y-axis direction
  • Uniform(0,1 ) Is a function to randomly generate uniformly distributed random numbers in the interval [0,1]
  • Round(.) is a rounding function
  • x t is the coordinate of the upper left corner of the t-th subregion in the x-axis direction
  • y t is the t-th subregion
  • H x is the height value of the sub-area
  • H y is the width value of the sub-area
  • h x is the height value of the sub-window
  • h y is the width value of the sub-window.
  • a random sampling multi-window synchronization mutual information matching method is used to calculate M possible background motion displacements of two adjacent frames of images.
  • FIG. 2 the random sampling multi-window synchronization mutual information provided by this embodiment of the application is shown in FIG.
  • the matching method is a schematic diagram of the process of calculating M possible background motion displacements of two adjacent frames of images, including the following steps:
  • Step 201 Obtain a rectangular neighborhood of the displacement of two adjacent frames of images in the x-axis and y-axis directions;
  • (dx, dy) is the displacement of the background motion of the image, and the rectangular neighborhood of the displacement is obtained: -D x ⁇ d x ⁇ D x , -D y ⁇ d y ⁇ D y .
  • D x and D y are two integer constants representing the size of the neighborhood.
  • Step 202 Extract the image blocks covered by all the sub-windows in the specified sub-window set on the previous frame of the two adjacent frames of images, and connect the image blocks in a specific order to generate a first image;
  • Step 203 randomly taking a displacement in the rectangular neighborhood of the displacement, and moving the designated sub-window set according to the displacement to obtain the moved sub-window set;
  • any displacement (dx, dy) in the rectangular neighborhood in step 201 is taken, and each sub-window in the moving sub-window set W m selected in step 202 is synchronously moved according to the displacement.
  • W m (dx,dy) is the set of sub-windows after W m is moved:
  • Step 204 Extract the image blocks covered by all the sub-windows in the next frame of the two adjacent frames of images in the moved sub-window set, and connect the extracted image blocks in a specific order to generate a second image;
  • the image block covered by each sub-window in the subsequent frame of image F 1 in the moved sub-window set W m (dx,dy) is extracted, and the extracted image block is calculated as B m1 ( dx,dy),...B mk (dx,dy),...,B mK (dx,dy), where B mk (dx,dy) is the sub-window W mk (dx,dy) in image F 1
  • B mk (dx,dy) is the sub-window W mk (dx,dy) in image F 1
  • Step 205 Calculate the mutual information of the first image and the second image
  • the mutual information MI (A m ,B m (dx,dy)) of the image A m and the image B m (dx,dy) is calculated:
  • MI(A m ,B m (dx,dy)) H(A m )+H(B m (dx,dy))-H(A m ,B m (dx,dy))
  • Step 206 Calculate the mutual information corresponding to all the displacements in the displacement rectangle neighborhood of the designated sub-window set, and take the displacement corresponding to the maximum mutual information as the background motion displacement corresponding to the designated sub-window set;
  • the mutual information of the extracted two images corresponding to all the displacements in the displacement rectangle neighborhood of the specified sub-window set is calculated, and the mutual information value is determined to be the largest
  • the displacement corresponding to the time is the displacement of the background motion corresponding to the sub-window set.
  • Step 207 Calculate the background motion displacements corresponding to all sub-window sets to obtain M background motion displacements of two adjacent frames of images.
  • the mutual information values of the two images extracted corresponding to all the displacements in the displacement rectangle neighborhood of each group of sub-window sets are calculated according to the method detailed in step 201 to step 206, and each group of sub-windows is obtained respectively.
  • the window set corresponds to the displacement corresponding to the maximum mutual information value, and the background motion displacement of M two adjacent frames of image is obtained, denoted as (dx 1 ,dy 1 ),...,(dx m ,dy m ),...,(dx M ,dy M ).
  • the potential function voting method is used to determine the background motion displacement between two adjacent frames of images in the image sequence, including:
  • the calculation formula of the potential function value is:
  • P(dx m , dy m ) is the m-th sub-window set
  • the potential function value of the corresponding displacement dx i is the displacement in the x-axis direction of the displacement corresponding to the i-th sub-window set
  • dy i is the displacement in the y-axis direction of the displacement corresponding to the i-th sub-window set
  • the potential function is used
  • the voting method determines the background motion displacement of F 1 relative to F 0 in the image sequence.
  • a radial basis function as the kernel function.
  • P(dx m , dy m ) is the m-th sub-window set
  • the potential function value of the corresponding displacement dx i is the displacement in the x-axis direction of the displacement corresponding to the i-th sub-window set
  • dy i is the displacement in the y-axis direction of the displacement corresponding to the i-th sub-window set
  • the displacement corresponding to the maximum value of the potential function calculated above is the background motion displacement (dx op , dy op ) of the next frame image F 1 of the infrared image sequence relative to the current frame image F 0:
  • the area to be matched is determined according to the maximum displacement of two adjacent frames of images in the background motion, and the area to be matched is divided into a certain number of sub-areas, and a certain number of sub-window sets are generated in a random method in these sub-areas.
  • the information matching method calculates the background motion displacement corresponding to these sub-window sets, and then uses the potential function voting method to calculate the background motion displacement between two adjacent frames of images according to the background motion displacement corresponding to these sub-window sets.
  • the method of the present application is based on image block matching and does not rely on the extraction of feature points. Therefore, it has high robustness for the background motion estimation of infrared image sequences with fewer feature points and interference by noise.
  • a second aspect of the present application provides a background motion estimation device for infrared image sequences.
  • a background motion estimation device for infrared image sequences provided by an embodiment of this application, the device includes:
  • the motion characteristic acquisition module 601 is used to acquire the motion characteristic of the camera in the scene
  • the maximum displacement estimation module 602 is used for estimating the maximum possible displacement of the camera's field of view in two adjacent frames of the infrared image sequence taken by the camera according to the motion characteristics;
  • the to-be-matched area determining module 603 is configured to determine a to-be-matched area of two adjacent frames of images based on the maximum displacement, and the to-be-matched area is the same background area in the two adjacent frames of images;
  • the sub-window set generating module 604 is configured to divide the area to be matched into T sub-areas and generate M groups of sub-window sets with the same shape in the T sub-areas of the area to be matched by a random method, wherein each group of sub-window sets is composed of It is composed of K sub-windows, T, K, M are all positive integers, and K is greater than T; Each sub-window is completely located in a sub-region and each sub-region contains at least one sub-window;
  • the sub-window set displacement calculation module 605 is configured to calculate M possible background motion displacements of two adjacent frames of images by using a random sampling multi-window synchronization mutual information matching method according to M sets of sub-windows;
  • the background motion displacement calculation module 606 is configured to determine the background motion displacement between two adjacent frames of images in the image sequence by using the potential function voting method according to the M background motion displacements.
  • An embodiment of the application provides a background motion estimation device for infrared image sequences, which divides the area to be matched into a certain number of sub-areas, and generates a certain number of sub-window sets in a random method in these sub-areas, and uses synchronous mutual information
  • the matching method calculates the background motion displacements corresponding to these sub-window sets, and then uses the potential function voting method to calculate the background motion displacements between two adjacent frames of images according to the background motion displacements corresponding to these sub-window sets.
  • the method of the present application is based on image block matching and does not rely on the extraction of feature points. Therefore, it has high robustness for the background motion estimation of infrared image sequences with fewer feature points and interference by noise.
  • the third aspect of the present application also provides a readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed, each step in FIG. 1 is executed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本申请涉及一种用于红外图像序列的背景运动估计方法、装置及存储介质，方法包括：根据背景运动中相邻两帧图像的最大位移量确定待匹配区域，将待匹配区域划分成T个子区域，在这些子区域中用随机方法生成M个子窗口集合，用同步互信息匹配方法计算出这些子窗口集合对应的背景运动位移量，再根据这些子窗口集合对应的背景运动位移量使用势函数投票法计算出相邻两帧图像之间的背景运动位移量。本申请方法基于图像块匹配，不依赖于特征点的提取，因此对具有特征点较少的情况，以及受噪声干扰的红外图像序列的背景运动估计具有较高的鲁棒性。

Description

用于红外图像序列的背景运动估计方法、装置及存储介质 技术领域
本申请属于图像处理技术领域，尤其涉及用于红外图像序列的背景运动估计方法、装置及存储介质。
背景技术
红外目标检测技术在红外预警、自动导航等技术领域有很重要的应用，特别是在红外图像弱小目标检测中，由于红外成像系统成像距离较远，成像的分辨率较低，目标常常表现为几个到十几个像素的亮点，且没有明显的纹理特征，同时还有大量的噪声干扰，这给红外弱小目标检测带来了极大的困难。因此，在背景相对静止的情况下，常采用背景减除法对目标进行检测，但在相机追踪目标而导致背景发生运动时，则需要进行背景运动估计，以进行相应的运动补偿，使背景保持相对静止。因此，在进行复杂背景的红外弱小目标的检测中，背景运动估计是其中较为重要的一环，对目标检测的准确性有较大的影响。
在红外远距离场景下，由红外相机镜头追踪目标的移动或抖动而导致背景发生运动时，相邻两帧图像的背景之间的运动可以近似看成是平移变换。对于这种情况，目前的背景运动估计主要分为基于图像块匹配的运动估计方法、基于特征点匹配的运动估计方法、光流法三类。
基于图像块匹配的背景运动估计方法是在当前帧图像的固定位置上选取一个子图作为匹配模板，然后将这些模板所在的窗口放到图像序列的下一帧图像中搜索滑动，通过相似性度量函数，计算模板与对应模板窗口覆盖下的子图之间的相似性关系，找到模板图像在下一帧图像中的最佳匹配位置，并以此计算出背景运动的位移量。通常情况下，这种方法可以取得较好的背景运动估计效果。但是，当匹配模板图像选择在图像中灰度分布较为均匀的区域时，由于该区域中的图像块与模板图像均具有较高的相似度，将大大降低对背景运动估计的精度。对此，有人通过每次选取一个图像模板进行运动估计，然后用不同区 域的模板估计N次，用N次的某种统计平均作为最终背景运动估计结果，但是由于背景的不可预知性，当这种方法选择的N个模板图像块多数处在灰度平坦区域时，估计的结果就会不准确。此外，在图像中存在运动目标的情况下，由于目标运动的不可预性，在选取的模板子图中，若其中包含有目标区域，模板图像在下一帧图像中的匹配位置会受到目标运动的影响，使得对背景运动估计的准确性下降。
综上，对于图像分辨率较低、存在较多灰度平坦区域、且受噪声干扰影响较大的红外图像序列的背景运动估计问题，目前已有的方法中，基于块匹配的背景运动估计方法在模板图像选在灰度平坦区域中，或模板图像中包含有运动目标时，会导致背景运动估计结果不精确。
发明内容
本申请提供一种用于红外图像序列的背景运动估计方法，用以解决现有技术中基于图像块匹配的背景运动估计方法在模板图像处于灰度平坦区域以及模板图像块中包含运动目标区域而导致背景估计结果不准确的技术问题。
本申请第一方面提供一种用于红外图像序列的背景运动估计方法，所述方法包括：
获取相机在所处场景中的运动特性；
根据所述运动特性估算所述相机拍摄的所述红外图像序列的相邻两帧图像中相机视场可能发生的最大位移量；
利用所述最大位移量确定所述相邻两帧图像的待匹配区域，所述待匹配区域为所述相邻两帧图像中的相同背景区域；
将所述待匹配区域划分为T个形状及尺寸相同的子区域，用随机方法在所述待匹配区域的T个子区域中生成M组形状相同的子窗口集合，其中，每一组子窗口集合均由K个子窗口组成，所述T、K、M均为正整数，且K大于T；每一个子窗口完整地位于一个子区域中且每个子区域至少含有一个子窗口；
根据所述M组子窗口集合，用随机采样多窗口同步互信息匹配方法计算出所述相邻两帧图像的M个背景运动位移量；
根据所述M个背景运动位移量，用势函数投票法确定图像序列中所述相邻两帧图像之间的背景运动位移量。
优选地，所述子区域形状为带状或矩形，所述子窗口形状为带状或矩形。
优选地，所述用随机方法在所述待匹配区域的T个子区域中生成M组子窗口集合包括：
获取每个子区域的参数，所述子区域的参数包括所述子区域左上角的点坐标以及所述子区域的高度及宽度值；所述子区域左上角的点坐标是基于以所述待匹配区域左上角为原点，向下方向为x轴，向右方向为y轴建立的二维直角坐标系确定的；
获取每个子窗口集合中子窗口的高度及宽度值；
根据每个子窗口的高度及宽度值以及所述子窗口所处子区域的参数使用随机生成公式计算得到每个子窗口的左上角坐标；
根据每个子窗口的左上角坐标及子窗口的高度及宽度值得到所述M组子窗口集合。
优选地，所述随机生成公式表示为：
x mk＝Round(x t+Uniform(0,1)*(H x-h x))
y mk＝Round(y t+Uniform(0,1)*(H y-h y))
式中x mk为第m组子窗口集合中第k个子窗口在x轴上的坐标，y mk为第m组子窗口集合中第k个子窗口在y轴上的坐标，Uniform(0,1)是随机生成区间[0,1]中均匀分布随机数的函数，Round(.)是取整函数，x t为第t个子区域的左上角在x轴方向的坐标，y t为第t个子区域的左上角在y轴方向的坐标，H x为子区域的高度值，H y为子区域的宽度值，h x为子窗口的高度值，h y为子窗口的宽度值。
优选地，所述根据所述M组子窗口集合，用随机采样多窗口同步互信息匹配方法计算出M个可能的相邻两帧图像背景运动位移量，包括以下步骤：
S1：获取所述相邻两帧图像在已建立的二维直角坐标系的x轴与y轴方向上位移量的矩形邻域；
S2：提取指定子窗口集合中所有子窗口在所述相邻两帧图像中前一帧图像 上覆盖的图像块，将所述图像块按预设顺序连接生成第一图像；
S3：随机取所述位移量的矩形邻域中的一位移量，按照所述位移量移动所述指定子窗口集合，得到移动后的子窗口集合；
S4：提取所述移动后的子窗口集合中所有子窗口在所述相邻两帧图像中后一帧图像上覆盖的图像块，将提取的图像块按所述预设顺序连接生成第二图像；
S5：计算所述第一图像和所述第二图像的互信息；
S6：按步骤S2至S5计算所述指定子窗口集合在所述位移量矩形邻域内所有位移量对应的互信息，取互信息最大时对应的位移量为所述指定子窗口集合对应的背景运动位移量；
S7：按步骤S2至S6计算所有子窗口集合对应的背景运动位移量，得到所述M个相邻两帧图像背景运动位移量。
优选地，所述计算所述第一图像和所述第二图像的互信息的计算公式为：
MI(A m,B m(dx,dy))＝H(A m)+H(B m(dx,dy))-H(A m,B m(dx,dy))
式中A m为所述第一图像，B m(dx,dy)为第一图像A m按位移量(dx,dy)移动后得到的所述第二图像，MI(A m,B m(dx,dy))为所述第一图像和所述第二图像的互信息，H(.)为图像熵函数，H(.,.)为图像联合熵函数。
优选地，所述根据所述M个背景运动位移量，用势函数投票法确定图像序列中所述相邻两帧图像之间的背景运动位移量，包括：
选取一个径向基函数作为核函数，计算所述M个相邻两帧图像背景运动位移量中每个位移量的势函数值；所述每个位移量的势函数值中最大的势函数值对应的位移量即为所述相邻两帧图像之间的背景运动位移量。
优选地，所述势函数值的计算公式为：
Figure PCTCN2020113690-appb-000001
式中(dx m,dy m)为第m个子窗口集合对应的位移量，总计M个子窗口集合，m＝1,2,…,M；P(dx m,dy m)为第m个子窗口集合对应的位移量的势函数值，dx i为第i个子窗口集合对应的位移量在x轴方向的位移量，dy i为第i个子窗口集合对应的位移量在y轴方向的位移量，σ为所述核函数的核宽参数，σ＝1。
本申请第二方面提供一种用于红外图像序列的背景运动估计装置，所述装置包括：
运动特性获取模块，用于获取相机在所处场景中的运动特性；
最大位移量估算模块，用于根据所述运动特性估算所述相机拍摄的所述红外图像序列的相邻两帧图像中相机视场可能发生的最大位移量；
待匹配区域确定模块，用于利用所述最大位移量为依据确定一个所述相邻两帧图像的待匹配区域，所述待匹配区域为所述相邻两帧图像中的相同背景区域；
子窗口集合生成模块，用于将所述待匹配区域划分为T个子区域并用随机方法在所述待匹配区域的T个子区域中生成M组形状相同的子窗口集合，其中，每一组子窗口集合均由K个子窗口组成，所述T、K、M均为正整数，且K大于T；每一个子窗口完整地位于一个子区域中且每个子区域至少含有一个子窗口；
子窗口集合位移量计算模块，用于根据所述M组子窗口集合，用随机采样多窗口同步互信息匹配方法计算出所述M个可能的相邻两帧图像背景运动位移量；
背景运动位移量计算模块，用于根据所述M个背景运动位移量，用势函数投票法确定图像序列中所述相邻两帧图像之间的背景运动位移量。
本申请第三方面还提供一种可读存储介质，其上存有计算机程序，其特征在于，所述计算机程序被执行时，执行第一方面中所述的各个步骤。
从上述本申请实施例可知，本申请中的针对红外远距离成像场景下，由红外相机转动或抖动导致红外成像中背景在图像中发生移动的运动估计问题，本申请提供了一种红外图像序列的背景运动估计方法，本方法通过根据背景运动中相邻两帧图像的最大位移量确定待匹配区域，将待匹配区域划分成一定数量个子区域，在这些子区域中用随机方法生成一定数量个子窗口集合，用同步互信息匹配方法计算出这些子窗口集合对应的背景运动位移量，再根据这些子窗口集合对应的背景运动位移量使用势函数投票法计算出相邻两帧图像之间的背 景运动位移量。本申请方法基于图像块匹配，不依赖于特征点的提取，因此对具有特征点较少的情况，以及受噪声干扰的红外图像序列的背景运动估计具有较高的鲁棒性，能有效克服在块匹配算法中选择的模板图像在平坦区域内造成背景估计结果匹配结果不可靠的问题，以及模板图像块中包含运动目标区域时导致背景运动的估计结果受运动目标干扰的问题。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种用于红外图像序列的背景运动估计方法的流程示意图；
图2为本申请实施例提供的用随机采样多窗口同步互信息匹配方法计算出M个可能的相邻两帧图像背景运动位移量的流程示意图；
图3为本申请实施例提供的相邻两帧图像的待匹配区域的示意图；
图4为本申请实施例提供的将待匹配区域划分为T个子区域的几种划分示例；
图5为本申请实施例提供的二维直角坐标系的示意图；
图6为本申请实施例提供的一种用于红外图像序列的背景运动估计装置的结构示意图。
具体实施方式
为使得本申请的申请目的、特征、优点能够更加的明显和易懂，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而非全部实施例。基于本申请中的实施例，本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例第一方面提供了一种用于红外图像序列的背景运动估计方 法，如图1所示，为本申请实施例提供的用于红外图像序列的背景运动估计方法的流程示意图，方法包括：
步骤101，获取相机在所处场景中的运动特性；
在本申请实施例中，在红外远距离成像场景下，红外相机转动或抖动导致成像中背景图像发生移动，需对此场景下的成像背景图像的移动量进行一个可靠的估计。本申请实施例提供的估计方法，首先获取相机在实际拍摄时所处的场景的运动特性。这里的运动特性是指相机在单位时间内在所处场景中发生位移量的各种可能情况。或者从另一方面理解，运动特性包括相机在所处的场景中可能的运动方向及运动速度。
步骤102，根据运动特性估算相机拍摄的红外图像序列的相邻两帧图像中相机视场可能发生的最大位移量；
在本申请实施例中，根据步骤101中获取到的相机在所处环境中的运动特性，估算相机拍摄的红外图像序列的相邻两帧图像中相机视场可能发生的最大位移量。假定两帧图像分别为F 0和F 1，其中F 0为前一帧图像，F 1为后一帧图像。
步骤103，利用最大位移量确定相邻两帧图像的待匹配区域，待匹配区域为相邻两帧图像中的相同背景区域；
在本申请实施例中，根据步骤102中估算的最大位移量，确定F 0和F 1这两帧图像的待匹配区域R 0，如图3所示，为本申请实施例提供的相邻两帧图像的待匹配区域的示意图，图中F 0为前一帧图像，F 1为后一帧图像，R 0为F 0与F 1这两帧相邻图像根据最大位移量确定的一待匹配区域。
步骤104，将待匹配区域划分为T个形状及尺寸相同的子区域，用随机方法在待匹配区域的T个子区域中生成M组形状相同的子窗口集合，其中，每一组子窗口集合均由K个子窗口组成，T、K、M均为正整数，且K大于T；每个子窗口完整地位于一个子区域中且每个子区域至少含有一个子窗口；
在本申请实施例中，将步骤103中确定的待匹配区域R 0划分为T个子区域，此处划分为均匀划分，该T个子区域的形状及尺寸相同。如图4所示，为本申请 实施例提供的几种将待匹配区域划分为T个子区域的几种划分示例。按图4示例将待匹配区域均匀地划分为3个子区域R 1、R 2、R 3。采用随机方法在这T个子区域中生成M组子窗口集合。每个子窗口集合由K个形状相同的子窗口组成，每个子窗口均完整地位于一个子区域中，而且每个子区域中至少含有一个子窗口。此处的T、K、M均为正整数且K大于T。
步骤105，根据M组子窗口集合，用随机采样多窗口同步互信息匹配方法计算出相邻两帧图像的M个背景运动位移量；
在本申请实施例中，根据随机生成的M组子窗口集合，采用随机采样多窗口同步互信息匹配方法计算出每个子窗口集合对应的背景运动位移量，得到M个背景运动位移量。
步骤106，根据M个背景运动位移量，用势函数投票法确定图像序列中相邻两帧图像之间的背景运动位移量。
在本申请实施例中，根据求得的M个背景运动位移量，用势函数投票法求解每个位移量对应的势函数值，根据势函数值的大小确定图像序列中相邻两帧图像之间的背景运动位移量。
本申请通过根据背景运动中相邻两帧图像的最大位移量确定待匹配区域，将待匹配区域划分成一定数量个子区域，在这些子区域中用随机方法生成一定数量个子窗口集合，用同步互信息匹配方法计算出这些子窗口集合对应的背景运动位移量，再根据这些子窗口集合对应的背景运动位移量使用势函数投票法计算出相邻两帧图像之间的背景运动位移量。本申请方法基于图像块匹配，不依赖于特征点的提取，因此对具有特征点较少的情况，以及受噪声干扰的红外图像序列的背景运动估计具有较高的鲁棒性，能有效克服在块匹配算法中选择的模板图像在平坦区域内造成背景估计结果匹配结果不可靠的问题，以及模板图像块中包含运动目标区域时导致背景运动的估计结果受运动目标干扰的问题。
优选地，图1中所指的T个子区域的形状为带状或矩形。
在本申请实施例中，T个子区域的形状为带状或者矩形形状，这些子区域 具有相同的宽和高。随机生成的子窗口集合中的子窗口也为带状或矩形，这些子窗口也具有相同的宽和高。
进一步地，用随机方法在待匹配区域的T个子区域中生成M组子窗口集合包括：
获取每个子区域的参数，子区域的参数包括子区域左上角的点坐标以及子区域的高度及宽度值；子区域左上角的点坐标是基于以待匹配区域左上角为原点，向下方向为x轴，向右方向为y轴建立的二维直角坐标系确定的；
获取每个子窗口集合中子窗口的高度及宽度值；
获取每个子区域中子窗口的个数，并确定每个子窗口所在的子区域；
根据每个子窗口的高度及宽度值以及子窗口所处子区域的参数使用随机生成公式计算得到每个子窗口的左上角坐标；
根据每个子窗口的左上角坐标及子窗口的高度及宽度值得到M组子窗口集合。
在本申请实施例中，以待匹配区域左上角为原点，向下方向为x轴，向右方向为y轴建立二维直角坐标系。如图5所示，为本申请实施例提供的二维直角坐标系的示意图。图中R t为任一子区域，W mk为任一子窗口，(x t，y t)为子区域R t左上角的坐标，(H x，H y)为子区域R t的高和宽。(x mk,y mk)为子窗口W mk左上角的坐标，(h x,h y)为子窗口W mk高和宽。计待匹配区域R 0被划分成高和宽分别为H x和H y，大小相同的T个子区域：R 1、R 2、…R t、…、R T，每个子区域由四个参数决定：
R t：(x t,y t,H x,H y),t＝1,2,…,T,k＝1,2,…,K
其中，(x t，y t)为子区域R t左上角的坐标，(H x，H y)为子区域R t的高和宽。
随机生成的M组子窗口集合：W 1、W 2、...W m、...、W M中的任一集合W m由K个高和宽分别为h x、h y的大小相等的子窗口组成：
W m＝{W m1、W m2、...W mk、...、W mK},m＝1,2,…,M
每个子窗口由四个参数决定：
W mk：(x mk,y mk,h x,h y),m＝1,2,…,M,k＝1,2,…,K
其中，(x mk,y mk)为子窗口W mk左上角的坐标，(h x,h y)为子窗口W mk高和宽，且:
h x<H x,h y<H y
因此，确定子区域的参数、子窗口集合的个数、子窗口的高度及宽度值、每个子区域中子窗口的个数，并确定每个子窗口所在的子区域，即可通过随机生成公式计算得到每个子窗口的左上角坐标，从而得到M组子窗口集合中所有子窗口的参数，即得到M组子窗口集合。
进一步地，随机生成公式表示为：
x mk＝Round(x t+Uniform(0,1)*(H x-h x))
y mk＝Round(y t+Uniform(0,1)*(H y-h y))
式中x mk为第m组子窗口集合中第k个子窗口在x轴方向的坐标，y mk为第m组子窗口集合中第k个子窗口在y轴方向上的坐标，Uniform(0,1)是随机生成区间[0,1]中均匀分布随机数的函数，Round(.)是取整函数，x t为第t个子区域的左上角在x轴方向的坐标，y t为第t个子区域的左上角在y轴方向的坐标，H x为子区域的高度值，H y为子区域的宽度值，h x为子窗口的高度值，h y为子窗口的宽度值。
进一步地，用随机采样多窗口同步互信息匹配方法计算出M个可能的相邻两帧图像背景运动位移量，如图2所示，为本申请实施例提供的用随机采样多窗口同步互信息匹配方法计算出M个可能的相邻两帧图像背景运动位移量的流程示意图，包括以下步骤：
步骤201，获取相邻两帧图像在x轴与y轴方向上位移量的矩形邻域；
在本申请实施例中，计(dx,dy)为图像背景运动的位移量，获取位移量的矩形邻域：-D x≤d x≤D x，-D y≤d y≤D y。其中D x与D y为表示邻域大小的两个整数常数。
步骤202，提取指定子窗口集合中所有子窗口在相邻两帧图像中前一帧图像上覆盖的图像块，将图像块按特定顺序连接生成第一图像；
在本申请实施例中，选取子窗口集合W m＝{W m1,...W mk,...,W mK}，在前帧图像F 0中，W m提取的K个子图像块分别为：A m1,...A mk,...,A mK，其中A mk是子窗口W mk在图像F 0中覆盖的图像块，将这些覆盖的图像块按照一定顺序连接生成一幅图像A m＝A m1...A mk...A mK。
步骤203，随机取位移量的矩形邻域中的一位移量，按照该位移量移动指定子窗口集合，得到移动后的子窗口集合；
在本申请实施例中，取步骤201中的矩形邻域中任意位移量(dx,dy)，按照该位移量同步移动步骤202中选取的移动子窗口集合W m中的每一个子窗口。W m(dx,dy)为W m移动后的子窗口集合：
W m(dx,dy)：{W m1(dx,dy),...W mk(dx,dy),...,W mK(dx,dy)}
则W mk(dx,dy)的参数表示为(x mk+dx,y mk+dy,h x,h y)
步骤204，提取移动后的子窗口集合中所有子窗口在相邻两帧图像中后一帧图像上覆盖的图像块，将提取的图像块按特定顺序连接生成第二图像；
在本申请实施例中，对移动后的子窗口集合W m(dx,dy)中的每个子窗口在后一帧图像F 1中覆盖的图像块进行提取，计提取的图像块为B m1(dx,dy),...B mk(dx,dy),...,B mK(dx,dy)，其中B mk(dx,dy)是子窗口W mk(dx,dy)在图像F 1中覆盖的图像块。将这些提取的图像块按与步骤202相同的顺序连接生成一幅图像B m(dx,dy)＝B m1(dx,dy)...B mk(dx,dy)...B mK(dx,dy)。
步骤205，计算第一图像和第二图像的互信息；
在本申请实施例中，计算图像A m和图像B m(dx,dy)的互信息MI(A m,B m(dx,dy))：
MI(A m,B m(dx,dy))＝H(A m)+H(B m(dx,dy))-H(A m,B m(dx,dy))
其中，式中A m为第一图像，B m(dx,dy)为第一图像A m按位移量(dx,dy)移动后得到的第二图像，MI(A m,B m(dx,dy))为第一图像和第二图像的互信息，H(.)为图像熵函数，H(.,.)为图像联合熵函数。
步骤206，计算指定子窗口集合在位移量矩形邻域内所有位移量对应的互信息，取互信息最大时对应的位移量为指定子窗口集合对应的背景运动位移量；
在本申请实施例中，按照步骤201至步骤205中详述的方法，计算指定子窗口集合在位移量矩形邻域内所有的位移量对应的提取的两幅图像的互信息，确定互信息值最大时对应的位移量为该子窗口集合对应的背景运动位移量。
步骤207，计算所有子窗口集合对应的背景运动位移量，得到M个相邻两帧图像背景运动位移量。
在本申请实施例中，按照步骤201至步骤206中详述的方法分别计算每组子窗口集合在位移量矩形邻域内所有位移量对应提取的两幅图像的互信息值，分别得到每组子窗口集合对应最大互信息值时对应的位移量，得到M个相邻两帧图像背景运动位移量，记为(dx 1,dy 1),…,(dx m,dy m),…,(dx M,dy M)。
进一步地，根据M个背景运动位移量，用势函数投票法确定图像序列中相邻两帧图像之间的背景运动位移量，包括：
选取一个径向基函数作为核函数，计算M个相邻两帧图像背景运动位移量中每个位移量的势函数值；每个位移量的势函数值中最大的势函数值对应的位移量即为相邻两帧图像之间的背景运动位移量。
优选地，势函数值的计算公式为：
Figure PCTCN2020113690-appb-000002
式中(dx m,dy m)为第m个子窗口集合对应的位移量，总计M个子窗口集合，m＝1,2,…,M；P(dx m,dy m)为第m个子窗口集合对应的位移量的势函数值，dx i为第i个子窗口集合对应的位移量在x轴方向的位移量，dy i为第i个子窗口集合对应的位移量在y轴方向的位移量，σ为核函数的核宽参数，σ＝1。
在本申请实施例中，根据步骤207中得到的M个背景运动位移量(dx 1,dy 1),…,(dx m,dy m),…,(dx M,dy M)，用势函数投票法确定图像序列中F 1相对于F 0的背景运动位移量。选取一个径向基函数作为核函数。一般可选高斯型的核函数：
Figure PCTCN2020113690-appb-000003
其中□.□ 2是欧氏范数运算,σ是核函数的核宽参数，通常可取σ＝1。计算每一对位移量(dx m,dy m)的势函数值：
Figure PCTCN2020113690-appb-000004
式中(dx m,dy m)为第m个子窗口集合对应的位移量，总计M个子窗口集合， m＝1,2,…,M；P(dx m,dy m)为第m个子窗口集合对应的位移量的势函数值，dx i为第i个子窗口集合对应的位移量在x轴方向的位移量，dy i为第i个子窗口集合对应的位移量在y轴方向的位移量，σ为核函数的核宽参数，σ＝1。
以上计算得到的势函数最大值对应的位移量就是红外图像序列的下一帧图像F 1对于当前帧图像F 0的背景运动位移量(dx op,dy op)：
Figure PCTCN2020113690-appb-000005
式中(dx op,dy op)为红外图像序列的下一帧图像F 1对于当前帧图像F 0的背景运动位移量，
Figure PCTCN2020113690-appb-000006
为势函数P(dx m,dy m)最大时对应的位移量(dx m,dy m)。
本申请通过根据背景运动中相邻两帧图像的最大位移量确定待匹配区域，将待匹配区域划分成一定数量个子区域，在这些子区域中用随机方法生成一定数量个子窗口集合，用同步互信息匹配方法计算出这些子窗口集合对应的背景运动位移量，再根据这些子窗口集合对应的背景运动位移量使用势函数投票法计算出相邻两帧图像之间的背景运动位移量。本申请方法基于图像块匹配，不依赖于特征点的提取，因此对具有特征点较少的情况，以及受噪声干扰的红外图像序列的背景运动估计具有较高的鲁棒性。
本申请第二方面提供一种用于红外图像序列的背景运动估计装置，如图6所示，为本申请实施例提供的一种用于红外图像序列的背景运动估计装置，装置包括：
运动特性获取模块601，用于获取相机在所处场景中的运动特性；
最大位移量估算模块602，用于根据运动特性估算相机拍摄的红外图像序列的相邻两帧图像中相机视场可能发生的最大位移量；
待匹配区域确定模块603，用于利用最大位移量为依据确定一个相邻两帧图像的待匹配区域，待匹配区域为相邻两帧图像中的相同背景区域；
子窗口集合生成模块604，用于将待匹配区域划分为T个子区域并用随机方法在待匹配区域的T个子区域中生成M组形状相同的子窗口集合，其中，每一组子窗口集合均由K个子窗口组成，T、K、M均为正整数，且K大于T； 每一个子窗口完整地位于一个子区域中且每个子区域至少含有一个子窗口；
子窗口集合位移量计算模块605，用于根据M组子窗口集合，用随机采样多窗口同步互信息匹配方法计算出M个可能的相邻两帧图像背景运动位移量；
背景运动位移量计算模块606，用于根据M个背景运动位移量，用势函数投票法确定图像序列中相邻两帧图像之间的背景运动位移量。
在本申请实施例中，可以理解的是，图6装置的各个模块的实现过程与图1中的各个步骤相同，此处不再赘述。
本申请实施例提供的一种用于红外图像序列的背景运动估计装置，通过将待匹配区域划分成一定数量个子区域，在这些子区域中用随机方法生成一定数量个子窗口集合，用同步互信息匹配方法计算出这些子窗口集合对应的背景运动位移量，再根据这些子窗口集合对应的背景运动位移量使用势函数投票法计算出相邻两帧图像之间的背景运动位移量。本申请方法基于图像块匹配，不依赖于特征点的提取，因此对具有特征点较少的情况，以及受噪声干扰的红外图像序列的背景运动估计具有较高的鲁棒性。
本申请第三方面还提供一种可读存储介质，其上存有计算机程序，其特征在于，计算机程序被执行时，执行图1中的各个步骤。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其它实施例的相关描述。
以上为对本申请所提供的技术方案的描述，对于本领域的技术人员，依据本申请实施例的思想，在具体实施方式及应用范围上均会有改变之处，综上，本说明书内容不应理解为对本申请的限制。

Claims (10)

  1. 一种用于红外图像序列的背景运动估计方法，其特征在于，所述方法包括：
    获取相机在所处场景中的运动特性；
    根据所述运动特性估算所述相机拍摄的所述红外图像序列的相邻两帧图像中相机视场可能发生的最大位移量；
    利用所述最大位移量确定所述相邻两帧图像的待匹配区域，所述待匹配区域为所述相邻两帧图像中的相同背景区域；
    将所述待匹配区域划分为T个形状及尺寸相同的子区域，用随机方法在所述待匹配区域的T个子区域中生成M组形状相同的子窗口集合，其中，每一组子窗口集合均由K个子窗口组成，所述T、K、M均为正整数，且K大于T；每一个子窗口完整地位于一个子区域中且每个子区域至少含有一个子窗口；
    根据所述M组子窗口集合，用随机采样多窗口同步互信息匹配方法计算出所述相邻两帧图像的M个背景运动位移量；
    根据所述M个背景运动位移量，用势函数投票法确定图像序列中所述相邻两帧图像之间的背景运动位移量。
  2. 根据权利要求1所述的用于红外图像序列的背景运动估计方法，其特征在于，所述子区域形状为带状或矩形，所述子窗口形状为带状或矩形。
  3. 根据权利要求2所述的用于红外图像序列的背景运动估计方法，其特征在于，所述用随机方法在所述待匹配区域的T个子区域中生成M组子窗口集合包括：
    获取每个子区域的参数，所述子区域的参数包括所述子区域左上角的点坐标以及所述子区域的高度及宽度值；所述子区域左上角的点坐标是基于以所述待匹配区域左上角为原点，向下方向为x轴，向右方向为y轴建立的二维直角坐标系确定的；
    获取每个子窗口集合中子窗口的高度及宽度值；
    根据每个子窗口的高度及宽度值以及所述子窗口所处子区域的参数使用随 机生成公式计算得到每个子窗口的左上角坐标；
    根据每个子窗口的左上角坐标及子窗口的高度及宽度值得到所述M组子窗口集合。
  4. 根据权利要求3所述的用于红外图像序列的背景运动估计方法，其特征在于，所述随机生成公式表示为：
    x mk＝Round(x t+Uniform(0,1)*(H x-h x))
    y mk＝Round(y t+Uniform(0,1)*(H y-h y))
    式中x mk为第m组子窗口集合中第k个子窗口在x轴上的坐标，y mk为第m组子窗口集合中第k个子窗口在y轴上的坐标，Uniform(0,1)是随机生成区间[0,1]中均匀分布随机数的函数，Round(.)是取整函数，x t为第t个子区域的左上角在x轴方向的坐标，y t为第t个子区域的左上角在y轴方向的坐标，H x为子区域的高度值，H y为子区域的宽度值，h x为子窗口的高度值，h y为子窗口的宽度值。
  5. 根据权利要求1所述的用于红外图像序列的背景运动估计方法，其特征在于，所述根据所述M组子窗口集合，用随机采样多窗口同步互信息匹配方法计算出所述相邻两帧图像的M个背景运动位移量，包括以下步骤：
    S1：获取所述相邻两帧图像在已建立的二维直角坐标系的x轴与y轴方向上位移量的矩形邻域；
    S2：提取指定子窗口集合中所有子窗口在所述相邻两帧图像中前一帧图像上覆盖的图像块，将所述图像块按预设顺序连接生成第一图像；
    S3：随机取所述位移量的矩形邻域中的一位移量，按照所述位移量移动所述指定子窗口集合，得到移动后的子窗口集合；
    S4：提取所述移动后的子窗口集合中所有子窗口在所述相邻两帧图像中后一帧图像上覆盖的图像块，将提取的图像块按所述预设顺序连接生成第二图像；
    S5：计算所述第一图像和所述第二图像的互信息；
    S6：按步骤S2至S5计算所述指定子窗口集合在所述位移量矩形邻域内所有位移量对应的互信息，取互信息最大时对应的位移量为所述指定子窗口集合对应的背景运动位移量；
    S7：按步骤S2至S6计算所有子窗口集合对应的背景运动位移量，得到所述M个相邻两帧图像背景运动位移量。
  6. 根据权利要求5所述的用于红外图像序列的背景运动估计方法，其特征在于，所述计算所述第一图像和所述第二图像的互信息的计算公式为：
    MI(A m,B m(dx,dy))＝H(A m)+H(B m(dx,dy))-H(A m,B m(dx,dy))
    式中A m为所述第一图像，B m(dx,dy)为第一图像A m按位移量(dx,dy)移动后得到的所述第二图像，MI(A m,B m(dx,dy))为所述第一图像和所述第二图像的互信息，H(.)为图像熵函数，H(.,.)为图像联合熵函数。
  7. 根据权利要求5所述的用于红外图像序列的背景运动估计方法，其特征在于，所述根据所述M个背景运动位移量，用势函数投票法确定图像序列中所述相邻两帧图像之间的背景运动位移量，包括：
    选取一个径向基函数作为核函数，计算所述M个相邻两帧图像背景运动位移量中每个位移量的势函数值；所述每个位移量的势函数值中最大的势函数值对应的位移量即为所述相邻两帧图像之间的背景运动位移量。
  8. 根据权利要求7所述的用于红外图像序列的背景运动估计方法，其特征在于，所述势函数值的计算公式为：
    Figure PCTCN2020113690-appb-100001
    式中(dx m,dy m)为第m个子窗口集合对应的背景运动位移量，总计M个子窗口集合，m＝1,2,…,M；P(dx m,dy m)为第m个子窗口集合对应的位移量的势函数值，dx i为第i个子窗口集合对应的位移量在x轴方向的位移量，dy i为第i个子窗口集合对应的位移量在y轴方向的位移量，σ为所述核函数的核宽参数，σ＝1。
  9. 一种用于红外图像序列的背景运动估计装置，其特征在于，所述装置包括：
    运动特性获取模块，用于获取相机在所处场景中的运动特性；
    最大位移量估算模块，用于根据所述运动特性估算所述相机拍摄的所述红外图像序列的相邻两帧图像中相机视场可能发生的最大位移量；
    待匹配区域确定模块，用于利用所述最大位移量为依据确定一个所述相邻两帧图像的待匹配区域，所述待匹配区域为所述相邻两帧图像中的相同背景区域；
    子窗口集合生成模块，用于将所述待匹配区域划分为T个子区域并用随机方法在所述待匹配区域的T个子区域中生成M组形状相同的子窗口集合，其中，每一组子窗口集合均由K个子窗口组成，所述T、K、M均为正整数，且K大于T；每一个子窗口完整地位于一个子区域中且每个子区域至少含有一个子窗口；
    子窗口集合位移量计算模块，用于根据所述M组子窗口集合，用随机采样多窗口同步互信息匹配方法计算出所述M个可能的相邻两帧图像背景运动位移量；
    背景运动位移量计算模块，用于根据所述M个背景运动位移量，用势函数投票法确定图像序列中所述相邻两帧图像之间的背景运动位移量。
  10. 一种可读存储介质，其上存有计算机程序，其特征在于，所述计算机程序被执行时，执行权利要求1中所述的各个步骤。
PCT/CN2020/113690 2019-09-25 2020-09-07 用于红外图像序列的背景运动估计方法、装置及存储介质 WO2021057455A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/278,972 US11669978B2 (en) 2019-09-25 2020-09-07 Method and device for estimating background motion of infrared image sequences and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910910191.8A CN110689554B (zh) 2019-09-25 2019-09-25 用于红外图像序列的背景运动估计方法、装置及存储介质
CN201910910191.8 2019-09-25

Publications (1)

Publication Number Publication Date
WO2021057455A1 true WO2021057455A1 (zh) 2021-04-01

Family

ID=69110570

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/113690 WO2021057455A1 (zh) 2019-09-25 2020-09-07 用于红外图像序列的背景运动估计方法、装置及存储介质

Country Status (3)

Country Link
US (1) US11669978B2 (zh)
CN (1) CN110689554B (zh)
WO (1) WO2021057455A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110689554B (zh) * 2019-09-25 2022-04-12 深圳大学 用于红外图像序列的背景运动估计方法、装置及存储介质
CN112613568B (zh) * 2020-12-29 2022-08-05 清华大学 基于可见光及红外多光谱图像序列的目标识别方法和装置
CN114519753A (zh) * 2022-02-14 2022-05-20 上海闻泰信息技术有限公司 图像生成方法、系统、电子设备、存储介质和产品

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875419A (zh) * 2016-12-29 2017-06-20 北京理工雷科电子信息技术有限公司 基于ncc匹配帧差的弱小动目标跟踪丢失重检方法
CN107146239A (zh) * 2017-04-21 2017-09-08 武汉大学 卫星视频运动目标检测方法及系统
US20180137641A1 (en) * 2015-04-20 2018-05-17 Zte Corporation Target tracking method and device
CN110246153A (zh) * 2019-04-30 2019-09-17 安徽四创电子股份有限公司 一种基于视频监控的运动目标实时检测跟踪方法
CN110689554A (zh) * 2019-09-25 2020-01-14 深圳大学 用于红外图像序列的背景运动估计方法、装置及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7277580B2 (en) * 2001-12-12 2007-10-02 Sony Corporation Multiple thresholding for video frame segmentation
JP4340968B2 (ja) * 2004-05-07 2009-10-07 ソニー株式会社 画像処理装置および方法、記録媒体、並びにプログラム
MY188908A (en) * 2012-12-10 2022-01-13 Mimos Berhad Method for camera motion estimation with presence of moving object
CN104200461B (zh) * 2014-08-04 2017-08-25 西安电子科技大学 基于互信息图像选块和sift特征的遥感图像配准方法
CN104463911A (zh) * 2014-12-09 2015-03-25 上海新跃仪表厂 基于复杂背景估计的红外运动小目标检测方法
CN106447704B (zh) * 2016-10-13 2019-04-19 西北工业大学 基于显著区域特征和边缘度的可见光-红外图像配准方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180137641A1 (en) * 2015-04-20 2018-05-17 Zte Corporation Target tracking method and device
CN106875419A (zh) * 2016-12-29 2017-06-20 北京理工雷科电子信息技术有限公司 基于ncc匹配帧差的弱小动目标跟踪丢失重检方法
CN107146239A (zh) * 2017-04-21 2017-09-08 武汉大学 卫星视频运动目标检测方法及系统
CN110246153A (zh) * 2019-04-30 2019-09-17 安徽四创电子股份有限公司 一种基于视频监控的运动目标实时检测跟踪方法
CN110689554A (zh) * 2019-09-25 2020-01-14 深圳大学 用于红外图像序列的背景运动估计方法、装置及存储介质

Also Published As

Publication number Publication date
CN110689554B (zh) 2022-04-12
US11669978B2 (en) 2023-06-06
CN110689554A (zh) 2020-01-14
US20210358133A1 (en) 2021-11-18

Similar Documents

Publication Publication Date Title
WO2021057455A1 (zh) 用于红外图像序列的背景运动估计方法、装置及存储介质
CN108470356B (zh) 一种基于双目视觉的目标对象快速测距方法
CN107452015B (zh) 一种具有重检测机制的目标跟踪系统
US11093778B2 (en) Method and system for selecting image region that facilitates blur kernel estimation
JP2016029564A (ja) 対象検出方法及び対象検出装置
JPH0935061A (ja) 画像処理方法
CN110827321B (zh) 一种基于三维信息的多相机协同的主动目标跟踪方法
CN110516639B (zh) 一种基于视频流自然场景的人物三维位置实时计算方法
CN114926514B (zh) 一种事件图像与rgb图像的配准方法及装置
JP2008288684A (ja) 人物検出装置及びプログラム
CN108596947A (zh) 一种适用于rgb-d相机的快速目标跟踪方法
CN113409334B (zh) 一种基于质心的结构光角点检测方法
CN116883897A (zh) 一种低分辨率目标识别方法
JP3534551B2 (ja) 動き検出装置
JP2005196359A (ja) 移動物体検出装置、移動物体検出方法及び移動物体検出プログラム
CN111798506B (zh) 一种图像处理方法、终端及计算机可读存储介质
CN108280815B (zh) 一种面向监控场景结构的几何校正方法
JP2019176261A (ja) 画像処理装置
US10855918B2 (en) Image processing device, image processing method, image pickup apparatus, and program storage medium that calculates a matching degree between an estimated target of interest and tracked feature points, then selects a feature point to which tracking is continued according to the matching degree
CN113096154A (zh) 一种基于斜装深度相机的目标检测与跟踪方法及系统
KR101534776B1 (ko) 깊이 정보를 이용한 템플릿 매칭 기반 고속 얼굴 추적 방법
CN115273064B (zh) 复杂运动背景下稀疏事件点小目标分割方法
JP2019192048A (ja) 撮像装置
CN108596950A (zh) 一种基于主动漂移矫正的刚体目标跟踪方法
CN112801032B (zh) 一种用于运动目标检测的动态背景匹配方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20867653

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 30.08.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20867653

Country of ref document: EP

Kind code of ref document: A1