CN101241592A - High frame frequency infrared image sequence movement target real time restoration method - Google Patents

Abstract

The present invention discloses a real-time recovery method of moving-target in high frame frequency infrared image sequence. The method includes steps as follows: inputting high frame frequency infrared image frame, confirming sporting direction and distance parameter of moving-target in image frame, processing median filtering in sporting direction, confirming membership degree of seed pixel and neighborhood pixel according with given membership degree function, confirming pixel value of seed pixel according with judgement criterion, outputting image frame recoveried by moving. The method reduces noise effect by median filtering to input image sequence; has no strict request for size and moving speed of target, and has efficient in working; the median filtering, convolution operation and confirming membership degree operation of seed pixel and neighborhood pixel are processed in air space, and not need global properties of noise which makes arithmetic simply, no ringing effect and hardware realized real-time.

Description

The real time restoration method of moving target in the high frame frequency infrared image sequence

One, technical field

The invention belongs to the real time restoration technology of infrared moving target, under particularly a kind of situation at target and the relative high-speed motion of detector, moving target real time restoration method in the high frame frequency gazing type infrared imaging system.

Two, background technology

Along with the development of material and technological level, infrared detector has all obtained using widely in national defence and industry and civil area.Especially have in the scene of rapid movement at the observation dynamic scene, adopt high frame frequency gazing type infrared focal plane imaging array to become the trend of development.

One of key issue of high frame frequency gazing type infrared imaging system is a conditions of streaking.Hangover be since the detector photosensitive area in the vertical transitions process of electric charge storage region, photosensitive unit still producing optical charge, and collected by potential well and to cause, and shows as point target and is imaged as a line target.Conditions of streaking is the unavoidable problem of frame transfer planar array detector, and the speed of related movement of detector and target is big more, the detector integrates time is long more, and conditions of streaking is serious more.Another characteristics of high frame frequency imaging system are because detector integrates time weak point, generally about 1us, the optical charge of collecting is less, makes that the noise of vision signal such as photon noise, shot noise, transfer noise, dark current noise etc. are obvious relatively, and the signal to noise ratio (S/N ratio) of image is low.

The motion recovery algorithm that is applied directly in the infrared video seldom sees bibliographical information.At length introduce the restored method of moving target in traditional visible images in the article of " Beijing Institute of Technology's journal " the 10th phase in 2004 " comparative study of digital picture deblurring algorithm ", mainly contained liftering, least mean-square error rate ripple, the filtering of constraint least square side etc.The general character of these methods is it all is according to the image degradation model of being set up, and carries out suitable hypothesis or simplification, realizes certain estimation of original image.For example liftering is directly inverted according to degenrate function, and shortcoming is not consider characteristics of noise.Least mean-square error filtering is according to the power spectrum characteristic of noise, and theoretical proof is the degradation estimation function rationally, and its recovery effect is very good, but estimates that under usual conditions the power spectrum of image and noise also has difficulties, and calculates more complicated.Constraint least square filtering is a kind of optimum algorithm that restores that carries out according to the variance of noise and characteristics of mean.More than several image recovery methods be to be used for the main method that visible light, infrared image moving target restore.

The gordian technique that conventional images restores mainly contains two aspects: the estimation of degenrate function and noise statistics feature.Generally all adopt the linear movement fuzzy model for the motion existing document of degenerating, difference is the noise statistics feature difference utilized.When using these several restored methods to be used for high frame frequency infrared focal plane detector imaging moving target to restore, have following shortcoming: (1) is very short owing to the high frame frequency detector integrates time, contrast shortcoming low, that signal to noise ratio (S/N ratio) is low is more outstanding originally directly to have caused infrared image, and the recovery effect of above several technology under noise situations is bad; (2) these algorithms generally are frequency domain filterings, occur ringing effect easily, and there is periodically nicking in the image after promptly restoring; (3) the most important thing is that these algorithms generally all require the global characteristics of image or noise, operand is big, and it is real-time to be difficult to reach hardware.

Three, summary of the invention

The object of the present invention is to provide a kind ofly alleviate noise effect, applicability is strong, algorithm is simple, does not have ringing effect, and is easy to the real time restoration method of moving target in the high frame frequency infrared image sequence of hardware real-time implementation.

The technical solution that realizes the object of the invention is: the real time restoration method of moving target in a kind of high frame frequency infrared image sequence, and its step is as follows:

The first step, the input of high frame frequency Infrared video image frame;

Second goes on foot, and determines the direction of motion and the distance parameter of target in the picture frame;

The 3rd step, on direction of motion, carry out medium filtering, remove noise and detector blind element to restoring result's interference;

The 4th step, on direction of motion, carry out convolution operation, recover the target strength loss that the target hangover causes;

In the 5th step, determine sub pixel f according to kinematic parameter and definite membership function ₀Degree of membership { λ with neighborhood territory pixel _i;

The 6th goes on foot, and determines the pixel value of sub pixel according to judgment criterion;

The 7th step, the picture frame that output is restored through motion.

In the high frame frequency infrared image sequence of the present invention in the real time restoration method of moving target, medium filtering, convolution operation and definite sub pixel f ₀Degree of membership { λ with neighborhood territory pixel _iAll be to carry out along the one dimension direction of moving.

In the high frame frequency infrared image sequence of the present invention in the real time restoration method of moving target, medium filtering, convolution operation and definite sub pixel f ₀Degree of membership { λ with neighborhood territory pixel _iIt all is the spatial domain realization of image.

In the real time restoration method of moving target, (i, j t) carry out medium filtering on direction of motion to t input picture frame f constantly in the high frame frequency infrared image sequence of the present invention, the filtering formula is: and f (i, j, t)=median{f (i, j+k, t) | k=-3 ,-2 ..., 3}.

In the high frame frequency infrared image sequence of the present invention in the real time restoration method of moving target, on direction of motion, according to membership function ${\mathrm{\λ}}_i=e^{|f_i-f_0|},$ Calculate the degree of membership { λ between current seed pixel and its 8-neighborhood territory pixel _i, wherein, f _iBe current pixel f ₀The pixel value of 8-neighborhood (i=1 .., 8).

In the high frame frequency infrared image sequence of the present invention in the real time restoration method of moving target, the maximal value criterion that adopts when determining the value of sub pixel is

$f_0=\left\{\begin{array}{cc}{f}_i& {\mathrm{\&lambda;}}_i=\max \{{\mathrm{\&lambda;}}_1,...,{\mathrm{\&lambda;}}_8\}\&GreaterEqual;0.85\\ f_0& \max \{{\mathrm{\&lambda;}}_1,...,{\mathrm{\&lambda;}}_8\}<0.85\end{array}\right..$

The present invention compared with prior art has following remarkable advantage: (1) adopts medium filtering to alleviate The noise at high frame frequency infrared image noise distinct issues more to input image sequence, and the blind element of removing detector is to restoring result's influence; (2) size and the movement velocity to target do not have strict requirement, and applicability is strong; (3) to filtered image, carry out corresponding convolution operation according to the move distance parameter that obtains, recover because hangover causes the loss of target strength (gray-scale value); (4) degree of membership of medium filtering, convolution operation and definite sub pixel and neighborhood territory pixel is carried out in the spatial domain, because this process is directly finished in the spatial domain, the existing method of avoiding need arrive frequency field through fast fourier transform with image, also the result to be changed to the process of spatial domain at last from frequency and inversion, and do not require the global characteristics of noise, make algorithm simple, be easy to real-time implementation, and do not have the ringing effect of frequency field operation.

Below in conjunction with accompanying drawing the present invention is described in further detail.

Four, description of drawings

Accompanying drawing is the process flow diagram of the real time restoration method of moving target in the high frame frequency infrared image sequence of the present invention.

Five, embodiment

In conjunction with the accompanying drawings, the real time restoration method of moving target in the high frame frequency infrared image sequence of the present invention, its step is as follows:

The first step, the direction of motion and the distance parameter of target in the picture frame are determined in the input of high frame frequency Infrared video image frame;

Second step, on direction of motion, carry out medium filtering, remove noise and detector blind element to restoring result's interference.To constantly input picture frame f of t (i, j t) carry out medium filtering on direction of motion, the filtering formula is: f (i, j, t)=median{f (i, j+k, t) | k=-3 ,-2 ..., 3}.

The 3rd step, on direction of motion, carry out convolution operation, recover target strength (gray-scale value) loss that the target hangover causes;

In the 4th step, determine sub pixel f according to kinematic parameter and definite membership function ₀Degree of membership { λ with neighborhood territory pixel _i.On direction of motion, according to membership function ${\mathrm{\&lambda;}}_i=e^{|f_i-f_0|},$ Calculate the degree of membership { λ between current seed pixel and its 8-neighborhood territory pixel _i, wherein, f _iBe current pixel f ₀The pixel value of 8-neighborhood (i=1 .., 8).

Above-mentioned medium filtering, convolution operation and definite sub pixel f ₀Degree of membership { λ with neighborhood territory pixel _iAll be to carry out along the one dimension direction of moving, and all be the spatial domain realization of image.

The 5th step, determine the pixel value of sub pixel according to judgment criterion, the maximal value criterion that adopts when determining the value of sub pixel is

$f_0=\left\{\begin{array}{cc}{f}_i& {\mathrm{\&lambda;}}_i=\max \{{\mathrm{\&lambda;}}_1,...,{\mathrm{\&lambda;}}_8\}\&GreaterEqual;0.85\\ f_0& \max \{{\mathrm{\&lambda;}}_1,...,{\mathrm{\&lambda;}}_8\}<0.85\end{array}\right..$

The 6th step, the picture frame that output is restored through motion.

The real time restoration method of moving target in the high frame frequency infrared image sequence of the present invention is described with embodiment below.

A high frame frequency infrared imaging system, infrared eye pixel 320 * 240 at the uniform velocity horizontally rotates around the speed of turntable axis with 1 circle/second, and frame frequency reaches 200Hz, and the detector integrates time is 1ms.

The first step: the direction of motion of determining infrared target in this system is consistent with the line direction of detector, is horizontal direction.Because turntable rotates, and causes target relative motion distance L to be about 50 pixels;

Second step: to constantly input picture frame f of t (i, j t) carry out 1 * 7 medium filtering in the horizontal direction, and the filtering formula is:

f(i，j，t)＝median{f(i，j+k，t)|k＝-3，-2，...，3}

The image degradation that motion causes is the convolution process of degenrate function and image, adds noise.Medium filtering can effectively reduce picture noise in the protection edge, it is restorative to make image have.

The 3rd step: the distance parameter according to the first step obtains, carry out convolution operation in the horizontal direction to picture frame

$f^{\&prime;}(i,j,t)=f(i,j,t)\&CircleTimes;m\left(j\right)$

Wherein, convolution template $m\left(j\right)=\left\{\begin{array}{cc}L& j=\mathrm{1,1}+L,...,1+\mathrm{nL},j<=2*N\\ -L& j=\mathrm{2,2}+L,...,2+\mathrm{nL},j<=2*N\\ 0& \mathrm{others}\end{array}\right.,$ N is the pixel count of picture frame delegation, and L is the distance (pixel count) of target travel.

Two important parameters of degenrate function are exactly the direction and the distance of relative motion in integral time.This step operation is actually the inverse process of degeneration, and the target strength decay that makes hangover cause by convolution has obtained preliminary recovery.

The 4th step: in the horizontal direction,, calculate the degree of membership between current seed pixel and its 8-neighborhood territory pixel according to the membership function of following design.

${\mathrm{\&lambda;}}_i=e^{|f_i-f_0|}$

Wherein, f _iBe current pixel f ₀The pixel value of 8-neighborhood (i=1 .., 8).

The difference degree of 8-neighborhood territory pixel and center pixel if 8 difference degrees are smaller, represents that then the recovery that recovers by convolution operation is credible in the degree of membership sign window, if there is the pixel difference degree bigger, then needs further to handle.

The 5th step:, judge the pixel value that restores the back current pixel by following judgment criterion according to the degree of membership that the 4th step obtained.

$f_0=\left\{\begin{array}{cc}{f}_i& {\mathrm{\&lambda;}}_i=\max \{{\mathrm{\&lambda;}}_1,...,{\mathrm{\&lambda;}}_8\}\&GreaterEqual;0.85\\ f_0& \max \{{\mathrm{\&lambda;}}_1,...,{\mathrm{\&lambda;}}_8\}<0.85\end{array}\right.$

According to the value of the degree of membership of calculating, determine the value of current seed pixel according to the maximal value criterion, promptly the window center pixel upgrades according to the value of the pixel of the correspondence of degree of membership maximum.This part is the core of algorithm, because this process is directly finished in the spatial domain, the existing algorithm of avoiding need arrive frequency field through fast fourier transform with image, also the result to be changed to the process of spatial domain at last from frequency and inversion, make algorithm simple, be easy to real-time implementation, and do not have the ringing effect of frequency field operation.

The 6th step: the frame of video after output is restored through motion.

Claims (6)

1. the real time restoration method of moving target in the high frame frequency infrared image sequence, its step is as follows:

The first step, the direction of motion and the distance parameter of target in the picture frame are determined in the input of high frame frequency Infrared video image frame;

Second step, on direction of motion, carry out medium filtering, remove noise and detector blind element to restoring result's interference;

The 3rd step, on direction of motion, carry out convolution operation, recover the target strength loss that the target hangover causes;

In the 4th step, determine sub pixel f according to kinematic parameter and definite membership function ₀Degree of membership { λ with neighborhood territory pixel _i;

The 5th goes on foot, and determines the pixel value of sub pixel according to judgment criterion;

The 6th step, the picture frame that output is restored through motion.

2. the real time restoration method of moving target in the high frame frequency infrared image sequence according to claim 1 is characterized in that: medium filtering, convolution operation and definite sub pixel f ₀Degree of membership { λ with neighborhood territory pixel _iAll be to carry out along the one dimension direction of moving.

3. the real time restoration method of moving target in the high frame frequency infrared image sequence according to claim 1 is characterized in that: medium filtering, convolution operation and definite sub pixel f ₀Degree of membership { λ with neighborhood territory pixel _iIt all is the spatial domain realization of image.

4. the real time restoration method of moving target in the high frame frequency infrared image sequence according to claim 1, it is characterized in that: (i, j t) carry out medium filtering on direction of motion to t input picture frame f constantly, the filtering formula is: f (i, j, t)=median{f (i, j+k, t) | k=-3,-2 ..., 3}.

5. the real time restoration method of moving target in the high frame frequency infrared image sequence according to claim 1 is characterized in that: on direction of motion, according to membership function ${\mathrm{\&lambda;}}_i=e^{|f_i-f_0|},$ Calculate the degree of membership { λ between current seed pixel and its 8-neighborhood territory pixel _i, wherein, f _iBe current pixel f ₀The pixel value of 8-neighborhood (i=1 .., 8).

6. the real time restoration method of moving target in the high frame frequency infrared image sequence according to claim 1 is characterized in that: the maximal value criterion that adopts when determining the value of sub pixel is

$f_0=\left\{\begin{array}{cc}{f}_i& {\mathrm{\&lambda;}}_i=\max \{{\mathrm{\&lambda;}}_1,...,{\mathrm{\&lambda;}}_8\}\&GreaterEqual;0.85\\ f_0& \max \{{\mathrm{\&lambda;}}_1,...,{\mathrm{\&lambda;}}_8\}<0.85\end{array}\right..$

Images