CN113538374B - Infrared image blurring correction method for high-speed moving object - Google Patents

Infrared image blurring correction method for high-speed moving object Download PDF

Info

Publication number
CN113538374B
CN113538374B CN202110799635.2A CN202110799635A CN113538374B CN 113538374 B CN113538374 B CN 113538374B CN 202110799635 A CN202110799635 A CN 202110799635A CN 113538374 B CN113538374 B CN 113538374B
Authority
CN
China
Prior art keywords
image
degradation
fuzzy
prior
priori
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110799635.2A
Other languages
Chinese (zh)
Other versions
CN113538374A (en
Inventor
饶鹏
陈忻
张淑媛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Institute of Technical Physics of CAS
Original Assignee
Shanghai Institute of Technical Physics of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Institute of Technical Physics of CAS filed Critical Shanghai Institute of Technical Physics of CAS
Priority to CN202110799635.2A priority Critical patent/CN113538374B/en
Publication of CN113538374A publication Critical patent/CN113538374A/en
Application granted granted Critical
Publication of CN113538374B publication Critical patent/CN113538374B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/73Deblurring; Sharpening
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/80Geometric correction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10048Infrared image

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an infrared image blurring correction method for a high-speed moving object, which comprises the following steps: synchronously acquiring a degradation image and a high-speed motion state parameter corresponding to the current degradation image; selecting a priori degradation function from a priori degradation function library according to the high-speed motion state parameters; performing feature significance region optimization on the degraded image; defining a minimization cost function of aerodynamic optical effect correction based on prior information constraint; obtaining fuzzy kernels under different scales by utilizing a pyramid principle, and constraining estimation results of each layer by using a priori fuzzy kernel to obtain an estimation fuzzy kernel under the image scale; obtaining a corrected image through a non-blind image restoration algorithm according to the estimated degradation blur kernel; and evaluating the estimation quality of the fuzzy core, and updating the prior fuzzy core library. The invention realizes intelligent correction of the aerodynamic optical distortion image based on the aerodynamic theoretical model and the priori knowledge of test data by combining the pyramid principle.

Description

Infrared image blurring correction method for high-speed moving object
Technical Field
The invention belongs to the technical field of intersection combining aerodynamic optics, aerospace science and technology and information processing, and particularly relates to an infrared image blurring correction method for a high-speed moving object.
Background
The aerodynamic optical effect is a interdisciplinary. When an aircraft carrying an optical imaging system flies in the atmosphere at a high speed, the optical hood at the head of the aircraft interacts with the atmosphere severely to form a plurality of complex flow field structures such as shock waves, expansion waves, turbulent boundary layers and the like, so that aerodynamic heat, heat radiation and optical image transmission interference effects are caused for the optical imaging detection system, and imaging blurring, deviation and shaking are caused, and the effects are called aerodynamic optical effects. The higher the speed of the aircraft is, the more serious the aerodynamic optical effect is, the more uneven the refractive index of the light transmission medium is caused by the complex flow field around the optical window, and the effects such as wave front distortion, jitter, offset and the like can be caused during light transmission, so that the imaging quality can be seriously reduced, and the distorted blurred image is received by the optical system. Therefore, it is of great value to study how to realize correction of the airborne aerodynamic degradation image quality.
Conventional digital image correction algorithms are all corrected with the degradation function known, i.e. the point spread function or its parameters are determined first, and then the image is restored using methods such as inverse filtering and wiener filtering. However, the point spread function of the complex flow field caused by the aerodynamic optical effect cannot be predicted in advance, is randomly changed, and is difficult to determine the optical transfer function, so that the existing aerodynamic optical effect correction technology for the image mainly uses the only priori knowledge to perform the blind convolution algorithm, and has the defects of poor applicability, poor self-adaption capability, difficult guarantee of stability, generally needs complex iterative calculation and takes a long time.
Disclosure of Invention
In order to overcome the problems, the invention provides an infrared image blurring correction method for a high-speed moving object.
The technical scheme adopted by the invention is as follows:
an infrared image blurring correction method facing to a high-speed moving object comprises the following steps:
step one: synchronously acquiring a degradation image and a high-speed motion state parameter corresponding to the current degradation image;
step two: according to the high-speed motion state parameters, selecting an initialization fuzzy core from a degradation function library;
step three: feature saliency region preference is given to degraded images:
step four: defining a minimization cost function of aerodynamic optical effect correction based on a priori information constraints:
step five: obtaining fuzzy kernels under different scales by utilizing a pyramid principle, and constraining estimation results of each layer by using a priori fuzzy kernel to obtain an estimation fuzzy kernel under the image scale;
step six: obtaining a corrected image through a non-blind image restoration algorithm according to the estimated degradation blur kernel;
step seven: and (3) evaluating the quality of the degraded image and the corrected image, updating the prior degradation function library when the image quality improvement value is larger than a threshold value, otherwise, increasing the iteration number of pyramid solving in the step five, and re-executing the processing procedures from the step two to the step six.
The generation method of the priori degradation function library in the second step is specifically as follows:
firstly, obtaining an aerodynamic optical effect intensity parameter sigma under typical high-speed motion state parameters by a fluid modeling and wind tunnel experiment method t
Obtaining the intensity parameter sigma by a fitting method t The fitting method comprises polynomial fitting, gaussian fitting and least square fitting;
establishing a continuous degradation function model of aerodynamic optical effects:
wherein omega i As a weight parameter, sigma t As intensity parameter, x i And y i Is an offset control parameter;
setting n groups of typical high-speed motion state parameters to input a degradation function model, obtaining n degradation functions, and forming a priori degradation function library.
The initialization fuzzy core selection method in the second step is specifically as follows:
reading high-speed motion state parameters carried by an input degraded image, including speed;
and selecting a fuzzy core corresponding to the closest state parameter from the prior fuzzy core library by a method of linear weighted solution minimization loss function or nonlinear regression.
The image feature saliency region optimization method in the third step adopts a saliency edge optimization method, and the image region with the largest local gradient and the largest local gradient is selected as the image feature saliency region by calculating the local sum of the image gradient matrix; or adopting a CRF learning framework-based saliency area automatic selection method.
The method for constructing the cost function based on the prior in the fourth step comprises the following steps: and establishing a cost function based on priori knowledge, converting the estimation problem of the fuzzy core into a solution optimization problem, and adding regularization constraint terms to the image and the fuzzy core based on sparsity characteristics.
The prior fuzzy core constraint method in the fifth step is as follows: adding constraint in the estimation process of the fuzzy core by using the prior fuzzy core; fuzzy kernel prior termThrough the L2 regularization constraint term, the sparsity of residual errors between the fuzzy core estimation and the priori fuzzy core is guaranteed, and the accuracy of the fuzzy core estimation is improved.
The non-blind image restoration method in the step six is a method based on Lucy or a method based on super Laplace prior constraint.
The quality evaluation method in the seventh step is to calculate the images before and after correction by using an image quality evaluation function, wherein the image quality evaluation function comprises PSNR, SSIM or GMG.
The method for updating the prior degradation function library in the seventh step is a linear weighting method, and the updated prior degradation function is:
PSF i '=θ 1 ·PSF i2 ·PSF new
wherein PSF i As an original a priori degradation function, PSF new For the currently estimated degenerate blur kernel, the parameter θ 1 、θ 2 Respectively the weights of the two.
The invention has the following advantages:
1. the prior information is fully utilized, compared with the traditional regularization method, the accuracy of fuzzy kernel estimation is improved, and the problems of artifacts and ringing are overcome;
2. the method can correct the aerodynamic optical effect in real time in a high-speed motion state.
Drawings
Fig. 1 is a flow chart of an infrared image blur correction method for a high-speed moving object.
Detailed Description
The invention is further described below, but is not limited to these.
Examples
In the early stage, aerodynamic optical effect intensity parameters in a typical flight state are obtained by a fluid modeling and wind tunnel experiment method; and then, a mathematical model is established to obtain a continuous mapping relation between flight parameters and degradation functions of the aircraft, a pneumatic optical effect degradation model is obtained, a group of typical flight parameter inputs are set, and a degradation function library is obtained. Synchronously acquiring a degradation image and aircraft flight parameters corresponding to the current degradation image, wherein a degradation function with the closest flight state is selected from a fuzzy kernel library according to the aircraft flight parameters corresponding to the input degradation image to be processed; meanwhile, for an input original degraded image, solving a local gradient sum, finding out the maximum element values of the gradient and the matrix, and limiting the maximum value of the region coordinate value not to exceed the image size at the same time in the x and y coordinates of the gradient and the matrix to obtain the region coordinate; rounding the coordinate values to obtain four-corner coordinates of the final salient region, and intercepting the characteristic salient region from the original degraded image according to the coordinates. For the feature significance region of the extracted blurred image, defining a minimization cost function of aerodynamic optical effect correction based on prior information constraint:
where x represents a sharp image, y represents a blurred image, k represents a point spread function PSF, and γ, α, and λ are regularized constraint intensity parameters. The first term is a likelihood term, which maintains x k similarity to the observed data y. The second and third terms are strength prior constraints, represented by l2 and l0 norms regularization terms, respectively, to preserve the sparsity of the blur kernel and the image. The fourth term is a gradient prior constraint to ensure sparsity of image gradients. The last term is the prior fuzzy constraint, and the fuzzy kernel prior term isThrough the L2 regularization constraint term, the sparsity of residual errors between the fuzzy core estimation and the priori fuzzy core is guaranteed, and the accuracy of the fuzzy core estimation is improved. In the iteration process, firstly, an image multi-resolution pyramid is established, fuzzy kernels and fuzzy images are downsampled, and x and k are alternately and iteratively updated from the topmost image x and the fuzzy kernel k with the lowest resolution. And constraining the estimation result of each layer through the prior fuzzy core to obtain an estimated fuzzy core under the image scale of the layer, restoring the fuzzy image by using the estimated fuzzy core k, upsampling the restored image to obtain a potential intermediate image x of the next layer of the multi-resolution pyramid, and iterating until the finest resolution level is obtained. And obtaining a corrected image through a super-Laplace non-blind image restoration algorithm according to the estimated degradation fuzzy core. The images before and after correction are calculated using an image quality evaluation function such as PSNR, SSIM, or GMG. When the result is improved by more than 85%, updating the prior degradation function according to a linear weighting method:
PSF i '=θ 1 ·PSF i2 ·PSF new
wherein PSF i As an original a priori degradation function, PSF new For the currently estimated degenerate blur kernel, the weight parameter θ 1 、θ 2 Set to 0.3 and 0.7, respectively. Otherwise, the iteration times are increased, and the whole correction process is carried out again.
It is pointed out that several variations and modifications can be made by a person skilled in the art without departing from the inventive concept, which fall within the scope of the invention.

Claims (5)

1. An infrared image blurring correction method for a high-speed moving object is characterized by comprising the following steps of: the method comprises the following steps:
step one: synchronously acquiring a degradation image and a high-speed motion state parameter corresponding to the current degradation image;
step two: according to the high-speed motion state parameters, selecting an initialization fuzzy core from a degradation function library;
the generation method of the priori degradation function library specifically comprises the following steps:
firstly, obtaining an aerodynamic optical effect intensity parameter sigma under typical high-speed motion state parameters by a fluid modeling and wind tunnel experiment method t
The intensity parameter sigma is obtained by adopting a fitting method of polynomial fitting, gaussian fitting and least square fitting t Continuous mapping relation with high-speed motion state parameters;
establishing a continuous degradation function model of aerodynamic optical effects:
wherein omega i As a weight parameter, sigma t As intensity parameter, x i And y i Is an offset control parameter;
setting n groups of typical high-speed motion state parameters to input a degradation function model, obtaining n degradation functions, and forming a priori degradation function library;
the initialization fuzzy core selection method specifically comprises the following steps:
reading high-speed motion state parameters carried by an input degraded image, including speed;
selecting a fuzzy core corresponding to the closest state parameter from a first-check fuzzy core library by a method of linear weighted solution minimization loss function or nonlinear regression;
step three: feature saliency region optimization is carried out on the degraded image, the image feature saliency region optimization method adopts a saliency edge optimization method, and local gradients and the largest image region are selected as image feature saliency regions by calculating local sums of image gradient matrixes; or adopting a CRF learning framework-based saliency area automatic selection method;
step four: defining a minimized cost function of aerodynamic optical effect correction based on prior information constraint, and constructing the cost function based on prior comprises the following steps: establishing a cost function based on priori knowledge, converting the estimation problem of the fuzzy core into a solution optimization problem, and adding regularization constraint terms to the image and the fuzzy core based on sparsity characteristics;
step five: obtaining fuzzy kernels under different scales by utilizing a pyramid principle, and constraining estimation results of each layer by using a priori fuzzy kernel to obtain an estimation fuzzy kernel under the image scale;
step six: obtaining a corrected image through a non-blind image restoration algorithm according to the estimated degradation blur kernel;
step seven: and (3) evaluating the quality of the degraded image and the corrected image, updating the prior degradation function library when the image quality improvement value is larger than a threshold value, otherwise, increasing the iteration number of pyramid solving in the step five, and re-executing the processing procedures from the step two to the step six.
2. The method for correcting the blurring of an infrared image for a moving object at a high speed according to claim 1, wherein: the prior fuzzy core constraint method in the fifth step is as follows: adding constraint in the estimation process of the fuzzy core by using the prior fuzzy core; fuzzy kernel prior term
3. The method for correcting the blurring of an infrared image for a moving object at a high speed according to claim 1, wherein: the non-blind image restoration method in the step six is a method based on Lucy or a method based on super Laplace prior constraint.
4. The method for correcting the blurring of an infrared image for a moving object at a high speed according to claim 1, wherein: the quality evaluation method in the step seven is to calculate the images before and after correction by using an image quality evaluation function, wherein the image quality evaluation function comprises PSNR, SSIM or GMG.
5. The method for correcting the blurring of an infrared image for a moving object at a high speed according to claim 1, wherein: the method for updating the prior degradation function library in the step seven is a linear weighting method, and the updated prior degradation function is:
PSF i '=θ 1 ·PSF i2 ·PSF new
wherein PSF i As an original a priori degradation function, PSF new For the currently estimated degenerate blur kernel, the parameter θ 1 、θ 2 Respectively the weights of the two.
CN202110799635.2A 2021-07-15 2021-07-15 Infrared image blurring correction method for high-speed moving object Active CN113538374B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110799635.2A CN113538374B (en) 2021-07-15 2021-07-15 Infrared image blurring correction method for high-speed moving object

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110799635.2A CN113538374B (en) 2021-07-15 2021-07-15 Infrared image blurring correction method for high-speed moving object

Publications (2)

Publication Number Publication Date
CN113538374A CN113538374A (en) 2021-10-22
CN113538374B true CN113538374B (en) 2023-11-28

Family

ID=78099397

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110799635.2A Active CN113538374B (en) 2021-07-15 2021-07-15 Infrared image blurring correction method for high-speed moving object

Country Status (1)

Country Link
CN (1) CN113538374B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113793284B (en) * 2021-11-17 2022-03-15 武汉工程大学 Image restoration method for nonuniform blocking of aerodynamic optical effect space-variant blurred image
CN113962908B (en) * 2021-12-21 2022-05-10 武汉工程大学 Pneumatic optical effect large-visual-field degraded image point-by-point correction restoration method and system
CN114663304B (en) * 2022-03-16 2023-09-19 中国科学院光电技术研究所 Sparse prior-based optical synthetic aperture system imaging enhancement method
CN114529481B (en) * 2022-04-21 2022-07-26 武汉工程大学 Pneumatic optical thermal radiation effect correction method, device, equipment and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101236650A (en) * 2008-01-24 2008-08-06 上海交通大学 Real time rotating blurred image restoration method based on gradient loading optimization
CN106920220A (en) * 2017-02-24 2017-07-04 西北工业大学 Based on the turbulent flow method for blindly restoring image that dark primary and alternating direction multiplier method optimize
CN107742278A (en) * 2017-10-25 2018-02-27 重庆邮电大学 With reference to L0The motion blur image blind restoration method of norm and space scale information
CN108830802A (en) * 2018-05-16 2018-11-16 浙江大学 A kind of image fuzzy core estimation method based on short exposure image gradient guiding
WO2018227882A1 (en) * 2017-06-15 2018-12-20 北京大学深圳研究生院 A priori constraint and outlier suppression based image deblurring method
CN109919871A (en) * 2019-03-05 2019-06-21 重庆大学 Fuzzy core estimation method based on image and fuzzy core mixed constraints
CN110852973A (en) * 2019-11-12 2020-02-28 华中科技大学 Nonlinear restoration method and system for pulse noise blurred image
CN112132758A (en) * 2020-08-05 2020-12-25 浙江工业大学 Image restoration method based on asymmetric optical system point spread function model

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070286514A1 (en) * 2006-06-08 2007-12-13 Michael Scott Brown Minimizing image blur in an image projected onto a display surface by a projector
JP5213670B2 (en) * 2008-01-16 2013-06-19 三洋電機株式会社 Imaging apparatus and blur correction method
JP2015115733A (en) * 2013-12-11 2015-06-22 キヤノン株式会社 Image processing method, image processor, imaging device, and image processing program

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101236650A (en) * 2008-01-24 2008-08-06 上海交通大学 Real time rotating blurred image restoration method based on gradient loading optimization
CN106920220A (en) * 2017-02-24 2017-07-04 西北工业大学 Based on the turbulent flow method for blindly restoring image that dark primary and alternating direction multiplier method optimize
WO2018227882A1 (en) * 2017-06-15 2018-12-20 北京大学深圳研究生院 A priori constraint and outlier suppression based image deblurring method
CN107742278A (en) * 2017-10-25 2018-02-27 重庆邮电大学 With reference to L0The motion blur image blind restoration method of norm and space scale information
CN108830802A (en) * 2018-05-16 2018-11-16 浙江大学 A kind of image fuzzy core estimation method based on short exposure image gradient guiding
CN109919871A (en) * 2019-03-05 2019-06-21 重庆大学 Fuzzy core estimation method based on image and fuzzy core mixed constraints
CN110852973A (en) * 2019-11-12 2020-02-28 华中科技大学 Nonlinear restoration method and system for pulse noise blurred image
CN112132758A (en) * 2020-08-05 2020-12-25 浙江工业大学 Image restoration method based on asymmetric optical system point spread function model

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
A new type of fuzzy systems using pyramid membership functions (PMFs) and approximation properties;Mingzuo Jiang et.al;《Soft Computing》;全文 *
Dynamic PET Image Reconstruction Incorporating Multiscale Superpixel Clusters;Shuangliang Cao et.al;《IEEE Access》(第09期);全文 *
基于大气湍流相关参数的图像复原方法研究;邹皓;《中国优秀硕士学位论文全文数据库(信息科技辑)》(第01期);全文 *
基于波段选择估计PSF的高光谱图像运动模糊盲校正方法;南一冰;高昆;倪国强;;红外与毫米波学报(06);全文 *
气动光学效应分析与气动模糊图像复原;杨文霞等;《光学学报》;第29卷(第02期);全文 *
混合正则化约束的湍流退化图像复原算法;张姣;李俊山;隋中山;汪晓建;;激光与红外(07);全文 *
结合仿射变换和多层B样条配准的湍流畸变图像校正;张姣;李俊山;杨亚威;;光学精密工程(03);全文 *

Also Published As

Publication number Publication date
CN113538374A (en) 2021-10-22

Similar Documents

Publication Publication Date Title
CN113538374B (en) Infrared image blurring correction method for high-speed moving object
Pan et al. Learning dual convolutional neural networks for low-level vision
CN110097519B (en) Dual-monitoring image defogging method, system, medium and device based on deep learning
CN108549892B (en) License plate image sharpening method based on convolutional neural network
CN108564549B (en) Image defogging method based on multi-scale dense connection network
CN106920220B (en) The turbulent flow method for blindly restoring image optimized based on dark primary and alternating direction multipliers method
CN110796616B (en) Turbulence degradation image recovery method based on norm constraint and self-adaptive weighted gradient
CN105590304B (en) Super-resolution image reconstruction method and device
CN111462019A (en) Image deblurring method and system based on deep neural network parameter estimation
CN105913392A (en) Degraded image overall quality improving method in complex environment
CN106339996B (en) A kind of Image Blind deblurring method based on super Laplace prior
CN114418883B (en) Blind image deblurring method based on depth priori
CN111179196B (en) Multi-resolution depth network image highlight removing method based on divide-and-conquer
CN115358953B (en) Turbulence removing method based on image registration and dynamic target fusion
CN109767404A (en) Infrared image deblurring method under a kind of salt-pepper noise
Shen et al. Implicit euler ode networks for single-image dehazing
CN114494015A (en) Image reconstruction method based on blind super-resolution network
CN112837320A (en) Remote sensing image semantic segmentation method based on parallel hole convolution
CN113160179A (en) Image deblurring method based on dynamic region convolution
CN116363168A (en) Remote sensing video target tracking method and system based on super-resolution network
CN112085674B (en) Aerial image deblurring algorithm based on neural network
CN111401209B (en) Action recognition method based on deep learning
CN113421210A (en) Surface point cloud reconstruction method based on binocular stereo vision
CN116843544A (en) Method, system and equipment for super-resolution reconstruction by introducing hypersonic flow field into convolutional neural network
CN106033595A (en) Image blind deblurring method based on local constraint

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant