CN110009566A

CN110009566A - Infrared thermal imaging super-resolution instrument based on deep neural network

Info

Publication number: CN110009566A
Application number: CN201910273092.3A
Authority: CN
Inventors: 高泽华; 刘迪; 兰楚文
Original assignee: Beijing University of Posts and Telecommunications
Current assignee: Beijing University of Posts and Telecommunications
Priority date: 2019-04-04
Filing date: 2019-04-04
Publication date: 2019-07-12

Abstract

The invention proposes a kind of infrared thermal imaging super-resolution instrument based on deep neural network.This method uses deep neural network algorithm, learns the picture of low definition to picture high-definition, learns conversion rule under the premise of not changing the physical structure of infrared thermal imaging equipment, is allowed to export clearer image.The shooting effect that the present invention realizes infra-red thermal imaging system after adopting this method is apparent, speed is fast, the at low cost and module is small in size, algorithmic technique is advanced, updatability is strong.

Description

Infrared thermal imaging super-resolution instrument based on deep neural network

Technical field

The present invention relates to machine learning and infrared technique, in particular to a kind of infrared thermal imaging based on deep neural network Super-resolution instrument.

Background technique

High-precision sensor complex manufacturing technology, the yield rate of infrared thermal imaging are low, expensive, be unfavorable for it is infrared heat at As the step to high-precision development.How the popular direction of present infrared thermal imaging picture process field is effectively for thermal imaging In the infrared picture of low resolution detailed information extract, analysis and fitting, then by low pixel infrared image to high pixel The relationship of infrared image study.Currently, most of super-resolution technique is both for visible images and gray scale G- Design, But the relevant super-resolution of infrared thermal imaging is at present also in the exploratory stage.

As machine learning achieves immense success in optical correlation problem, some scholars have thought deeply about depth nerve Work of the network application in terms of image super-resolution.Using the method for deep learning, Dong is proposed based on convolutional Neural net The Image Super-resolution (Super-Resolution Convolutional Neural Network, abbreviation SRCNN) of network, Learn the optimal mapping function of low resolution to high-definition picture in extensive sample data set.In order to further enhance depth Spend network model reconstruction effect, Kim propose deep layer super-pixel network model (Very Deep Super Resolution, Abbreviation VDSR), by the way that SRCNN is carried out network depth extension (increasing to 20 layers from 3 layers), and jump connection is introduced, learned Practise the mapping relations of residual information between low resolution and high-definition picture.By to the network number of plies extend, it includes ginseng It keeps count of and greatly increases, the complex mappings so as to be fitted between low resolution and high-resolution pictures more accurately are closed System.Since complicated network model will lead to the problem of huge model parameter occupies a large amount of memory spaces, Kim is proposed based on deep Degree supervision and parameter sharing technology deep layer recursive convolution network (Deep-Recursive Convolutional Network, Abbreviation DRCN).By parameter sharing between multiple modules, greatly reduce the parameter summation of model.Tai is multiple residual by being superimposed Poor network module (ResNet) constructs the deep layer recurrence residual error network (Deep-Recursive comprising 52 layer networks Residual Network, abbreviation DRRN) structure, achieve the reconstruction effect of remote super VDSR method.In view of deep learning method Applied to the excellent performance that visible images super-resolution obtains, Choi proposes 4 layers infrared enhancing convolutional neural networks Deep learning method is applied to the super of infrared image by (Thermal Image Enhancement Network, abbreviation TEN) Resolution processing.

Summary of the invention

The invention proposes a kind of infrared thermal imaging super-resolution instrument based on deep neural network.It is infrared not changing Learn conversion rule under the premise of the physical structure of thermal imaging apparatus, is allowed to export clearer image.

Specific steps include the following aspects:

1, data acquisition and pretreatment

The data set of present invention training is to be increased after thermal infrared imager thermal imaging system is shot by the methods of cutting, overturning The latter made data set of its quantity is divided into training set, verifying collection and test set three parts after being aligned.And it is every in training set One group of picture all includes to think two kinds of images of corresponding high-low resolution.

2, noise reduction process

After image alignment, need to remove the noise on image by noise reduction module.And according to picture noise type Difference applies different denoising modes, generally there is Gaussian noise, salt-pepper noise, text noise, poisson noise etc..Wherein drop Module application of making an uproar is IFD.

3, deep neural network is built

The present invention builds deep neural network model using Pytorch frame.Specific network structure is input module- - n residual block-convolutional layer-feature summation module-up-sampling module-convolutional layer-output modules of convolutional layer.Wherein, n default value It is 16, and n can be adjusted, n is bigger, and network depth is deeper, and the effect for changing structure is better；The wherein structure of residual block are as follows: volume Lamination-ReLU function-convolutional layer-residual error factor module-feature summation module；Up-sample module results are as follows: convolutional layer-up-sampling Layer.The loss function that neural network uses is L1 loss function, and back-propagation process minimizes L1 using gradient descent algorithm Loss function is to reach the adjustment to the parameter of neuron in network.

4, training and verifying model

Being input in designed network for the data set multithreading pre-processed in advance is trained and is verified, then Constantly deep neural network is adjusted optimally by improving hyper parameter, and saves final model parameter；Wherein Hyper parameter mainly includes: learning rate, weight degradation, batch training load sample number, image size, frequency of training and the residual error factor Deng.

5, test result

Final trained network model and parameter extraction are come out, in order to avoid over-fitting, input in training and is not tested It demonstrate,proves the picture (picture of the test set separated before namely) concentrated and obtains high-resolution pictures, and high-resolution pictures are more Available clearer picture after secondary input network.Knot is judged finally by Y-PSNR and structural similarity Fruit.

6, it integrates

It is the image procossing that a set of software systems are incorporated into infrared thermal imaging instrument by trained data and model integrated In module, automatically process in real time.

This invention ensures that the picture reconstruction effect of infra-red thermal imaging system shooting is apparent, it is fast, at low cost to rebuild speed And the module is small in size, algorithmic technique is advanced, updatability is strong.

Detailed description of the invention

Fig. 1 is the propagated forward and backpropagation two parts of deep neural network, and constitutes entire nerve by this two parts Network structure.Wherein propagated forward structure by input module, convolutional layer, n residual block, feature summation module, up-sampling layer and Output module composition；It counter-propagates through and minimizes loss function using gradient descent algorithm realization.

The residual error block structure that Fig. 2 is made up of modules such as convolutional layer, activation primitive, the residual error factors feature phase Calais.

Fig. 3 is the detailed process of the entirely infrared thermal imaging super-resolution instrument based on deep neural network.

Claims

1. a kind of infrared thermal imaging super-resolution instrument based on deep neural network, it is characterized in that: in infrared thermal imaging equipment On the basis of, to acquired image using noise reduction algorithm first to thermal imaging picture noise reduction, then again by based on depth nerve The super resolution algorithm of network promotes its resolution ratio, learns conversion under the premise of not changing the physical structure of infrared thermal imaging equipment Rule is allowed to export super-resolution image, while also improving its signal-to-noise ratio.

2. the infrared thermal imaging super-resolution instrument according to claim 1 based on deep neural network, it is characterized in that: instruction The data set practiced is after infrared thermal imager is shot by the methods of alignment, cutting, overturning the increase latter made number of its quantity According to collection.

3. the increased data set of the methods of alignment cutting according to claim 2, it is characterized in that: the processes such as alignment cutting are Picture is first processed into black-and-white two color tune, i.e. binary conversion treatment, then is applied to wherein, lead to using the subtraction operation in the library Numpy It crosses and compares in picture two-by-two, find the smallest position of difference, then to image cropping processing.

4. the image after the completion of cutting is finally divided into training set, verifying collection and test set by cutting according to claim 3 Three parts.Training set, verifying collection can be used in training, can use test set in testing.Each group of number in training set According to all comprising low resolution picture and corresponding high-resolution label picture.It can be used and move in the case where data set very little Study is moved, and transfer learning is exactly the node weights of a trained network to be moved in one completely new network, Lai Tigao Resolution effect saves the time.

5. noise reduction algorithm according to claim 1, it is characterized in that: after by image alignment, need by noise reduction module come Remove the noise on image.And different denoising modes is applied according to the difference of picture noise type, generally have Gaussian noise, Salt-pepper noise, text noise, poisson noise etc..Wherein that noise reduction module application is infrared noise reduction (Infrared Denoising, abbreviation IFD), it is specifically for infrared image, the most directly effective noise reduction mode of use.

6. that the structure of deep neural network according to claim 1 is applied is IFD-ResNet, it is characterized in that: specifically Network structure is that input module-residual block-convolutional layer-feature summation module-up-sampling module-convolutional layer-of convolutional layer-n is defeated Module out.Wherein, n default value is 16, and can be adjusted to n, and n is bigger, and network depth is deeper, and the effect of structure is better.

7. residual block according to claim 6, the wherein structure of residual block are as follows: convolutional layer-ReLU function-convolutional layer-is residual Poor factor module-feature summation module；Up-sample module results are as follows: convolutional layer-up-sampling layer.The loss letter that neural network uses Number is L1 (MAE, i.e. mean absolute error) loss function, and back-propagation process minimizes L1 loss using gradient descent algorithm Function is to reach the adjustment to the parameter of neuron in network.