CN107833193A - A kind of simple lens global image restored method based on refinement network deep learning models - Google Patents

A kind of simple lens global image restored method based on refinement network deep learning models Download PDF

Info

Publication number
CN107833193A
CN107833193A CN201711160661.0A CN201711160661A CN107833193A CN 107833193 A CN107833193 A CN 107833193A CN 201711160661 A CN201711160661 A CN 201711160661A CN 107833193 A CN107833193 A CN 107833193A
Authority
CN
China
Prior art keywords
simple lens
deep learning
image
refinement
learning models
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711160661.0A
Other languages
Chinese (zh)
Inventor
张智福
余思洋
陈捷
刘稹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changsha Full Image Technology Co Ltd
Original Assignee
Changsha Full Image Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changsha Full Image Technology Co Ltd filed Critical Changsha Full Image Technology Co Ltd
Priority to CN201711160661.0A priority Critical patent/CN107833193A/en
Publication of CN107833193A publication Critical patent/CN107833193A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The present invention discloses a kind of simple lens global image restored method based on refinement network deep learning models.The data set of simple lens is obtained first, then the refinement network deep learning models restored for global image are built, the deep learning model can directly learn the characteristic of simple lens image space change, refinement network deep learning models are trained using obtained data set, for the blurred picture of the new shooting of simple lens, directly restored image can be quickly obtained using the model.The method proposed by the present invention simple lens image larger to fog-level spatial variability, without image is carried out into piecemeal processing, refinement network deep learnings model can directly learn the characteristic of fuzzy space change, make simple lens be calculated as picture processing procedure it is simpler quick, be easy to use in practice.

Description

A kind of global figure of the simple lens based on refinement-network deep learning models As restored method
Technical field
The present invention relates to a kind of image restoration field, refers specifically to one kind and is based on refinement-network deep learning moulds The simple lens global image restored method of type.
Background technology
In recent years, unzoned lens is calculated as calculating one new research direction of photography as being increasingly becoming, with depth The rise of learning algorithm is spent, the unzoned lens based on deep learning model occurs and calculates imaging method.Deep learning method can To simulate the learning process of people's brain, the picture feature of correlation is extracted from lot of experimental data, and is trained, once model is instructed White silk finishes, then using the characteristics of image of extraction, directly can carry out identical processing to new image using model.
The front end imaging device of unzoned lens only includes a piece of, two panels or three eyeglasses, by eyeglass aberration and the shadow of aberration Ring, what the image directly shot by unzoned lens typically obscured, and the image that simple lens (only including a piece of eyeglass) obtains With high blur spatial variability, i.e., the fog-level of fog-level and image border among image differs greatly.At present It can not learn the fuzzy space variation characteristic of simple lens image, common place well for the deep learning model of unzoned lens Reason method is that simple lens blurred picture is divided into several size identical image blocks first, by the fog-level of each image block It is consistent to be approximately considered, and to each image block with after existing deep learning algorithm process, then different images block is stitched together, Obtain final restored image.But the major defect of existing method is:Need individually to handle each image block, i.e., A corresponding deep learning model will be trained to each image block, spliced again after image restoration processing, consumes Time-consuming length, operation is impetuous, and can influence image restoration precision during stitching image block to a certain extent.Therefore, design is fast Fast effective deep learning simple lens image recovery method, it is that simple lens is calculated as urgent problem.
The content of the invention
The present invention is to overcome the above situation insufficient, it is desirable to provide one kind is based on refinement-network deep learning moulds The simple lens global image restored method of type, the deep learning model designed by the present invention can effectively learn simple lens image Fuzzy space variation characteristic, even if the simple lens image changed greatly to fuzzy space, it need to only train an overall global depth Learning model is spent, realizes that the rapid image of simple lens end to end restores.
A kind of simple lens global image restored method based on refinement-network deep learning models, its feature It is, comprises the following steps:
Step 1:Picture rich in detail corresponding to simple lens and blurred picture pair are generated, obtains data set;
Picture rich in detail corresponding to simple lens and blurred picture pair are generated in the step 1, obtains the specific method of data set Comprise the following steps:
Step 1.1:Chessboard table images are displayed in full screen on computer screen, computer screen is shot with simple lens, is clapped The chessboard table images taken the photograph;
Step 1.2:One picture rich in detail is displayed in full screen on computer screen, computer screen is shot with simple lens, obtains The blurred picture of shooting;
Step 1.3:The chessboard table images shot based on step 1.1, using angular-point detection method, will be used in step 1.2 Picture rich in detail be mapped to corresponding position in the blurred picture of shooting, and by suitably cutting the simple lens precisely matched Picture rich in detail and blurred picture pair;
Step 1.4:Prepare K different pictures rich in detail, be repeated in above-mentioned steps 1.2 and step 1.3, obtain K groups essence The simple lens picture rich in detail and blurred picture pair of quasi- matching, that is, obtain data set.
Step 2:Build the refinement-network deep learning models restored for global image, the depth The characteristic of simple lens image space change can directly be learnt by practising model;
It is used for the specific knot for the refinement-network deep learning models that global image restores in the step 2 Structure includes coding Encoder parts and decoding Decoder parts, wherein coding Encoder parts include six layers of convolutional layer c1, C2, c3, c4, c5, c6, decoding Decoder parts include five sub- Internets, and each subnet network layers include four warp laminations Deconv1, deconv2, deconv3 and deconv4, each subnet network layers of decoding Decoder parts are made to call during deconvolution Encode convolutional layer characteristics of image corresponding to Encoder parts, the output result of last layer is used as next layer of input;This structure Refinement-network deep learnings model can learn the fuzzy space variation characteristic of simple lens image.
Step 3:Deep learning model is trained using obtained data set, obtains the deep learning that training is completed Model;;
Python or Caffe etc. can be selected often in the training of Encoder-Decoder deep learnings model in the step 3 See framework.
Step 4:For the blurred picture of the new shooting of simple lens, blurred picture is directly inputted to the depth trained and completed Practise model, the picture rich in detail after quickly being restored.
Beneficial effect of the present invention:Refinement-network deep learnings model can effectively learn simple lens image Fuzzy space variation characteristic, the simple lens image larger to fuzzy space intensity of variation, it need to only train an overall global depth Learning model is spent, without splitting the image into small image block, deep learning is carried out to the small image block of different fog-levels respectively Model training, the refinement-network deep learnings model designed by the present invention can make simple lens image restoration process It is simpler quick, it is easy to use in practice.
Brief description of the drawings
Fig. 1 is the flow chart of the inventive method;
Fig. 2 is the chessboard table images of shooting;
Fig. 3 is the blurred picture shot in preprocessing process;
Fig. 4 is the picture rich in detail and blurred picture pair precisely matched;
Wherein (a) represents picture rich in detail, and (b) represents blurred picture.
Fig. 5 is the block schematic illustration of refinement-network deep learning models;
Fig. 6 is the structural representation of each sub-network in Decoder parts in refinement-network deep learning models Figure;
Fig. 7 is the blurred picture that simple lens is newly shot;
Fig. 8 is the image restoration result based on refinement-network deep learning models;.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.Based on this Embodiment in invention, the every other reality that those of ordinary skill in the art are obtained under the premise of creative work is not made Example is applied, belongs to the scope of protection of the invention.
As shown in figure 1, a kind of list based on refinement-network deep learning models that the present embodiment provides is thoroughly Mirror global image restored method, comprises the following steps:
Step 1:Picture rich in detail corresponding to simple lens and blurred picture pair are generated, obtains data set.Specific method is included such as Lower step:
Step 1.1:Chessboard table images are displayed in full screen on computer screen, computer screen is shot with simple lens, is clapped The chessboard table images taken the photograph, as shown in Figure 2;
Step 1.2:One picture rich in detail is displayed in full screen on computer screen, computer screen is shot with simple lens, obtains The blurred picture of shooting, as shown in Figure 3;
Step 1.3:The chessboard table images shot based on step 1.1, using angular-point detection method, will be used in step 1.2 Picture rich in detail be mapped to corresponding position in the blurred picture of shooting, and by suitably cutting the simple lens precisely matched Picture rich in detail and blurred picture pair, as shown in Figure 4;
Step 1.4:Prepare 10000 different pictures rich in detail, be repeated in above-mentioned steps 1.2 and step 1.3, obtain 10000 groups of simple lens pictures rich in detail precisely matched and blurred picture pair, i.e. refinement-network deep learnings model Obtain data set.
Step 2:Build the refinement-network deep learning models restored for global image, the depth The characteristic of simple lens image space change can directly be learnt by practising model.Wherein refinement-network deep learnings model Concrete structure include coding Encoder parts and decoding Decoder parts, wherein coding Encoder parts include six layers of volume Lamination c1, c2, c3, c4, c5, c6, decoding Decoder parts include five sub- Internets, and each subnet network layers include four Warp lamination deconv1, deconv2, deconv3 and deconv4, each subnet network layers of decoding Decoder parts make warp Call convolutional layer characteristics of image corresponding to coding Encoder parts during product, the output result of last layer is used as next layer of input; The refinement-network deep learnings model of this structure can learn the fuzzy space variation characteristic of simple lens image.
Input picture size is 1080 × 1920, and every layer of parts of coding Encoder convolutional layer makees down-sampling, down-sampling first Rate is 0.7, and every layer of parts of decoding Decoder warp lamination first up-samples, and up-sampling rate is similarly 0.7, the figure finally exported As size is similarly 1080 × 1920, the size for making convolution kernel when convolution and deconvolution is 3 × 3.
Step 3:Deep learning model is trained using obtained data set, obtains the deep learning that training is completed Model, using the Caffe environmental training refinement-network deep learning models in Ubuntu systems, use ADAGRAD optimized algorithms are trained, and initial learning rate is 0.01, and frequency of training is 600000 times, wherein, it is in frequency of training 300000th, 400000 and 500000 when, learning rate respectively divided by 10, reduce learning rate, training complete refinement- Network deep learnings model probably needs three day time.
Step 4:For the blurred picture of the new shooting of simple lens, as shown in fig. 6, directly blurred picture input has been trained Into refinement-network deep learning models, the picture rich in detail after quickly being restored, restoration result such as Fig. 7 institutes Show.
The above disclosed power for being only a kind of preferred embodiment of the present invention, the present invention can not being limited with this certainly Sharp scope, therefore the equivalent variations made according to the claims in the present invention, still belong to the scope that the present invention is covered.

Claims (4)

1. a kind of simple lens global image restored method based on refinement-network deep learning models, its feature exist In comprising the following steps:
Step 1:Picture rich in detail corresponding to simple lens and blurred picture pair are generated, obtains data set;
Step 2:Build the refinement-network deep learning models restored for global image, the deep learning mould Type can directly learn the characteristic of simple lens image space change;
Step 3:Deep learning model is trained using obtained data set, obtains the deep learning model that training is completed;
Step 4:For the blurred picture of the new shooting of simple lens, blurred picture is directly inputted to the deep learning mould trained and completed Type, the picture rich in detail after quickly being restored.
2. the simple lens global image according to claim 1 based on refinement-network deep learning models is answered Former method, it is characterised in that:Picture rich in detail corresponding to simple lens and blurred picture pair are generated in the step 1, obtains data set Specific method comprise the following steps:
Step 1.1:Chessboard table images are displayed in full screen on computer screen, computer screen is shot with simple lens, is shot Chessboard table images;
Step 1.2:One picture rich in detail is displayed in full screen on computer screen, computer screen is shot with simple lens, is shot Blurred picture;
Step 1.3:The chessboard table images shot based on step 1.1 are clear by what is used in step 1.2 using angular-point detection method Clear image is mapped to corresponding position in the blurred picture of shooting, and by suitably cutting the clear of the simple lens precisely matched Clear image and blurred picture pair;
Step 1.4:Prepare K different pictures rich in detail, be repeated in above-mentioned steps 1.2 and step 1.3, obtain accurate of K groups The simple lens picture rich in detail matched somebody with somebody and blurred picture pair, that is, obtain data set.
3. the simple lens global image according to claim 1 based on refinement-network deep learning models is answered Former method, it is characterised in that:It is used for the refinement-network deep learning moulds that global image restores in the step 2 The concrete structure of type includes coding Encoder parts and decoding Decoder parts, wherein coding Encoder parts include six layers Convolutional layer c1, c2, c3, c4, c5, c6, decoding Decoder parts include five sub- Internets, and each subnet network layers include four Individual warp lamination deconv1, deconv2, deconv3 and deconv4, each subnet network layers of decoding Decoder parts are made instead Call convolutional layer characteristics of image corresponding to coding Encoder parts during convolution, the output result of last layer is defeated as next layer Enter;The fuzzy space change that the refinement-network deep learnings model of this structure can learn simple lens image is special Property.
4. the simple lens global image according to claim 1 based on refinement-network deep learning models is answered Former method, it is characterised in that:The common frames such as Python or Caffe can be selected in the training of deep learning model in the step 4 Frame.
CN201711160661.0A 2017-11-20 2017-11-20 A kind of simple lens global image restored method based on refinement network deep learning models Pending CN107833193A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711160661.0A CN107833193A (en) 2017-11-20 2017-11-20 A kind of simple lens global image restored method based on refinement network deep learning models

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711160661.0A CN107833193A (en) 2017-11-20 2017-11-20 A kind of simple lens global image restored method based on refinement network deep learning models

Publications (1)

Publication Number Publication Date
CN107833193A true CN107833193A (en) 2018-03-23

Family

ID=61652182

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711160661.0A Pending CN107833193A (en) 2017-11-20 2017-11-20 A kind of simple lens global image restored method based on refinement network deep learning models

Country Status (1)

Country Link
CN (1) CN107833193A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108830801A (en) * 2018-05-10 2018-11-16 湖南丹尼尔智能科技有限公司 A kind of deep learning image recovery method of automatic identification vague category identifier
CN108921806A (en) * 2018-08-07 2018-11-30 Oppo广东移动通信有限公司 A kind of image processing method, image processing apparatus and terminal device
CN109191493A (en) * 2018-07-13 2019-01-11 上海大学 A kind of method for tracking target based on RefineNet neural network and sparse optical flow
CN110084763A (en) * 2019-04-29 2019-08-02 北京达佳互联信息技术有限公司 Image repair method, device, computer equipment and storage medium
CN113436137A (en) * 2021-03-12 2021-09-24 北京世纪好未来教育科技有限公司 Image definition recognition method, device, equipment and medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103856723A (en) * 2014-02-25 2014-06-11 中国人民解放军国防科学技术大学 PSF fast calibration method based on single-lens imaging
CN104778659A (en) * 2015-04-15 2015-07-15 杭州电子科技大学 Single-frame image super-resolution reconstruction method on basis of deep learning
CN106203506A (en) * 2016-07-11 2016-12-07 上海凌科智能科技有限公司 A kind of pedestrian detection method based on degree of depth learning art
CN106447626A (en) * 2016-09-07 2017-02-22 华中科技大学 Blurred kernel dimension estimation method and system based on deep learning
CN106780378A (en) * 2016-12-08 2017-05-31 中国人民解放军国防科学技术大学 A kind of blind convolved image restored method that two lenses lens have been corrected for aberration
CN106910175A (en) * 2017-02-28 2017-06-30 武汉大学 A kind of single image defogging algorithm based on deep learning
CN107146242A (en) * 2017-03-22 2017-09-08 四川精目科技有限公司 A kind of high precision image method for registering that kernel estimates are obscured for imaging system
CN107301667A (en) * 2017-06-01 2017-10-27 中国人民解放军国防科学技术大学 The PSF methods of estimation of picture are calculated as to simple lens based on chessboard table images

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103856723A (en) * 2014-02-25 2014-06-11 中国人民解放军国防科学技术大学 PSF fast calibration method based on single-lens imaging
CN104778659A (en) * 2015-04-15 2015-07-15 杭州电子科技大学 Single-frame image super-resolution reconstruction method on basis of deep learning
CN106203506A (en) * 2016-07-11 2016-12-07 上海凌科智能科技有限公司 A kind of pedestrian detection method based on degree of depth learning art
CN106447626A (en) * 2016-09-07 2017-02-22 华中科技大学 Blurred kernel dimension estimation method and system based on deep learning
CN106780378A (en) * 2016-12-08 2017-05-31 中国人民解放军国防科学技术大学 A kind of blind convolved image restored method that two lenses lens have been corrected for aberration
CN106910175A (en) * 2017-02-28 2017-06-30 武汉大学 A kind of single image defogging algorithm based on deep learning
CN107146242A (en) * 2017-03-22 2017-09-08 四川精目科技有限公司 A kind of high precision image method for registering that kernel estimates are obscured for imaging system
CN107301667A (en) * 2017-06-01 2017-10-27 中国人民解放军国防科学技术大学 The PSF methods of estimation of picture are calculated as to simple lens based on chessboard table images

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
HAMED H. AGHDAM 等: "Fusing Convolutional Neural Networks with a Restoration Network for Increasing Accuracy and Stability", 《ECCV 2016》 *
MD AMIRUL ISLAM 等: "Label Refinement Network for Coarse-to-Fine Semantic Segmentation", 《ARXIV》 *
XIAO-JIAO MAO 等: "Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections", 《30TH CONFERENCE ON NEURAL INFORMATION PROCESSING SYSTEMS (NIPS 2016)》 *
孙旭 等: "基于深度学习的图像超分辨率复原研究进展", 《自动化学报》 *
胡长胜 等: "基于深度特征学习的图像超分辨率重建", 《自动化学报》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108830801A (en) * 2018-05-10 2018-11-16 湖南丹尼尔智能科技有限公司 A kind of deep learning image recovery method of automatic identification vague category identifier
CN109191493A (en) * 2018-07-13 2019-01-11 上海大学 A kind of method for tracking target based on RefineNet neural network and sparse optical flow
CN109191493B (en) * 2018-07-13 2021-06-04 上海大学 Target tracking method based on RefineNet neural network and sparse optical flow
CN108921806A (en) * 2018-08-07 2018-11-30 Oppo广东移动通信有限公司 A kind of image processing method, image processing apparatus and terminal device
CN110084763A (en) * 2019-04-29 2019-08-02 北京达佳互联信息技术有限公司 Image repair method, device, computer equipment and storage medium
CN113436137A (en) * 2021-03-12 2021-09-24 北京世纪好未来教育科技有限公司 Image definition recognition method, device, equipment and medium

Similar Documents

Publication Publication Date Title
CN107833193A (en) A kind of simple lens global image restored method based on refinement network deep learning models
Shih et al. 3d photography using context-aware layered depth inpainting
JP6905602B2 (en) Image lighting methods, devices, electronics and storage media
CN110008817A (en) Model training, image processing method, device, electronic equipment and computer readable storage medium
TW202112306A (en) Method and apparatus for detecting a human body, computer device, and storage medium
CN107833186A (en) A kind of simple lens spatial variations image recovery method based on Encoder Decoder deep learning models
CN109416727A (en) Glasses minimizing technology and device in a kind of facial image
CN110009573B (en) Model training method, image processing method, device, electronic equipment and storage medium
CN109584358A (en) A kind of three-dimensional facial reconstruction method and device, equipment and storage medium
KR20130089649A (en) Method and arrangement for censoring content in three-dimensional images
CN107730469A (en) A kind of three unzoned lens image recovery methods based on convolutional neural networks CNN
CN105488771B (en) Light field image edit methods and device
CN111626960A (en) Image defogging method, terminal and computer storage medium
CN110503619A (en) Image processing method, device and readable storage medium storing program for executing
CN110443764A (en) Video repairing method, device and server
CN110223251A (en) Suitable for manually with the convolutional neural networks underwater image restoration method of lamp
Li et al. Uphdr-gan: Generative adversarial network for high dynamic range imaging with unpaired data
Panetta et al. Deep perceptual image enhancement network for exposure restoration
Hong et al. Near-infrared image guided reflection removal
CN109788270A (en) 3D-360 degree panorama image generation method and device
CN116051407A (en) Image restoration method
CN113436107B (en) Image enhancement method, intelligent device and computer storage medium
CN107133981A (en) Image processing method and device
CN110751668A (en) Image processing method, device, terminal, electronic equipment and readable storage medium
CN112788254A (en) Camera image matting method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180323