CN110211059A - A kind of image rebuilding method based on deep learning - Google Patents
A kind of image rebuilding method based on deep learning Download PDFInfo
- Publication number
- CN110211059A CN110211059A CN201910409034.9A CN201910409034A CN110211059A CN 110211059 A CN110211059 A CN 110211059A CN 201910409034 A CN201910409034 A CN 201910409034A CN 110211059 A CN110211059 A CN 110211059A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- label
- data
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000013135 deep learning Methods 0.000 title claims abstract description 14
- 238000012549 training Methods 0.000 claims abstract description 41
- 230000004913 activation Effects 0.000 claims abstract description 38
- 238000013528 artificial neural network Methods 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims abstract description 9
- 238000003062 neural network model Methods 0.000 claims abstract description 8
- 238000005070 sampling Methods 0.000 claims abstract description 5
- 230000000694 effects Effects 0.000 claims description 10
- 239000013598 vector Substances 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 2
- 238000013507 mapping Methods 0.000 abstract description 3
- 239000011229 interlayer Substances 0.000 abstract description 2
- 230000010354 integration Effects 0.000 abstract 1
- 239000010410 layer Substances 0.000 description 24
- 230000003321 amplification Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007850 degeneration Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The invention discloses a kind of image rebuilding methods based on deep learning, the following steps are included: key problem in technology is S1 its object is to be trained to obtain low resolution to the mapping function between high-resolution to high-resolution data using depth learning technology: carrying out down-sampling processing to data set；S2: utilizing residual error principle, and the convolution activation result of different interlayers is added；S3: training data, which is divided into, has label and without two class of label, the corresponding two kinds of loss numbers of two kinds of situations；S4: two class situations of integration obtain final loss function.The present invention inputs any one low-resolution image into trained neural network model, and the output of neural network is the super-resolution image after rebuilding.The present invention effectively improves acquired picture quality under the premise of not changing image system hardware equipment.
Description
Technical field
The present invention relates to technical field of image processing, in particular to a kind of image rebuilding method based on deep learning.
Background technique
Image super-resolution rebuilding technology plays an important role in the resolution ratio for promoting chip image at present.Super-resolution
Rate (Super-Resolution) is the resolution ratio that original image is improved by the method for hardware or software, passes through a series of low points
The image of resolution is come to obtain a high-resolution image process be exactly super-resolution rebuilding.Super-resolution is in video compress and passes
Defeated, medical image auxiliary diagnosis, the fields such as safety monitoring and satellite imagery have a wide range of applications.
For research method, image super-resolution rebuilding technology can be divided into based on interpolation, based on reconstruction and based on
Three classes, the generally existing apparent sawtooth effect of method based on interpolation are practised, the method based on reconstruction considers the degeneration mould of image
Type, and can be greatly improved in conjunction with the priori knowledge of image, performance compared with interpolation method, but it is applied to effect when chip image
It is still bad；The main thought of super-resolution algorithms based on study is between study low-resolution image and high-definition picture
Corresponding relationship depth with the rise of machine learning, is based on according to the super-resolution rebuilding of this corresponding relationship guide image
The super-resolution rebuilding algorithm of study gradually emerges, and when handling common natural image, such methods show outstanding
Performance, but rebuild be made of intensive circuit chip image when, can not by the detail section of image processing it is fine.
Super-resolution mainly has following two evaluation criterion: (1) the reconstruction effect of image, and the target of reconstruction is to restore image
High-frequency information, improve the quality of image, promote the visual effect of reconstruction image as much as possible；(2) the reconstruction efficiency of image, mesh
Be exactly guarantee rebuild effect while, as far as possible improve rebuild speed.Super-resolution rebuilding according to technical principle not
It is same to be divided into following three types: the method based on difference, the method based on reconstruction, the method based on study.Wherein,
Method based on study is method more popular at present, and the method based on study is usually to pass through a data set study high score
Then mapping relations between resolution image and low resolution image rebuild high-definition picture using the mapping relations learnt.
Currently used learning method includes SRCNN, ESPCN, VDSR etc..
But in existing learning method, for the image of different scale, the reinforcing effect of super-resolution is inconsistent.
Summary of the invention
The main purpose of the present invention is to provide a kind of image rebuilding method based on deep learning, can effectively solve to carry on the back
The problems in scape technology.
To achieve the above object, the technical scheme adopted by the invention is as follows:
A kind of image rebuilding method based on deep learning, comprising the following steps:
S1. n times of down-sampling processing is carried out to training dataset, the wide height of former high-resolution training data IH is W, H respectively, is obtained
To the wide height of low resolution training data IL be W/n, H/n respectively；
S2. it is corresponded by original high-resolution image IH and by the low resolution IL image that step 1 obtains, obtains label
Training data, optionally takes low training data of the resolution training dataset as no label, and the data volume without label is greater than there is mark
These two types data are saved as HDF5 (Hierarchical Data Format) file by the data volume of label；
S3. planned network structure determines deep neural network input layer number, output layer number of nodes, hides the number of plies and hide
Node layer number, the connection weight W and biasing b of each layer of random initializtion, gives learning rate η, selectes activation primitive RELU, select
Loss function Loss.
S4. the low resolution training data IL size of input is amplified n times, the amplification is will be at low resolution picture interpolation
Reason, i.e., amplified image pixel are that IS (n × i+1, n × j+1)=IL (i+1, j+1), i indexes for image lateral position, j
For image lengthwise position index；IS remaining do not have respective value obtain pixel value be 255；
S5. convolution, activation processing, wherein selected convolution kernel are carried out using 40 layers of convolutional neural networks to by amplified image
It is 3 × 3, activation primitive is f (x)=max (x, 0), using residual error principle, by the convolution activation result of first layer and the tenth layer
Convolution activation result be added；The convolution activation result of eleventh floor and the 20th layer of convolution activation result are added；By
The convolution activation result of two eleventh floors and the 30th layer of convolution activation result are added；By the convolution activation result of third eleventh floor
The high resolution graphics rebuild is added with the convolution activation result of first layer after being added with the 40th layer of convolution activation result
As IS；
S6. step 5 is executed repeatedly, until neural network output layer error reaches default required precision or frequency of training reaches maximum
The number of iterations terminates training, saves network structure and parameter, obtains trained neural network model；
S7. any one low-resolution image is inputted into trained neural network model, and the output of neural network is attached most importance to
Super-resolution image after building.
Wherein, the training dataset of use, which is divided into, has label and without label data, the label is low-resolution image institute
Corresponding original high-resolution image, it is described to have label data and obtained without label data by above-mentioned steps 2, there is label and without mark
Two kinds of situations of label have respectively corresponded two kinds of loss numbers.
Wherein, when having label, using the calculation method of Euclidean distance, image array has W × H element (pixel), uses
W × H element value (A1, A2 ..., AW × H) constitutes the one-dimensional vector of original high-resolution image, with (a1, a2
..., aW × H) one-dimensional vector for rebuilding high-definition picture is constituted, then this is calculated using Euclidean distance formula mathematically
The distance between two vectors just illustrate that two images are more similar apart from smaller；The Euclidean distance formula are as follows:
Original high-resolution image IH=(A1, A2 ..., AW × H)
Reconstruction high-definition picture IS=(a1, a2 ..., aW × H)
Euclidean distance。
Wherein, the high-definition picture rebuild when no label does not have original high-definition picture to be corresponding to it, so will
Low-resolution image IL is as the references object for rebuilding high-definition picture IS, to keep the structure of IL；It is described by low resolution
Image IL is as the references object for rebuilding high-definition picture IS.
Wherein, loss function total during hands-on by two parts Loss1 and Loss2 in conjunction with and obtain, Loss1
Effect be to make IS=IH, the effect of Loss2 be make IS keep IL architectural characteristic.
Wherein,, wherein whether there is or not labels, training mark for the data of k expression training
The k=1 when data of label, k=0 when training the data without label.
Compared with prior art, the invention has the following beneficial effects: this method not to change image system hardware
Under the premise of equipment, acquired picture quality is effectively improved.
Specific embodiment
To be easy to understand the technical means, the creative features, the aims and the efficiencies achieved by the present invention, below with reference to
Specific embodiment, the present invention is further explained.
A kind of image rebuilding method based on deep learning proposed by the present invention, comprising the following steps:
S1. n times of down-sampling processing is carried out to training dataset, the wide height of former high-resolution training data IH is W, H respectively, is obtained
To the wide height of low resolution training data IL be W/n, H/n respectively；
S2. it is corresponded by original high-resolution image IH and by the low resolution IL image that step 1 obtains, obtains label
Training data, optionally takes low training data of the resolution training dataset as no label, and the data volume without label is greater than there is mark
These two types data are saved as HDF5 (Hierarchical Data Format) file by the data volume of label；
S3. planned network structure determines deep neural network input layer number, output layer number of nodes, hides the number of plies and hide
Node layer number, the connection weight W and biasing b of each layer of random initializtion, gives learning rate η, selectes activation primitive RELU, select
Loss function Loss.
S4. the low resolution training data IL size of input is amplified n times, the amplification is will be at low resolution picture interpolation
Reason, i.e., amplified image pixel are that IS (n × i+1, n × j+1)=IL (i+1, j+1), i indexes for image lateral position, j
For image lengthwise position index；IS remaining do not have respective value obtain pixel value be 255；
S5. convolution, activation processing, wherein selected convolution kernel are carried out using 40 layers of convolutional neural networks to by amplified image
It is 3 × 3, activation primitive is f (x)=max (x, 0), using residual error principle, by the convolution activation result of first layer and the tenth layer
Convolution activation result be added；The convolution activation result of eleventh floor and the 20th layer of convolution activation result are added；By
The convolution activation result of two eleventh floors and the 30th layer of convolution activation result are added；By the convolution activation result of third eleventh floor
The high resolution graphics rebuild is added with the convolution activation result of first layer after being added with the 40th layer of convolution activation result
As IS；
S6. step 5 is executed repeatedly, until neural network output layer error reaches default required precision or frequency of training reaches maximum
The number of iterations terminates training, saves network structure and parameter, obtains trained neural network model；
S7. any one low-resolution image is inputted into trained neural network model, and the output of neural network is attached most importance to
Super-resolution image after building.
Wherein, the training dataset of use, which is divided into, has label and without label data, the label is low-resolution image institute
Corresponding original high-resolution image, it is described to have label data and obtained without label data by above-mentioned steps 2, there is label and without mark
Two kinds of situations of label have respectively corresponded two kinds of loss numbers.
Wherein, when having label, using the calculation method of Euclidean distance, image array has W × H element (pixel), uses
W × H element value (A1, A2 ..., AW × H) constitutes the one-dimensional vector of original high-resolution image, with (a1, a2
..., aW × H) one-dimensional vector for rebuilding high-definition picture is constituted, then this is calculated using Euclidean distance formula mathematically
The distance between two vectors just illustrate that two images are more similar apart from smaller；The Euclidean distance formula are as follows:
Original high-resolution image IH=(A1, A2 ..., AW × H)
Reconstruction high-definition picture IS=(a1, a2 ..., aW × H)
Euclidean distance。
Wherein, the high-definition picture rebuild when no label does not have original high-definition picture to be corresponding to it, so will
Low-resolution image IL is as the references object for rebuilding high-definition picture IS, to keep the structure of IL；It is described by low resolution
Image IL is as the references object for rebuilding high-definition picture IS.
Wherein, loss function total during hands-on by two parts Loss1 and Loss2 in conjunction with and obtain, Loss1
Effect be to make IS=IH, the effect of Loss2 be make IS keep IL architectural characteristic.
Wherein,, wherein whether there is or not labels, training mark for the data of k expression training
The k=1 when data of label, k=0 when training the data without label.
A kind of image rebuilding method based on deep learning proposed by the present invention, it is critical that (1) carries out data set
Down-sampling processing；(2) residual error principle is utilized, the convolution activation result of different interlayers is added；(3) training data, which is divided into, label
With no two class of label, the corresponding two kinds of loss numbers of two kinds of situations；(4) two class situations are integrated, final loss function is obtained.The present invention
Any one low-resolution image is inputted into trained neural network model, the output of neural network is super after rebuilding
Image in different resolution.The present invention effectively improves acquired picture quality under the premise of not changing image system hardware equipment.
The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention.The technology of the industry
Personnel are it should be appreciated that the present invention is not limited to the above embodiments, and the above embodiments and description only describe this
The principle of invention, without departing from the spirit and scope of the present invention, various changes and improvements may be made to the invention, these changes
Change and improvement all fall within the protetion scope of the claimed invention.The claimed scope of the invention by appended claims and its
Equivalent thereof.
Claims (6)
1. a kind of image rebuilding method based on deep learning, which comprises the following steps:
S1. n times of down-sampling processing is carried out to training dataset, the wide height of former high-resolution training data IH is W, H respectively, is obtained
To the wide height of low resolution training data IL be W/n, H/n respectively；
S2. it is corresponded by original high-resolution image IH and by the low resolution IL image that step 1 obtains, obtains label
Training data, optionally takes low training data of the resolution training dataset as no label, and the data volume without label is greater than there is mark
These two types data are saved as HDF5 (Hierarchical Data Format) file by the data volume of label；
S3. planned network structure determines deep neural network input layer number, output layer number of nodes, hides the number of plies and hide
Node layer number, the connection weight W and biasing b of each layer of random initializtion, gives learning rate η, selectes activation primitive RELU, select
Loss function Loss；
S4. the low resolution training data IL size of input is amplified n times, it is described amplify be by low resolution picture interpolation processing,
I.e. amplified image pixel is IS (n × i+1, n × j+1)=IL (i+1, j+1), i for image lateral position index, and j is
Image lengthwise position index；IS remaining do not have respective value obtain pixel value be 255；
S5. convolution, activation processing, wherein selected convolution kernel are carried out using 40 layers of convolutional neural networks to by amplified image
It is 3 × 3, activation primitive is f (x)=max (x, 0), using residual error principle, by the convolution activation result of first layer and the tenth layer
Convolution activation result be added；The convolution activation result of eleventh floor and the 20th layer of convolution activation result are added；By
The convolution activation result of two eleventh floors and the 30th layer of convolution activation result are added；By the convolution activation result of third eleventh floor
The high resolution graphics rebuild is added with the convolution activation result of first layer after being added with the 40th layer of convolution activation result
As IS；
S6. step 5 is executed repeatedly, until neural network output layer error reaches default required precision or frequency of training reaches maximum
The number of iterations terminates training, saves network structure and parameter, obtains trained neural network model；
S7. any one low-resolution image is inputted into trained neural network model, and the output of neural network is attached most importance to
Super-resolution image after building.
2. a kind of image rebuilding method based on deep learning according to claim 1, it is characterised in that: the training of use
Data set, which is divided into, label and without label data, the label is original high-resolution image corresponding to low-resolution image,
It is described to have label data and obtained without label data by above-mentioned steps 2, there are label and two kinds of situations without label to respectively correspond
Two kinds of loss numbers.
3. a kind of image rebuilding method based on deep learning according to claim 1, it is characterised in that: when having label,
Using the calculation method of Euclidean distance, image array has W × H element (pixel), with W × H element value (A1, A2
..., AW × H) constitute original high-resolution image one-dimensional vector, with (a1, a2 ..., aW × H) constitute rebuild high-resolution
Then the one-dimensional vector of rate image calculates the distance between the two vectors using Euclidean distance formula mathematically, distance is got over
It is small just to illustrate that two images are more similar；The Euclidean distance formula are as follows:
Original high-resolution image IH=(A1, A2 ..., AW × H)
Reconstruction high-definition picture IS=(a1, a2 ..., aW × H)
Euclidean distance。
4. a kind of image rebuilding method based on deep learning according to claim 1, it is characterised in that: weight when no label
The high-definition picture built does not have original high-definition picture to be corresponding to it, so low-resolution image IL is high as rebuilding
The references object of image in different resolution IS, to keep the structure of IL；It is described using low-resolution image IL as rebuild high resolution graphics
As the references object of IS.
5. a kind of image rebuilding method based on deep learning according to claim 1, it is characterised in that: in hands-on
In the process total loss function by two parts Loss1 and Loss2 in conjunction with and obtain, the effect of Loss1 is to make IS=IH, Loss2
Effect be make IS keep IL architectural characteristic.
6. a kind of image rebuilding method based on deep learning according to claim 1, it is characterised in that:, wherein whether there is or not labels for the data of k expression training, when training has the data of label
K=1, k=0 when training the data without label.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910409034.9A CN110211059A (en) | 2019-05-17 | 2019-05-17 | A kind of image rebuilding method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910409034.9A CN110211059A (en) | 2019-05-17 | 2019-05-17 | A kind of image rebuilding method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110211059A true CN110211059A (en) | 2019-09-06 |
Family
ID=67787465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910409034.9A Withdrawn CN110211059A (en) | 2019-05-17 | 2019-05-17 | A kind of image rebuilding method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110211059A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113361689A (en) * | 2021-06-09 | 2021-09-07 | 上海联影智能医疗科技有限公司 | Training method of super-resolution reconstruction network model and scanning image processing method |
| CN113706383A (en) * | 2021-08-30 | 2021-11-26 | 上海亨临光电科技有限公司 | Super-resolution method, system and device for terahertz image |
-
2019
- 2019-05-17 CN CN201910409034.9A patent/CN110211059A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113361689A (en) * | 2021-06-09 | 2021-09-07 | 上海联影智能医疗科技有限公司 | Training method of super-resolution reconstruction network model and scanning image processing method |
| CN113706383A (en) * | 2021-08-30 | 2021-11-26 | 上海亨临光电科技有限公司 | Super-resolution method, system and device for terahertz image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108765296B (en) | Image super-resolution reconstruction method based on recursive residual attention network | |
| CN109389556B (en) | Multi-scale cavity convolutional neural network super-resolution reconstruction method and device | |
| CN109087273B (en) | Image restoration method, storage medium and system based on enhanced neural network | |
| CN108122197A (en) | A kind of image super-resolution rebuilding method based on deep learning | |
| CN106910161A (en) | A kind of single image super resolution ratio reconstruction method based on depth convolutional neural networks | |
| CN107730451A (en) | A kind of compressed sensing method for reconstructing and system based on depth residual error network | |
| CN107633486A (en) | Structure Magnetic Resonance Image Denoising based on three-dimensional full convolutional neural networks | |
| CN105488776B (en) | Super-resolution image reconstruction method and device | |
| CN106991646A (en) | A kind of image super-resolution method based on intensive connection network | |
| CN108961186A (en) | A kind of old film reparation recasting method based on deep learning | |
| Zhu et al. | Efficient single image super-resolution via hybrid residual feature learning with compact back-projection network | |
| CN110930309A (en) | Face super-resolution method and device based on multi-view texture learning | |
| CN110349087B (en) | RGB-D image high-quality grid generation method based on adaptive convolution | |
| CN109389667B (en) | High-efficiency global illumination drawing method based on deep learning | |
| Chen et al. | Single image super-resolution using deep CNN with dense skip connections and inception-resnet | |
| CN113421187B (en) | Super-resolution reconstruction method, system, storage medium and equipment | |
| CN111932461A (en) | Convolutional neural network-based self-learning image super-resolution reconstruction method and system | |
| CN112215755A (en) | Image super-resolution reconstruction method based on back projection attention network | |
| CN107977930A (en) | A kind of image super-resolution method and its system | |
| CN114223019A (en) | Feedback decoder for parameter efficient semantic image segmentation | |
| CN110211059A (en) | A kind of image rebuilding method based on deep learning | |
| CN111986092B (en) | Dual-network-based image super-resolution reconstruction method and system | |
| CN113449612A (en) | Three-dimensional target point cloud identification method based on sub-flow sparse convolution | |
| CN111080648A (en) | Real-time image semantic segmentation algorithm based on residual learning | |
| CN114565628B (en) | Image segmentation method and system based on boundary perception attention |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20190906 |
|
| WW01 | Invention patent application withdrawn after publication |