CN106339984A - Distributed image super-resolution method based on K-means driven convolutional neural network - Google Patents
Distributed image super-resolution method based on K-means driven convolutional neural network Download PDFInfo
- Publication number
- CN106339984A CN106339984A CN201610739816.5A CN201610739816A CN106339984A CN 106339984 A CN106339984 A CN 106339984A CN 201610739816 A CN201610739816 A CN 201610739816A CN 106339984 A CN106339984 A CN 106339984A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image block
- little
- training
- little image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
- G06T3/4076—Super resolution, i.e. output image resolution higher than sensor resolution by iteratively correcting the provisional high resolution image using the original low-resolution image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Abstract
The invention provides a distributed image super-resolution method based on a K-means driven convolutional neural network. The method comprises the following steps: cutting a low-resolution image for super resolution according to position to obtain a plurality of low-resolution small image blocks; extracting content structure features of the plurality of low-resolution small image blocks; carrying out classification on the plurality of low-resolution small image blocks according to the content structure features and by utilizing K clustering centers of a training model; carrying out up-sampling on the plurality of low-resolution small image blocks to obtain high-resolution no-detail small image blocks; inputting the high-resolution no-detail small image blocks corresponding to the low-resolution small image blocks to a convolutional neural network model corresponding to the corresponding clustering center to obtain output of the model, and adding the output and the high-resolution no-detail small image blocks to obtain high-resolution small image blocks; and jointing the plurality of high-resolution small image blocks according to positions to obtain a final high-resolution image. The super-resolution effect of the distributed image super-resolution method is high.
Description
Technical field
The present invention relates to Image Super-resolution technology, more particularly, to a kind of distribution driving convolutional neural networks based on k average
Formula image super-resolution method.
Background technology
Single image super-resolution is the classical problem of computer vision field, its objective is from single low-resolution image
Recover containing high-definition picture in greater detail.Single image super-resolution is widely used in each of computer vision field
Plant in application, such as safety and protection monitoring imaging, imaging of medical etc. need the field of more image details.The going out of single image super-resolution
The low defect of imageable target resolution ratio that the hardware resolution that now compensate for image capture device is low, wide-long shot leads to, shape
Become high quality graphic.
In the case that image capture device does not collect high-definition picture, to be improved using single image super-resolution
The resolution ratio of image is the process of " from scratch ", be based on machine learning model according to previous experiences and the present situation Lai
The process giving a forecast.Using the test of single image super-resolution is the learning ability to previous experiences for the model, that is, test mould
The learning ability to training dataset for the type.In addition, single image resolution ratio also can test the pardon of training dataset, that is,
The abundant degree training data of test training dataset concentrates the sample image species comprising abundanter, the study of model
Just more comprehensively.Because the presence of standard exercise data set, whether most models are simultaneously enriched without the concern for training dataset
Problem.
However, the existing single image super resolution algorithm such as bicubic, sc, anr, srcnn, due to itself algorithm
Learning ability limited, its super-resolution effect does not reach optimum.
Content of the invention
Brief overview with regard to the present invention is given below, to provide basic with regard to certain aspects of the invention
Understand.It should be appreciated that this general introduction is not the exhaustive general introduction with regard to the present invention.It is not intended to determine the pass of the present invention
Key or pith, nor is it intended to limit the scope of the present invention.Its purpose only provides some concepts in simplified form,
In this, as the preamble in greater detail discussed after a while.
In consideration of it, a kind of the invention provides distributed image super-resolution side driving convolutional neural networks based on k average
Method, leads to its super-resolution at least to solve existing single image super resolution algorithm because the learning ability of itself algorithm is limited
Effect is not reaching to the problem of optimum.
According to an aspect of the invention, it is provided a kind of distributed image driving convolutional neural networks based on k average
Ultra-resolution method, distributed image ultra-resolution method includes: the low-resolution image of super-resolution is treated in opsition dependent cutting, low to obtain
The little image block of the corresponding multiple low resolution of image in different resolution；Extract the respective content structure of the little image block of multiple low resolution special
Levy；According to the respective content structure feature of the little image block of multiple low resolution, utilize k cluster centre of training pattern to multiple
The little image block of low resolution is classified, and respectively obtains each affiliated cluster centre of the little image block of multiple low resolution, wherein,
Training pattern includes k cluster centre, and each cluster centre corresponding convolutional neural networks model, and wherein k is positive integer；
To multiple low resolution, little image block carries out up-sampling process, to obtain the wherein corresponding high score of the little image block of each low resolution
The little image block of resolution no details；For each of little image block of multiple low resolution, by little for this low resolution image block pair
The little image block of the high-resolution answered no details is input to the corresponding convolutional Neural of the affiliated cluster centre of the little image block of this low resolution
In network model, obtain the output result of this convolutional neural networks model, by by this output result and the little figure of this low resolution
Obtain the little figure of the corresponding high-resolution of the little image block of this low resolution as the corresponding high-resolution of the block no little image addition of details
As block；Little for each for little for multiple low resolution image block self-corresponding high-resolution image block is pressed its position in low-resolution image
The relation of putting carries out splicing, to obtain the corresponding high-definition picture of low-resolution image.
Further, training pattern is obtained ahead of time in the following way: by all high-resolution in standard exercise data set
Rate training image is cut into multiple training little image blocks of high-resolution in an overlapping manner, by multiple training little images of high-resolution
Block constitutes new training set；Extract the content structure feature of each training little image block of high-resolution in new training set；Instructed according to new
Practice the content structure feature concentrating each training little image block of high-resolution, using k mean cluster mode to institute in new training set
There is the training little image block of high-resolution to be classified, obtain each training high-resolution in k cluster centre and new training set
The cluster labels of little image block；For each of k cluster centre, using convolutional neural networks method to this cluster centre
Corresponding all training little image blocks of high-resolution carry out super-resolution training, obtain the corresponding convolutional neural networks of this cluster centre
Model；By k cluster centre and wherein each cluster centre corresponding convolutional neural networks model composing training model.
Further, each cluster centre corresponding convolutional neural networks model includes corresponding first convolution of this model
Layer, the second convolutional layer and the 3rd convolutional layer；Wherein, the first convolutional layer adopts the convolution nuclear matrix of 128 9*9And
128 bias b1, the output result of this layer is z1=d*w1+b1,Represent input data,Represent
Output result, this layer of output result is through the first activation primitive f1(z1)=max (z1, 0) process after be input to the second convolutional layer；
Second convolutional layer adopts the convolution nuclear matrix of 64 5*5And 64 bias b2, the output result of this layer is z2=
f1(z1)*w2+b2,This layer of output result is through the second activation primitive f2(z2)=max (z2, 0) process
After be input to the 3rd convolutional layer；3rd convolutional layer adopts the convolution nuclear matrix of 1 5*5And 1 bias b3, should
What layer was defeated goes out result is z3=f2(z2)*w3+b3,This layer of output result is input simultaneously to damage with label l
Lose layer；Loss layer is used for calculating the mean square deviation between third layer output result and corresponding labelShould
Mean square deviation is as loss output.
By above description, the distributed image super-resolution driving convolutional neural networks based on k average of the present invention
Method, on the one hand it take the thought of " dividing and rule ", and the low-resolution image treating super-resolution is cut into little image block simultaneously
Extract the content structure feature of little image block, based on k cluster centre in training pattern, little image block is classified, with this
Carry out the little image blocks different to content complex degree of structure to be respectively processed, process content structure complexity can be applied to relatively
High image；And on the other hand, the method additionally uses the strong model convolutional neural networks model of learning ability to all kinds of
Little image block is respectively processed to obtain the little image block of corresponding high-resolution, and then finally obtains corresponding height by splicing
Image in different resolution is such that it is able to improve the learning ability of algorithm itself so that super-resolution effect hinge structure substantially improves.
Additionally, in the training stage, namely during training pattern builds in advance, and super-resolution process is similarly also adopted by
The thought of " dividing and rule ", will be cut into little image block in an overlapping manner by the image in standard exercise data set, extract
The content structure feature of little image block is simultaneously clustered to little image block, to content complex structure by the way of k mean cluster
The different little image block of degree is respectively processed, the image higher thus, it is possible to be applied to process content structure complexity；And
And also use the strong model convolutional neural networks model of learning ability and carry out all kinds of little image block to k mean cluster gained
It is trained respectively obtaining the corresponding convolutional neural networks model of the little image block of every class, improve the learning ability of algorithm itself,
And then super-resolution effect can be improved.
Additionally, using the convolutional neural networks model of three-layer coil lamination, can further improve driving based on k average of the present invention
The disposal ability of the distributed image ultra-resolution method of dynamic convolutional neural networks so that the learning ability of algorithm itself is higher, energy
The more complicated image of enough process, and super-resolution effect is preferable.
By the detailed description to highly preferred embodiment of the present invention below in conjunction with accompanying drawing, the these and other of the present invention is excellent
Point will be apparent from.
Brief description
The present invention can be by reference to being better understood below in association with the description given by accompanying drawing, wherein in institute
Have and employ same or analogous reference in accompanying drawing to represent same or like part.Described accompanying drawing is together with following
Describe the part comprising in this manual and being formed this specification together in detail, and be used for being further illustrated this
The preferred embodiment of invention and the principle and advantage explaining the present invention.In the accompanying drawings:
Fig. 1 is the one of the distributed image ultra-resolution method driving convolutional neural networks based on k average illustrating the present invention
The flow chart planting exemplary process；
Fig. 2 is the flow chart of an example of the building process of the training pattern illustrating the present invention；
Fig. 3 a is that the cluster of training stage in the preferred embodiment of distributed image ultra-resolution method illustrate the present invention is pre-
The flow chart processing operation chart；
Fig. 3 b is the model instruction of training stage in the preferred embodiment of distributed image ultra-resolution method illustrate the present invention
The flow chart practicing operation chart；
Fig. 3 c is super-resolution stage schematic diagram in the preferred embodiment of distributed image ultra-resolution method illustrate the present invention
Flow chart；
Fig. 3 d is the convolutional Neural net adopting in the preferred embodiment of distributed image ultra-resolution method illustrate the present invention
The flow chart of network model schematic.
It will be appreciated by those skilled in the art that element in accompanying drawing be used for the purpose of simple and clear for the sake of and illustrate,
And be not necessarily drawn to scale.For example, in accompanying drawing, the size of some elements may be exaggerated with respect to other elements, with
Just it is favorably improved the understanding to the embodiment of the present invention.
Specific embodiment
Hereinafter in connection with accompanying drawing, the one exemplary embodiment of the present invention is described.For clarity and conciseness,
All features of actual embodiment are not described in the description.It should be understood, however, that developing any this actual enforcement
A lot of decisions specific to embodiment, to realize the objectives of developer, for example, symbol must be made during example
Close those restrictive conditions related to system and business, and these restrictive conditions may have with the difference of embodiment
Changed.Additionally, it also should be appreciated that although development is likely to be extremely complex and time-consuming, but to having benefited from the disclosure
For those skilled in the art of content, this development is only routine task.
Here is in addition it is also necessary to illustrate is a bit, in order to avoid having obscured the present invention because of unnecessary details, in the accompanying drawings
Illustrate only and the apparatus structure closely related according to the solution of the present invention and/or process step, and eliminate and the present invention
The little other details of relation.
The embodiment provides a kind of distributed image super-resolution side driving convolutional neural networks based on k average
Method, this distributed image ultra-resolution method includes: the low-resolution image of super-resolution is treated in opsition dependent cutting, to obtain low resolution
The little image block of the corresponding multiple low resolution of image；Extract the respective content structure feature of the little image block of multiple low resolution；Root
According to the respective content structure feature of the little image block of multiple low resolution, utilize k cluster centre of training pattern to multiple low points
The little image block of resolution is classified, and respectively obtains each affiliated cluster centre of the little image block of multiple low resolution, wherein, training
Model includes k cluster centre, and each cluster centre corresponding convolutional neural networks model, and wherein k is positive integer；To many
The little image block of individual low resolution carries out up-sampling process, to obtain the wherein corresponding high-resolution of the little image block of each low resolution
The no little image block of details；For each of little image block of multiple low resolution, will be corresponding for little for this low resolution image block
The little image block of high-resolution no details is input to the corresponding convolutional neural networks of the affiliated cluster centre of the little image block of this low resolution
In model, obtain the output result of this convolutional neural networks model, by by this output result and the little image block of this low resolution
The little image addition of corresponding high-resolution no details and obtain the little image block of the corresponding high-resolution of the little image block of this low resolution；
Little for each for little for multiple low resolution image block self-corresponding high-resolution image block is pressed its position in low-resolution image close
System carries out splicing, to obtain the corresponding high-definition picture of low-resolution image.
The distributed image super-resolution being driven convolutional neural networks based on k average of the present invention to be described with reference to Fig. 1
The handling process 100 of one example of method.
As shown in figure 1, handling process start after execution step s110.
In step s110, the low-resolution image of super-resolution is treated in opsition dependent cutting, to obtain low-resolution image correspondence
The little image block of multiple low resolution.Then, execution step s120.
In step s120, extract the respective content structure feature of the little image block of multiple low resolution.Then, execution step
s130.
In step s130, according to the respective content structure feature of the little image block of multiple low resolution, utilize training pattern
K cluster centre to multiple low resolution, little image block is classified, respectively obtain the little image block of multiple low resolution each
Affiliated cluster centre, wherein, training pattern includes k cluster centre, and each cluster centre corresponding convolutional Neural net
Network model, wherein k are positive integer.Then, execution step s140.
In step s140, to multiple low resolution, little image block carries out up-sampling process, to obtain wherein each low point
The corresponding high-resolution of the little image block of the resolution no little image block of details.Then, execution step s150.
In step s150, for each of little image block of multiple low resolution, select this low in training pattern
The affiliated cluster centre of the little image block of resolution ratio corresponding convolutional neural networks model, by corresponding for little for this low resolution image block height
The little image block of resolution ratio no details is input to the corresponding convolutional neural networks mould of the affiliated cluster centre of the little image block of this low resolution
In type, obtain the output result of this convolutional neural networks model, by by this output result and the little image block pair of this low resolution
The little image addition of the high-resolution answered no details and obtain the little image block of the corresponding high-resolution of the little image block of this low resolution.By
This, obtained the little image block of the respective high-resolution of the little image block of multiple low resolution.Then, execution step s160.
In step s160, little for each for little for multiple low resolution image block self-corresponding high-resolution image block is pressed it low
Position relationship in image in different resolution carries out splicing, to obtain the corresponding high-definition picture of low-resolution image.Process
Flow process 100 terminates.
By above description, drive the distributed image ultra-resolution method of convolutional neural networks, one based on k average
Aspect takes the thought of " dividing and rule ", the low-resolution image treating super-resolution is cut into little image block and extracts little image
The content structure feature of block, is classified to little image block based on k cluster centre in training pattern, content is tied with this
The different little image block of structure complexity is respectively processed, and can be applied to the higher image of process content structure complexity；
And on the other hand, the method additionally uses the strong model convolutional neural networks (cnn) of learning ability to all kinds of little image blocks
It is respectively processed to obtain the little image block of corresponding high-resolution, and then corresponding high resolution graphics is finally obtained by splicing
As such that it is able to improve the learning ability of algorithm itself so that super-resolution effect hinge structure substantially improves.
According to a kind of implementation, training pattern can be obtained ahead of time by step s210 as shown in Figure 2～s250.
As shown in Fig. 2 in step s210, by all high-resolution training images in standard exercise data set with overlap
Mode according to position be cut into multiple training the little image blocks of high-resolution, by multiple training the little image blocks of high-resolution constitute newly
Training set.Then, execution step s220.
In step s220, extract the content structure feature of each training little image block of high-resolution in new training set.So
Afterwards, execution step s230.
In step s230, according to the content structure feature of each training little image block of high-resolution in new training set, profit
With k means clustering method in new training set all training the little image blocks of high-resolution classify, obtain k cluster centre with
And in new training set each training little image block of high-resolution cluster labels.Then, execution step 240.
It should be noted that " k means clustering method " is method or the title of algorithm, k here does not refer to certain numerical value；
And the k in " k cluster centre " then refers to the number of cluster centre, it is positive integer.
In step s240, for each of k cluster centre, using convolutional neural networks method in this cluster
The corresponding all training little image blocks of high-resolution of the heart carry out super-resolution training, obtain this cluster centre corresponding convolutional Neural net
Network model.Then, execution step s250.
In step s250, by k cluster centre and wherein each cluster centre corresponding convolutional neural networks model
Composing training model.
In the design process of single image super-resolution machine learning model: (1) is from the angle of image itself
The more complicated image of content structure requires higher to the learning ability of algorithm；(2) from the angle Algorithm Learning of algorithm
Ability is stronger, and super-resolution effect is better.And the existing single image super resolution algorithm such as bicubic, sc, anr, srcnn,
These algorithms all can not take into account above 2 points, therefore its learning ability all limited so that super-resolution effect is also poor.
In this implementation, the distributed image super-resolution side driving convolutional neural networks based on k average of the present invention
Method one side takes the thought of " dividing and rule ", that is, by the cutting in an overlapping manner of the image in standard exercise data set
Become little image block, extract the content structure feature of little image block and by the way of k mean cluster, little image block is clustered,
The little image block different to content complex degree of structure is respectively processed, thus, it is possible to be applied to process content structure complexity
Higher image；And on the other hand, the method additionally uses the strong model convolutional neural networks (cnn) of learning ability to k
All kinds of little image block of mean cluster gained is trained obtaining the corresponding convolutional neural networks model of the little image block of every class respectively,
Improve the learning ability of algorithm itself, and then super-resolution effect can be improved.
Additionally, according to a kind of implementation, each cluster centre corresponding convolutional neural networks model can include this mould
Corresponding first convolutional layer of type, the second convolutional layer and the 3rd convolutional layer.
Wherein, the first convolutional layer adopts the convolution nuclear matrix of 128 9*9And 128 bias b1, this layer
Output result be z1=d*w1+b1,Represent input data,Represent output result, this layer exports
Result is through the first activation primitive f1(z1)=max (z1, 0) process after be input to the second convolutional layer.
Second convolutional layer adopts the convolution nuclear matrix of 64 5*5And 64 bias b2, the output knot of this layer
Fruit is z2=f1(z1)*w2+b2,This layer of output result is through the second activation primitive f2(z2)=max (z2,
0) it is input to the 3rd convolutional layer after processing.
3rd convolutional layer adopts the convolution nuclear matrix of 1 5*5And 1 bias b3, this layer defeated to go out result
For z3=f2(z2)*w3+b3,This layer of output result is input simultaneously to loss layer with label l.
Loss layer is used for calculating the mean square deviation between third layer output result and corresponding labelWill
This mean square deviation is as loss output.
Thus, by the convolutional neural networks model of three-layer coil lamination in this implementation, can further improve the present invention
Based on k average drive convolutional neural networks distributed image ultra-resolution method disposal ability so that of algorithm itself
Habit ability is higher, can process more complicated image, and super-resolution effect is preferable.
Preferred embodiment
A preferred embodiment of the present invention is described below, in this embodiment, by the mistake building in advance of training pattern
The Cheng Zuowei training stage, and using the process obtaining the corresponding high-definition picture of low-resolution image treating super-resolution as oversubscription
Distinguish the stage, describe the processing procedure in this two stages separately below in detail.
1. the training stage
As shown in Figure 3 a, the purpose of cluster preprocessing is according to picture material complex degree of structure by standard exercise data set
Difference classified, its specific operation process is as follows:
Step 011:
All high-resolution training images in standard exercise data set are carried out cutting in an overlapping manner, obtains one
The individual new training dataset (as new training set) being made up of the training little image block of high-resolution.Same high-resolution training figure
The part of the simple part of content structure and content structure complexity may be comprised in picture simultaneously, be cut into training high-resolution
The little image block of rate can make the content structure of the single image of training data concentration more single as far as possible.
Step 012:
(1) gaussian filtering process: i is carried out to the little image block of new training concentration training high-resolution obtaining in step 011g
(x, y)=i (x, y) * g (x, y), whereinRepresent the training little image block of high-resolution,Represent Gaussian filter, σ is the standard deviation of Gaussian function, and * represents convolution algorithm,Represent filtered image block.
(2) according to scale factor s pairCarry out down-sampled process obtain down-sampled after image block: ig↓
(x, y)=ig(x, y) ↓ s, ↓ represent down-sampled process,
(3) by down-sampled process after image block deduct its own average and be organized into vector form, using this vectorial as
The content structure feature of the little image block of new training concentration training high-resolution obtaining in step 011
Step 013:
Using k means clustering algorithm, the content structure feature obtaining in step 012 is clustered, obtain in k cluster
The heartEach of which row represent a cluster centre) and the little image block institute of each high-resolution of sign
Belong to the cluster labels id (output of step 013 will be used for model training operation and super-resolution stage) of classification.Wherein, c is that k is individual poly-
The matrix that class center is constituted, k represents kth row (i.e. k-th cluster centre, each cluster centre is a vector), and i represents
Value at position i in k-th cluster centre (i.e. k-th vector).
As shown in Figure 3 b, the purpose of model training is elder generation training data being concentrated with convolutional neural networks (cnn) model
Test and learnt, its specific operation process is as follows:
Step 021:
Train the little figure of high-resolution using the cluster labels id that cluster preprocessing operation obtains to all of in new training set
As block is classified, obtain k class training data subset.Wherein, the image block content structure between every class training data subset is poor
Away from larger, and the image block content structure in same class is close.
Step 022:
iin(x, y)=((i (x, y) * g (x, y)) ↓ s) ↑ s, ↑ represent that up-sampling is processed.To in 1～k class training data subset
The little image block of training high-resolutionCarry out gaussian filtering process (ig(x, y)=i (x, y) * g (x, y)) and
Down-sampled process (↓) can be obtained each train the little image block of the corresponding low resolution of the little image block of high-resolution(now obtained the high-resolution of same image block with low-resolution image block to), to this low resolution
The little image block of rate carries out up-sampling process and can obtain this corresponding high-resolution of little image block of training high-resolution no little figure of details
As block(now image block resolution ratio be improved but details therein does not improve).Wherein, iin(x,y)
Represent input picture.
Step 023:
Using little for the high-resolution obtaining in step 022 no details image block as convolutional neural networks mould as shown in Figure 3 b
The input of type 1～k.With the high-resolution no details obtaining in the little image block of training high-resolution in step 021 and step 022
Little image block makees difference ir(x, y)=i (x, y)-iin(x, y), by this differenceAs convolutional neural networks model k
Input value label.This step can obtain k convolutional neural networks model for the super-resolution stage.
2. the super-resolution stage
As shown in Figure 3 c, the purpose in super-resolution stage is by the distributed image oversubscription based on cluster and convolutional neural networks
The method of distinguishing is used for single image super-resolution, and its specific operation process is as follows:
Step 031:
The low-resolution image opsition dependent cutting treating super-resolution of input is obtained the little image block of low resolution.
Step 032:
Little for low resolution image block is deducted its average and is organized into vector formUsing this vectorial as
The content structure feature of the little image block of low resolution, calculates the k cluster centre that this content structure feature is obtained with the training stage
Between Euclidean distance, that is,According to the shortest principle of distance, i.e. vectorial d
K in (), minimum numerical value corresponding position k is this low resolution little image block generic k, according to the little image block of low resolution
Generic k selects corresponding convolutional neural networks model.
Step 033:
To the low resolution obtaining in step 031 little image block carry out up-sampling process obtain the little image block of this low resolution
The little image block of corresponding high-resolution no details.
Step 034:
The little image block classification of high-resolution no details that step 033 is obtained is input to the convolutional Neural that step 032 selects
In network model, and it is obtained the little image block of corresponding high-resolution with model output addition.
Step 035:
Judge whether cutting out of low-resolution image is terminated: if terminated, go to step 036；Otherwise, go to step
031.
Step 036:
Little for the high-resolution of all positions image block opsition dependent relation is carried out splicing and can get final high score
Resolution image.
Although the present invention is described according to the embodiment of limited quantity, benefit from above description, the art
Interior it is clear for the skilled person that it can be envisaged that other embodiments in the scope of the present invention thus describing.Additionally, it should be noted that
Language used in this specification primarily to the purpose of readable and teaching and select, rather than in order to explain or limit
Determine subject of the present invention and select.Therefore, in the case of without departing from the scope of the appended claims and spirit, for this
For the those of ordinary skill of technical field, many modifications and changes will be apparent from.For the scope of the present invention, to this
It is illustrative and not restrictive for inventing done disclosure, and it is intended that the scope of the present invention be defined by the claims appended hereto.
Claims (3)
1. the distributed image ultra-resolution method of convolutional neural networks is driven it is characterised in that described distributed figure based on k average
As ultra-resolution method includes:
The low-resolution image of super-resolution is treated in opsition dependent cutting, to obtain the corresponding multiple low resolution of described low-resolution image
Little image block；
Extract the respective content structure feature of the little image block of the plurality of low resolution；
According to the respective content structure feature of the little image block of the plurality of low resolution, using training pattern k cluster centre
To the plurality of low resolution, little image block is classified, and respectively obtains the little image block of the plurality of low resolution each affiliated
Cluster centre, wherein, described training pattern includes k cluster centre, and the corresponding convolutional neural networks of each cluster centre
Model, wherein k are positive integer；
To the plurality of low resolution, little image block carries out up-sampling process, to obtain the wherein little image block pair of each low resolution
The little image block of the high-resolution answered no details；
For each of little image block of the plurality of low resolution, by corresponding for little for this low resolution image block high-resolution
No the little image block of details is input in the corresponding convolutional neural networks model of the affiliated cluster centre of the little image block of this low resolution, obtains
To the output result of this convolutional neural networks model, by by corresponding with the little image block of this low resolution for this output result high score
The little image addition of resolution no details and obtain the little image block of the corresponding high-resolution of the little image block of this low resolution；
Little for each for little for the plurality of low resolution image block self-corresponding high-resolution image block is pressed it in described low resolution figure
Position relationship in picture carries out splicing, to obtain the corresponding high-definition picture of described low-resolution image.
2. distributed image ultra-resolution method according to claim 1 is it is characterised in that described training pattern is by as follows
Mode is obtained ahead of time:
All high-resolution training images in standard exercise data set are cut into multiple training high-resolution in an overlapping manner
The little image block of rate, constitutes new training set by the plurality of training little image block of high-resolution；
Extract the content structure feature of each training little image block of high-resolution in described new training set；
According to the content structure feature of each training little image block of high-resolution in described new training set, using k mean cluster side
Formula is classified to training little image blocks of high-resolution all in described new training set, obtain k cluster centre and described newly
The cluster labels of each training little image block of high-resolution in training set；
For each of described k cluster centre, corresponding to this cluster centre all using convolutional neural networks method
The training little image block of high-resolution carries out super-resolution training, obtains this cluster centre corresponding convolutional neural networks model；
Described training mould is constituted by described k cluster centre and wherein each cluster centre corresponding convolutional neural networks model
Type.
3. distributed image ultra-resolution method according to claim 1 and 2 is it is characterised in that each cluster centre corresponds to
Convolutional neural networks model include corresponding first convolutional layer of this model, the second convolutional layer and the 3rd convolutional layer；
Wherein, the first convolutional layer adopts the convolution nuclear matrix of 128 9*9And 128 bias b1, the output of this layer
Result is z1=d*w1+b1,Represent input data,Represent output result, this layer of output result warp
Cross the first activation primitive f1(z1)=max (z1, 0) process after be input to the second convolutional layer；
Second convolutional layer adopts the convolution nuclear matrix of 64 5*5And 64 bias b2, the output result of this layer is
z2=f1(z1)*w2+b2,This layer of output result is through the second activation primitive f2(z2)=max (z2, 0) and place
It is input to the 3rd convolutional layer after reason；
3rd convolutional layer adopts the convolution nuclear matrix of 1 5*5And 1 bias b3, this layer of defeated result that goes out is z3
=f2(z2)*w3+b3,This layer of output result is input simultaneously to loss layer with label l；
Loss layer is used for calculating the mean square deviation between third layer output result and corresponding labelThis is equal
Variance yields is as loss output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610739816.5A CN106339984B (en) | 2016-08-27 | 2016-08-27 | Distributed image ultra-resolution method based on K mean value driving convolutional neural networks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610739816.5A CN106339984B (en) | 2016-08-27 | 2016-08-27 | Distributed image ultra-resolution method based on K mean value driving convolutional neural networks |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106339984A true CN106339984A (en) | 2017-01-18 |
| CN106339984B CN106339984B (en) | 2019-09-13 |
Family
ID=57822376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610739816.5A Active CN106339984B (en) | 2016-08-27 | 2016-08-27 | Distributed image ultra-resolution method based on K mean value driving convolutional neural networks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106339984B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108805844A (en) * | 2018-06-30 | 2018-11-13 | 合肥工业大学 | A kind of lightweight Recurrent networks construction method based on priori filtering |
| CN109118459A (en) * | 2017-06-23 | 2019-01-01 | 南开大学 | Image significance object detection method and device |
| CN109564677A (en) * | 2018-11-09 | 2019-04-02 | 香港应用科技研究院有限公司 | Super-resolution synthesis system and method based on random forest grader weighted results |
| CN110136056A (en) * | 2018-02-08 | 2019-08-16 | 华为技术有限公司 | The method and apparatus of image super-resolution rebuilding |
| CN110785709A (en) * | 2017-06-30 | 2020-02-11 | 科磊股份有限公司 | Generating high resolution images from low resolution images for semiconductor applications |
| CN111034183A (en) * | 2017-09-04 | 2020-04-17 | 株式会社半导体能源研究所 | Image processing method, semiconductor device, and electronic apparatus |
| WO2020093423A1 (en) * | 2018-11-09 | 2020-05-14 | Hong Kong Applied Science and Technology Research Institute Company Limited | Systems and methods for super-resolution synthesis based on weighted results from random forest classifier |
| CN111242217A (en) * | 2020-01-13 | 2020-06-05 | 支付宝实验室(新加坡)有限公司 | Training method and device of image recognition model, electronic equipment and storage medium |
| CN111489293A (en) * | 2020-03-04 | 2020-08-04 | 北京思朗科技有限责任公司 | Super-resolution reconstruction method and device for image |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110026840A1 (en) * | 2009-07-28 | 2011-02-03 | Samsung Electronics Co., Ltd. | System and method for indoor-outdoor scene classification |
| CN103617597A (en) * | 2013-10-25 | 2014-03-05 | 西安电子科技大学 | A remote sensing image fusion method based on difference image sparse representation |
| CN104778671A (en) * | 2015-04-21 | 2015-07-15 | 重庆大学 | Image super-resolution method based on SAE and sparse representation |
| CN104992407A (en) * | 2015-06-17 | 2015-10-21 | 清华大学深圳研究生院 | Image super-resolution method |
| CN105488776A (en) * | 2014-10-10 | 2016-04-13 | 北京大学 | Super-resolution image reconstruction method and apparatus |
| CN105590304A (en) * | 2014-10-24 | 2016-05-18 | 北京大学 | Reconstruction method and apparatus for super-resolution image |
-
2016
- 2016-08-27 CN CN201610739816.5A patent/CN106339984B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110026840A1 (en) * | 2009-07-28 | 2011-02-03 | Samsung Electronics Co., Ltd. | System and method for indoor-outdoor scene classification |
| CN103617597A (en) * | 2013-10-25 | 2014-03-05 | 西安电子科技大学 | A remote sensing image fusion method based on difference image sparse representation |
| CN105488776A (en) * | 2014-10-10 | 2016-04-13 | 北京大学 | Super-resolution image reconstruction method and apparatus |
| CN105590304A (en) * | 2014-10-24 | 2016-05-18 | 北京大学 | Reconstruction method and apparatus for super-resolution image |
| CN104778671A (en) * | 2015-04-21 | 2015-07-15 | 重庆大学 | Image super-resolution method based on SAE and sparse representation |
| CN104992407A (en) * | 2015-06-17 | 2015-10-21 | 清华大学深圳研究生院 | Image super-resolution method |
Non-Patent Citations (1)
| Title |
|---|
| 任晓芳: "基于选择学习机制的深度图像超分辨率算法", 《视频应用与工程》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11430205B2 (en) | 2017-06-23 | 2022-08-30 | Huawei Technologies Co., Ltd. | Method and apparatus for detecting salient object in image |
| CN109118459A (en) * | 2017-06-23 | 2019-01-01 | 南开大学 | Image significance object detection method and device |
| CN110785709B (en) * | 2017-06-30 | 2022-07-15 | 科磊股份有限公司 | Generating high resolution images from low resolution images for semiconductor applications |
| CN110785709A (en) * | 2017-06-30 | 2020-02-11 | 科磊股份有限公司 | Generating high resolution images from low resolution images for semiconductor applications |
| CN111034183A (en) * | 2017-09-04 | 2020-04-17 | 株式会社半导体能源研究所 | Image processing method, semiconductor device, and electronic apparatus |
| CN111034183B (en) * | 2017-09-04 | 2022-05-13 | 株式会社半导体能源研究所 | Image processing method, semiconductor device, and electronic apparatus |
| CN110136056A (en) * | 2018-02-08 | 2019-08-16 | 华为技术有限公司 | The method and apparatus of image super-resolution rebuilding |
| CN110136056B (en) * | 2018-02-08 | 2023-08-29 | 华为技术有限公司 | Method and device for reconstructing super-resolution image |
| CN108805844B (en) * | 2018-06-30 | 2021-10-08 | 合肥工业大学 | Lightweight regression network construction method based on prior filtering |
| CN108805844A (en) * | 2018-06-30 | 2018-11-13 | 合肥工业大学 | A kind of lightweight Recurrent networks construction method based on priori filtering |
| WO2020093423A1 (en) * | 2018-11-09 | 2020-05-14 | Hong Kong Applied Science and Technology Research Institute Company Limited | Systems and methods for super-resolution synthesis based on weighted results from random forest classifier |
| US10685428B2 (en) | 2018-11-09 | 2020-06-16 | Hong Kong Applied Science And Technology Research Institute Co., Ltd. | Systems and methods for super-resolution synthesis based on weighted results from a random forest classifier |
| CN109564677A (en) * | 2018-11-09 | 2019-04-02 | 香港应用科技研究院有限公司 | Super-resolution synthesis system and method based on random forest grader weighted results |
| CN109564677B (en) * | 2018-11-09 | 2022-09-27 | 香港应用科技研究院有限公司 | Super-resolution synthesis system and method based on random forest classifier weighting result |
| CN111242217A (en) * | 2020-01-13 | 2020-06-05 | 支付宝实验室(新加坡)有限公司 | Training method and device of image recognition model, electronic equipment and storage medium |
| CN111489293A (en) * | 2020-03-04 | 2020-08-04 | 北京思朗科技有限责任公司 | Super-resolution reconstruction method and device for image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106339984B (en) | 2019-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106339984A (en) | Distributed image super-resolution method based on K-means driven convolutional neural network | |
| CN110097131B (en) | Semi-supervised medical image segmentation method based on countermeasure cooperative training | |
| CN108647585B (en) | Traffic identifier detection method based on multi-scale circulation attention network | |
| CN109255364B (en) | Scene recognition method for generating countermeasure network based on deep convolution | |
| CN106157319B (en) | The conspicuousness detection method in region and Pixel-level fusion based on convolutional neural networks | |
| CN104850845B (en) | A kind of traffic sign recognition method based on asymmetric convolutional neural networks | |
| US20190205758A1 (en) | Gland segmentation with deeply-supervised multi-level deconvolution networks | |
| CN103886308B (en) | A kind of pedestrian detection method of use converging channels feature and soft cascade grader | |
| CN106372648A (en) | Multi-feature-fusion-convolutional-neural-network-based plankton image classification method | |
| CN112070158B (en) | Facial flaw detection method based on convolutional neural network and bilateral filtering | |
| CN108021923A (en) | A kind of image characteristic extracting method for deep neural network | |
| CN106600560A (en) | Image defogging method for automobile data recorder | |
| CN108596338A (en) | A kind of acquisition methods and its system of neural metwork training collection | |
| CN109858429A (en) | A kind of identification of eye fundus image lesion degree and visualization system based on convolutional neural networks | |
| CN107945200A (en) | Image binaryzation dividing method | |
| CN106846261A (en) | Underwater picture processing method based on convolutional neural networks | |
| EP3837632A1 (en) | Methods for generating a deep neural net and for localising an object in an input image, deep neural net, computer program product, and computer-readable storage medium | |
| CN114842019A (en) | Battery plate surface defect detection method, system, storage medium and equipment | |
| CN110223304A (en) | A kind of image partition method, device and computer readable storage medium based on multipath polymerization | |
| CN116091886A (en) | Semi-supervised target detection method and system based on teacher student model and strong and weak branches | |
| CN113628297A (en) | COVID-19 deep learning diagnosis system based on attention mechanism and transfer learning | |
| CN116912218A (en) | Deep learning-based battery surface defect detection method and device | |
| CN113762265A (en) | Pneumonia classification and segmentation method and system | |
| CN111612803B (en) | Vehicle image semantic segmentation method based on image definition | |
| CN116958073A (en) | Small sample steel defect detection method based on attention feature pyramid mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |