CN108416736A - A kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood - Google Patents
A kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood Download PDFInfo
- Publication number
- CN108416736A CN108416736A CN201810236623.7A CN201810236623A CN108416736A CN 108416736 A CN108416736 A CN 108416736A CN 201810236623 A CN201810236623 A CN 201810236623A CN 108416736 A CN108416736 A CN 108416736A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- low
- anchor point
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 91
- 238000013507 mapping Methods 0.000 claims abstract description 62
- 239000011159 matrix material Substances 0.000 claims abstract description 61
- 238000012360 testing method Methods 0.000 claims abstract description 59
- 230000008569 process Effects 0.000 claims description 17
- 238000000513 principal component analysis Methods 0.000 claims description 12
- 230000009467 reduction Effects 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000013178 mathematical model Methods 0.000 description 3
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 241000283070 Equus zebra Species 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000010835 comparative analysis Methods 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000002969 morbid Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood, step includes：Off-line training is carried out, the training of low-resolution dictionary, positive mapping matrix and inverse mapping matrix is included；Extract the characteristic image block of low resolution test image；An anchor point is selected, a super-resolution rebuilding is carried out to low resolution test image, obtains primary reconstruction high-definition picture；The primary reconstruction high-definition picture of acquisition is mapped to different low-resolution images, selection and the original immediate low-resolution image of low resolution test image resolution ratio, corresponding anchor is set to secondary anchor point；Secondary super-resolution rebuilding is carried out according to secondary anchor point, obtains final high-definition picture.The present invention is low by training, high resolution space inverse mapping, can choose better anchor point, so can super-resolution rebuilding mass it is more preferable, texture is apparent, the more rich high-definition picture of details.
Description
Technical field
The invention belongs to technical field of image processing, more particularly to a kind of image oversubscription returned based on secondary anchor point neighborhood
Resolution method for reconstructing.
Background technology
With internet and information-based development, image is as a kind of most direct and most important information carrier,
Spread over each corner of life.Requirement of the people to high resolution digital image is growing day by day, still, in image processing process
In, the operations such as image obtains, converts, transmits often will produce the problem of the fuzzy of image, distortion, plus noise etc. degrade.Therefore,
In order to obtain the high-definition picture for meeting people's demand, image super-resolution rebuilding technology is just produced.Image super-resolution
Reconstruction technique is the technology that a panel height image in different resolution is obtained by the low resolution image of one or more.Due in image deterioration mistake
Some detailed information are had lost in journey, are asked so being that a morbid state is inverse from low-resolution image reconstruction High-Resolution Map correlation essence
Topic.Image super-resolution rebuilding has become one of the important topic that image domains scholars study, presently, there are super-resolution
Rate algorithm for reconstructing is roughly divided into three classes：Algorithm based on interpolation, the algorithm based on reconstruct, the algorithm based on study.
Image Super-resolution Reconstruction algorithm based on interpolation is by establishing between the adjacent pixel of unknown pixel point
Interpolation kernel function calculates the pixel value of unknown point, and the high-resolution of different amplification is obtained by the pixel for being inserted into unknown
Image.It is divided into linear interpolation algorithm and non-linear interpolation algorithm according to the property of interpolation kernel function, and interpolation algorithm.Wherein,
Traditional linear interpolation algorithm includes arest neighbors interpolation algorithm, bilinear interpolation algorithm and bicubic interpolation algorithm etc..It is non-thread
Property interpolation algorithm include based on wavelet coefficient algorithm and be based on marginal information algorithm.The computational methods of this kind of algorithm are simple and efficient,
But the high-definition picture recovered will appear phenomena such as edge blurry and unintelligible texture.
Super-resolution algorithms based on reconstruction are established from high-definition picture to low-resolution image according to prior information
The mathematical model of process obtains super-resolution image then by solving the inverse process of the mathematical model.Super based on reconstruction
In resolution algorithm, representative algorithm includes iterative backprojection, full variational regularization, Markov random field and is based on
The method of gradient profile.This kind of maximum advantage of algorithm is exactly the artificial trace that can be very good to inhibit overlapping shape, obtains profile
Sharp-edged high-definition picture.However, actual high-definition picture may not necessarily meet mathematical model, and bad mould
Type will produce bad reconstruction image.
Super-resolution algorithms based on study are current relatively popular a kind of algorithms.This kind of algorithm passes through machine learning
Method establishes the mapping relations between high-low resolution image, then goes out corresponding high resolution graphics using this reconstruction of relations
Picture.The earliest algorithm based on study is to propose that they learn high and low point by markov network by Freemen et al.
Relationship between resolution image block simultaneously rebuilds high-definition picture.Chang et al. is proposed based on the image being locally linear embedding into
Super-resolution algorithms.Yang et al. propositions rebuild single image by rarefaction representation.Zeyde et al. Yang algorithms into
It has gone and has been further improved.In recent years, Timofte et al. combines neighborhood embedded mobile GIS and rarefaction representation algorithm successively to propose that ANR is calculated
Method and A+ algorithms, A+ algorithms are trained sparse dictionary using K-SVD algorithms, are sought in training sample using the atom of sparse dictionary as anchor point
It looks for neighbour to carry out neighborhood recurrence, obtains corresponding low, high resolution space the mapping of each anchor point.But existing A+ algorithms choosing
The anchor point selected not is best anchor point, and the high-definition picture quality of acquisition is not good enough, and detail textures are not enough clearly enriched.
Invention content
The purpose of the present invention is to provide it is a kind of based on secondary anchor point neighborhood return image super-resolution rebuilding method, with
Solve above-mentioned technical problem.The super resolution ratio reconstruction method of the present invention has continued the study thoughts based on learning algorithm,
Inverse mapping low by training, high resolution space can choose better anchor point, and then being capable of super-resolution rebuilding pledge
Amount is more preferable, and texture is apparent, the more rich high-definition picture of details.
In order to achieve the above objectives, the present invention uses following technical scheme：
A kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood, specific steps include：
Step 1, off-line training is carried out, the process of the off-line training includes low-resolution dictionary, positive mapping matrix and inverse
The training of mapping matrix stores the result of off-line training；
Step 2, low resolution test image is inputted, the characteristic image block of the low resolution test image is extracted；
Step 3, an anchor point is selected, a Super-resolution reconstruction is carried out to the low resolution test image inputted in step 2
It builds, obtains primary reconstruction high-definition picture；
Step 4, the primary reconstruction high-definition picture that step 3 obtains is mapped to by the inverse mapping matrix in applying step 1
Different low-resolution images is chosen and original low-resolution test image point in obtaining the different low-resolution image
The corresponding anchor point of the low-resolution image selected is set to secondary anchor point by the immediate low-resolution image of resolution；
Step 5, two are carried out to the low resolution test image described in step 2 according to the secondary anchor point obtained in step 4
Secondary super-resolution rebuilding obtains final high-definition picture.
Further, in step 1, low-resolution dictionary is trained by K- singular value decomposition algorithms, is returned by neighborhood
Return and trains positive mapping matrix and inverse mapping matrix.
Further, step 1 specifically includes：
Step 1.1, high-resolution training image is acquired, the low resolution training extracted per panel height resolution ratio training image is special
Levy image block collection and high-resolution training characteristics image block collection；
Step 1.2, low-resolution dictionary is trained by K- singular value decomposition algorithms, be denoted asWherein DlTable
Show low-resolution dictionary matrix, diFor i-th of atom of dictionary, N is the total number of dictionary atom, the low resolution that training is obtained
In the storage to parameter library of rate dictionary；
Step 1.3, using the dictionary atom in step 1.2 as anchor point, with the low resolution training characteristics figure of step 1.1 acquisition
Picture block integrates as neighborhood, calculates the Euclidean distance between anchor point and low resolution training characteristics image block, and one is found for each anchor point
A low resolution Neighbourhood set Nl, corresponding to find out corresponding high-resolution in step 1.1 high-resolution training image set of blocks
Neighbourhood set Nh；
Step 1.4, according to the corresponding low resolution Neighbourhood set of i-th of anchor point and high-resolution Neighbourhood set, calculating should
The positive mapping matrix and inverse mapping matrix, mathematical formulae of anchor point are expressed as：
Wherein, PiIndicate positive mapping matrix, QiIndicate that inverse mapping matrix, I indicate unit matrix, λ is balance factor；It calculates
The corresponding positive mapping matrix of all anchor points and inverse mapping matrix, finally will be in result of calculation storage to parameter library.
Further, step 1.1 specifically includes：
First, high-resolution training image is acquired, to every panel height resolution ratio training image HsDown-sampling and up-sampling are carried out,
It obtains with described per panel height image in different resolution HsThe identical low resolution variability training sample of size
Then, using high-pass filter f1=[1, -1], f2=f1 T, f3=[1, -2,1], f4=f3 TBecome respectively with low resolution
Rate training sampleConvolution is carried out, Gradient Features collection is obtainedThe quantity of Gradient Features collection is that 4, r is gradient
The subscript of feature set is corresponding with the subscript of high-pass filter；
Next, it is n to extract dimension from feature set, size isI-th of position characteristic image block；It is logical
The 4 Gradient Features collection obtained are crossed, 4 characteristic image blocks are obtained in the positions i, 4 characteristic blocks of acquisition are pressed and Gradient Features
Collect corresponding sequence and pull into column vector, constitutes the low resolution training characteristics image block of the positions iSize is 4n × 1；
Finally, using Principal Component Analysis Algorithm to the low resolution training characteristics image block of acquisitionDimension-reduction treatment is carried out,
The mathematical formulae of dimension-reduction treatment is expressed as：
Wherein,It is the transformation matrix of principal component analysis dimensionality reduction,It is the low resolution after dimension-reduction treatment
Rate training characteristics image block；The low resolution training characteristics image block after all dimension-reduction treatment is extracted, set is denoted as
Wherein N1For the total number of training characteristics image block；
High-resolution training image HsThe calculation formula of radio-frequency component be：
Wherein,Indicate HsRadio-frequency component, low resolution variability training sampleIndicate full resolution pricture HsLow frequency at
Point；Extract high-resolution training image blocksExtract all high-resolution training image blocksIt is denoted as set
Further, step 2 specifically includes：
Step 2.1, input low resolution test image Lt；
Step 2.2, low resolution test image L step 2.1 inputtedtIt is up-sampled, obtains amplified low point
Resolution image
Step 2.3, with 4 high-pass filter f1=[1, -1], f2=f1 T, f3=[1, -2,1], f4=f3 TRespectively with low point
Resolution imageConvolution is carried out, Gradient Features collection is obtainedThe quantity of Gradient Features collection is that 4, r is Gradient Features
The subscript of collection is corresponding with the subscript of high-pass filter；
Step 2.4, it is n dimension to be extracted from feature set, and size isI-th of low resolution test image
Characteristic image block
Step 2.5, using Principal Component Analysis Algorithm to low resolution test feature image blockCarry out dimension-reduction treatment, drop
Tieing up the mathematical formulae handled is：
Wherein,It is the transformation matrix of principal component analysis dimensionality reduction, the characteristic image block after dimension-reduction treatmentAll low resolution test feature image blocks are extracted, set is denoted asWherein N2For test feature image
The total number of block.
Further, step 3 specifically includes：
Block-by-block input setIn low resolution test feature image block, select anchor point to carry out an oversubscription
Resolution is rebuild, and primary reconstruction high-definition picture is obtained.
Further, the specific steps of anchor point selection include in step 3：
Step 3.1, by calculating i-th piece of low resolution test feature image blockWith anchor pointBetween it is European away from
From, select withThe nearest anchor point of Euclidean distanceAs an anchor point；
Step 3.2, according to the subscript m of the anchor point obtained in step 3.11, it is corresponding just to obtain an anchor point
Mapping matrixPass through calculation formula：
Obtain the corresponding high-definition picture block of i-th piece of low resolution test feature image block
Step 3.3, block-by-block has handled all low resolution test feature image blocks, by all high resolution graphics of acquisition
As blockIt is put into entire image on corresponding position, lap takes mean value, obtains the high resolution graphics of width entirety
Picture
Further, step 4 specifically includes：
Step 4.1, by the high-definition picture of acquisitionPiecemeal is carried out, high-definition picture set of blocks is obtained
Step 4.2, all inverse mapping matrixes are utilizedHandle high-definition picture blockIt is different to obtain N blocks
Low resolution characteristic image block calculates original low-resolution test feature image blockThe low resolution different from the N blocks is special
The Euclidean distance between image block is levied, the corresponding inverse mapping matrix of low resolution characteristic image block of Euclidean distance minimum is selected
Subscript m2, the index as secondary anchor point.
Further, step 5 specifically includes：
Step 5.1, according to the index of the secondary anchor point of acquisition, corresponding positive mapping matrix is obtainedCalculate secondary oversubscription
The high-definition picture block that resolution is rebuildCalculation formula is：
Step 5.2, block-by-block rebuilds high-definition picture block, obtains high-definition picture set of blocks
Step 5.3, all high-definition picture blocks obtained in step 5.2 are put on suitable position and the overlapping that is averaged
Region obtains the radio-frequency component of full resolution pricture
Step 5.4, by the low-resolution image containing low-frequency componentWith the full resolution pricture containing radio-frequency componentPhase
Add, obtains high-definition picture Xh。
Compared with prior art, the invention has the advantages that：
The image super-resolution rebuilding method of the present invention returned based on secondary anchor point neighborhood, by selecting secondary anchor point
Method carries out super-resolution rebuilding to low-resolution image.The present invention has continued the study thoughts based on learning algorithm, passes through instruction
Practice low, high resolution space forward and inverse mapping, better anchor point can be chosen, and then being capable of super-resolution rebuilding mass
More preferably, texture is apparent, the more rich high-definition picture of details.
An anchor is first selected according to the Euclidean distance between anchor point and low resolution test feature image block in the present invention
Point carries out first time super-resolution rebuilding.Then all anchors are obtained according to the high-definition picture of first time super-resolution rebuilding
The low-resolution image of the space of points, calculate between these low-resolution images and original low-resolution test image it is European away from
From the corresponding anchor point of low-resolution image of chosen distance minimum is secondary anchor point, carries out second of super-resolution rebuilding.This hair
It is bright to choose better anchor point by super-resolution rebuilding twice, obtain quality more preferably high-definition picture.
The present invention is low by training, high resolution space inverse mapping is proposed and a kind of returned based on secondary single anchor point neighborhood
The super-resolution algorithms of single image.So-called secondary single anchor point neighborhood returns, using A+ algorithms carry out for the first time selection anchor point into
Row super-resolution algorithms are rebuild, according to trained all inverse mapping matrixes, will carry out the image of a super-resolution from
High resolution space is mapped to different low-resolution spatials, finds empty with the immediate low resolution of original resolution space
Between.According to the corresponding index in most similar low resolution space, anchor point is reselected, secondary super-resolution rebuilding is carried out.
Forward and inverse mapping training process and traditional anchor point neighborhood regression algorithm in the off-line training of the present invention is dramatically different,
Anchor point neighborhood regression algorithm in the present invention be by and meanwhile the positive mapping matrix of training and inverse mapping matrix, by low-resolution image
It is mapped to higher dimensional space from lower dimensional space, then lower dimensional space is mapped to from higher dimensional space, a process is completed to anchor in this way
The second selecting of point, can obtain better anchor point.
Description of the drawings
The present invention is described in further details in the following with reference to the drawings and specific embodiments.
Fig. 1 is the off-line training step for the image super-resolution rebuilding method of the present invention returned based on secondary anchor point neighborhood
Flow diagram；
Fig. 2 is the test phase flow for the image super-resolution rebuilding method of the present invention returned based on secondary anchor point neighborhood
Schematic diagram；
Fig. 3 is the experimental result contrast schematic diagram that image man is amplified to 2 times using the method for reconstructing of the present invention；
Fig. 4 is the experimental result contrast schematic diagram that image head is amplified to 3 times using the method for reconstructing of the present invention；
Fig. 5 is the experimental result contrast schematic diagram that image zebra is amplified to 4 times using the method for reconstructing of the present invention.
Specific implementation mode
With reference to figure 1 and Fig. 2, a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood of the invention,
Specific steps include：
Step 1:Carry out off-line training, the process of the off-line training includes low-resolution dictionary, positive mapping matrix and inverse
The training of mapping matrix stores the result of off-line training.Off-line training process includes three training process.First was trained
Journey is to train low-resolution dictionary by K- singular value decomposition algorithms；Second training process be by neighborhood regression training just
Mapping matrix；Third training process is by neighborhood regression training inverse mapping matrix, this process is returned with traditional anchor point neighborhood
Reduction method is dramatically different, this anchor point neighborhood regression algorithm be by and meanwhile the positive mapping matrix of training and inverse mapping matrix, by low point
Resolution image is mapped to higher dimensional space from lower dimensional space, then is mapped to lower dimensional space from higher dimensional space, in this way a process
Complete the second selecting to anchor point.
Step 1 specifically includes：
Step 1.1, high-resolution training image is acquired, the low resolution training extracted per panel height resolution ratio training image is special
Image block collection and high-resolution training characteristics image block collection are levied, low resolution is relative concept, the resolution of image with high-resolution
Rate by image pixel number determine, high-resolution training image herein refers to the low resolution test than being inputted in step 2
Image more than image pixel after being trained by high-resolution training image, can be such that low resolution test image completes secondary super
Resolution reconstruction, high-resolution (HD, High Definition), generally refer to vertical resolution more than or equal to 720 image or
Video, also referred to as high-definition image or HD video, size are usually 1280 × 720 and 1920 × 1080；Image super-resolution is
Refer to and high-definition picture is recovered by a width low-resolution image or image sequence；
Step 1.2, low-resolution dictionary is trained by K- singular value decomposition algorithms, be denoted asWherein DlTable
The low-resolution dictionary matrix shown, diFor i-th of atom of dictionary, N is the total number of dictionary atom, low point that training is obtained
In the storage to parameter library of resolution dictionary；
Step 1.3, using the dictionary atom in step 1.2 as anchor point, with the low resolution training characteristics figure of step 1.1 acquisition
Picture block integrates as neighborhood, calculates the Euclidean distance between anchor point and low resolution training characteristics image block, and one is found for each anchor point
A low resolution Neighbourhood set Nl.Correspondence finds out corresponding high-resolution Neighbourhood set in high-resolution training image set of blocks
Nh；
Step 1.4, according to the corresponding low resolution Neighbourhood set of i-th of anchor point and high-resolution Neighbourhood set, calculating should
The positive mapping matrix and inverse mapping matrix, mathematical formulae of anchor point are expressed as：
Wherein, PiIndicate positive mapping matrix, QiIndicate that inverse mapping matrix, I indicate unit matrix.All anchor points are calculated to correspond to
Positive mapping matrix and inverse mapping matrix, finally will result of calculation storage in parameter library.
Following steps training is carried out to the existing high-resolution training image being collected into step 1.1：
First to every panel height resolution ratio training image HsDown-sampling and up-sampling are carried out, is obtained and high-definition picture size
Identical low resolution variability training sample
Then high-pass filter f is used1=[1, -1], f2=f1 T, f3=[1, -2,1], f4=f3 TRespectively with imageIt carries out
Convolution, to obtain Gradient Features collection
Next it is n dimension to be extracted from feature set, and size isIth feature image block.Due to obtaining
4 Gradient Features collection can obtain 4 characteristic image blocks simultaneously so being concentrated in this 4 Gradient Features in this position i, will
4 characteristic blocks obtained pull into column vector in order, constitute the low resolution training characteristics image block of i-th this positionGreatly
Small is 4n × 1.
Finally utilize Principal Component Analysis Algorithm to low resolution characteristic image blockCarry out dimension-reduction treatment, the mathematics of dimensionality reduction
Formula is as follows：
Wherein,It is the transformation matrix of principal component analysis dimensionality reduction, the characteristic image block after dimensionality reduction
All low resolution training characteristics image blocks are extracted, set is denoted asWherein N1For the sum of training characteristics image block
Mesh.
Assume low-resolution image simultaneouslyIndicate full resolution pricture HsLow-frequency component, then can be obtained by following equation
Obtain HsRadio-frequency component：
Wherein,Indicate HsRadio-frequency component.Same extraction high-resolution training image blocksExtract all high scores
Resolution training image blocks are denoted as set
Step 2, low resolution test image is inputted, the characteristic image block of the low resolution test image is extracted.
Step 2 specific steps include：
Step (1) inputs low resolution test image Lt。
Step (2) up-samples the low resolution test image of input, obtains amplified low-resolution image
Step (3) and then by 4 high-pass filter f1=[1, -1], f2=f1 T, f3=[1, -2,1], f4=f3 TNot with it is low
Image in different resolutionConvolution is carried out, Gradient Features collection is obtained
Next extraction dimension is n to step (4), and size isLow resolution test image ith feature figure
As block
Step (5) finally utilizes Principal Component Analysis Algorithm to low resolution test feature image blockDimension-reduction treatment is carried out,
The mathematical formulae of dimensionality reduction is as follows：
Wherein,It is the transformation matrix of principal component analysis dimensionality reduction, the characteristic image block after dimensionality reduction
All low resolution test feature image blocks are extracted, set is denoted asWherein N2For the sum of test feature image block
Mesh.
Step 3, an anchor point is selected, a Super-resolution reconstruction is carried out to the low resolution test image inputted in step 2
It builds, obtains primary reconstruction high-definition picture；Selection arest neighbors anchor point carries out super-resolution to low resolution test image for the first time
Rate is rebuild.
Step 3 specifically includes：Block-by-block inputs low resolution test feature image blockSelection anchor point carries out for the first time
Super-resolution rebuilding, the selection of a specific anchor point is shown in steps are as follows.
Step (1) is by calculating i-th piece of low resolution test feature image blockWith anchor pointBetween it is European away from
From, select withApart from nearest anchor pointAs an anchor point.
Step (2) is according to the subscript m of an anchor point1, obtain the corresponding mapping matrix of the anchor pointThen following meter is made
It calculates,
Obtain the corresponding high-definition picture block of i-th piece of low resolution test feature image block
Step (3) block-by-block has handled all low resolution test feature image blocks, all high-resolution that then will be obtained
Rate image blockIt is put into entire image on corresponding position, lap takes mean value.Finally obtain the height of width entirety
Image in different resolution
Step 4, using inverse mapping matrix, the high-definition picture once rebuild is mapped to different low-resolution images,
Selection and the immediate low-resolution image of original low resolution variability image, corresponding anchor point are the anchor point of second of selection.
The primary reconstruction high-definition picture that step 3 obtains is mapped to different low resolutions by the inverse mapping matrix i.e. in applying step 1
Rate image is chosen in obtaining the different low-resolution image with the original low resolution test image resolution ratio most
The low-resolution image corresponding anchor selected is set to secondary anchor point by close low-resolution image.
The specific steps of step 4 include：
Step (1) is by high-definition picturePiecemeal is carried out, high-definition picture set of blocks is obtained
Step (2) utilizes all inverse mapping matrixesHandle high-definition picture blockObtain different low of N blocks
Resolution characteristics image block calculates original low-resolution test feature image blockLow resolution characteristic images different from these
Euclidean distance between block, the subscript m of the corresponding inverse mapping matrix of low resolution characteristic image block of chosen distance minimum2, make
For the index of secondary anchor point.
Step 5, image super-resolution rebuilding is carried out according to the anchor point of second selecting, the high-resolution wanted to output
Rate image；The secondary oversubscription of the low resolution test image described in step 2 is carried out according to the secondary anchor point obtained in step 4
Resolution is rebuild, and final high-definition picture is obtained.
The specific steps of step 5 include：.
Step (1) obtains corresponding mapping matrix according to the index of secondary anchor pointIt is calculated according to following Mathematical Formula
The high-definition picture block of secondary super-resolution rebuilding
Step (2) block-by-block rebuilds high-definition picture block, obtains high-definition picture set of blocks
Obtained all high-definition picture blocks are put on suitable position and the overlapping region that is averaged by step (3), are obtained
The radio-frequency component of full resolution pricture
Step (4) is by the low-resolution image containing low-frequency componentWith the full resolution pricture containing radio-frequency componentPhase
Add, output high-definition picture Xh。
Experimental result comparative analysis：
The present invention weighs super-resolution image weight by calculating Y-PSNR (PSNR) and structural similarity (SSIM)
The result built.PSNR is to weigh the reconstruction matter of image based on the error between reconstruction image and original high-definition image pixel
Amount.PSNR values are bigger, show that the picture quality rebuild is better, distortion is smaller.And SSIM is mainly used to weigh reconstruction image and original
The parameter of structural similarity between beginning high-definition image.SSIM values are between 0 to 1.SSIM values are bigger, the knot between two images
Structure is more similar.
With reference to figure 3, when table 1 is by 2 times of image magnification, the Y-PSNR and structural similarity of various algorithm reconstruction images
Comparison result.
Each method comparison result when table 1. amplifies 2 times
With reference to figure 4, when table 2 is by 3 times of image magnification, the Y-PSNR and structural similarity of various algorithm reconstruction images
Comparison result.
Each method comparison result when table 2. amplifies 3 times
With reference to figure 5, when table 3 is by 4 times of image magnification, the Y-PSNR and structural similarity of various algorithm reconstruction images
Comparison result.
Each method comparison result when table 3. amplifies 4 times
Secondary anchor point neighborhood returns Super-Resolution of Images Based and has carried out two to anchor point on the basis of an anchor point maps
Secondary selection, an anchor point provide foundation for the mapping of secondary anchor point, can be obtained in conjunction with the experimental data in table 1, table 2 and table 3,
The method of the present invention is respectively compared with bicubic interpolation method and ANR algorithms, and end value is than bicubic interpolation method and ANR
The end value of algorithm is high.It is better than bicubic with reference to the quality of figure 3, Fig. 4 and Fig. 5, the method for the present invention high-definition picture rebuild
The high-definition picture that interpolation method and ANR algorithms are rebuild.
The present invention is a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood, including off-line training mistake
Journey and on-line testing process.Step 1 is off-line training process, and step 2 to step 5 is on-line testing process.The present invention is to pass through
Inverse mapping matrix is trained and applied, different low resolution figures is obtained according to the high-definition picture of a super-resolution rebuilding
Picture carries out second of anchor point selection using the relationship between these different low-resolution images and original low-resolution image.
The anchor point selected in this way makes the picture quality higher of super-resolution rebuilding, and image texture is apparent, and image detail is more rich.
Claims (9)
1. a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood, which is characterized in that specific steps include：
Step 1, off-line training is carried out, the process of the off-line training includes low-resolution dictionary, positive mapping matrix and inverse mapping
The training of matrix stores the result of off-line training；
Step 2, low resolution test image is inputted, the characteristic image block of the low resolution test image is extracted；
Step 3, an anchor point is selected, a super-resolution rebuilding is carried out to the low resolution test image inputted in step 2, is obtained
It obtains and once rebuilds high-definition picture；
Step 4, the primary reconstruction high-definition picture that step 3 obtains is mapped to difference by the inverse mapping matrix in applying step 1
Low-resolution image, chosen and original low-resolution test image resolution ratio in obtaining the different low-resolution image
The corresponding anchor point of the low-resolution image selected is set to secondary anchor point by immediate low-resolution image；
Step 5, the low resolution test image described in step 2 is carried out according to the secondary anchor point obtained in step 4 secondary super
Resolution reconstruction obtains final high-definition picture.
2. a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood according to claim 1, special
Sign is, in step 1, trains low-resolution dictionary by K- singular value decomposition algorithms, is gone out just by neighborhood regression training
Mapping matrix and inverse mapping matrix.
3. a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood according to claim 1, special
Sign is that step 1 specifically includes：
Step 1.1, high-resolution training image is acquired, the low resolution training characteristics figure per panel height resolution ratio training image is extracted
As block collection and high-resolution training characteristics image block collection；
Step 1.2, low-resolution dictionary is trained by K- singular value decomposition algorithms, be denoted asWherein DlIndicate low
Resolution ratio dictionary matrix, diFor i-th of atom of dictionary, N is the total number of dictionary atom, the low resolution word that training is obtained
In allusion quotation storage to parameter library；
Step 1.3, using the dictionary atom in step 1.2 as anchor point, with the low resolution training characteristics image block of step 1.1 acquisition
Integrate as neighborhood, calculate the Euclidean distance between anchor point and low resolution training characteristics image block, for each anchor point find one it is low
Resolution ratio Neighbourhood set Nl, corresponding to find out corresponding high-resolution neighborhood in step 1.1 high-resolution training image set of blocks
Set Nh；
Step 1.4, according to the corresponding low resolution Neighbourhood set of i-th of anchor point and high-resolution Neighbourhood set, the anchor point is calculated
Positive mapping matrix and inverse mapping matrix, mathematical formulae be expressed as：
Wherein, PiIndicate positive mapping matrix, QiIndicate that inverse mapping matrix, I indicate unit matrix, λ is balance factor；It calculates all
The corresponding positive mapping matrix of anchor point and inverse mapping matrix finally store result of calculation into parameter library.
4. a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood according to claim 3, special
Sign is that step 1.1 specifically includes：
First, high-resolution training image is acquired, to every panel height resolution ratio training image HsDown-sampling and up-sampling are carried out, is obtained
With described per panel height image in different resolution HsThe identical low resolution variability training sample of size
Then, using high-pass filter f1=[1, -1], f2=f1 T, f3=[1, -2,1], f4=f3 TIt is instructed respectively with low resolution variability
Practice sampleConvolution is carried out, Gradient Features collection is obtainedThe quantity of Gradient Features collection is that 4, r is Gradient Features collection
Subscript it is corresponding with the subscript of high-pass filter；
Next, it is n to extract dimension from feature set, size isI-th of position characteristic image block；By obtaining
4 Gradient Features collection, obtain 4 characteristic image blocks in the positions i, by 4 characteristic blocks of acquisition press and Gradient Features collection phase
Corresponding sequence pulls into column vector, constitutes the low resolution training characteristics image block of the positions iSize is 4n × 1；
Finally, using Principal Component Analysis Algorithm to the low resolution training characteristics image block of acquisitionCarry out dimension-reduction treatment, dimensionality reduction
The mathematical formulae of processing is expressed as：
Wherein,It is the transformation matrix of principal component analysis dimensionality reduction,It is the low resolution instruction after dimension-reduction treatment
Practice characteristic image block；The low resolution training characteristics image block after all dimension-reduction treatment is extracted, set is denoted asWherein
N1For the total number of training characteristics image block；
High-resolution training image HsThe calculation formula of radio-frequency component be：
Wherein,Indicate HsRadio-frequency component, low resolution variability training sampleIndicate full resolution pricture HsLow-frequency component；It carries
Take high-resolution training image blocksExtract all high-resolution training image blocksIt is denoted as set
5. a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood according to claim 1, special
Sign is that step 2 specifically includes：
Step 2.1, input low resolution test image Lt；
Step 2.2, low resolution test image L step 2.1 inputtedtIt is up-sampled, obtains amplified low resolution figure
Picture
Step 2.3, with 4 high-pass filter f1=[1, -1], f2=f1 T, f3=[1, -2,1], f4=f3 TRespectively with low resolution
ImageConvolution is carried out, Gradient Features collection is obtainedThe quantity of Gradient Features collection is that 4, r is Gradient Features collection
Subscript is corresponding with the subscript of high-pass filter；
Step 2.4, it is n dimension to be extracted from feature set, and size isLow resolution test image ith feature figure
As block
Step 2.5, using Principal Component Analysis Algorithm to low resolution test feature image blockCarry out dimension-reduction treatment, dimension-reduction treatment
Mathematical formulae be：
Wherein,It is the transformation matrix of principal component analysis dimensionality reduction, the characteristic image block after dimension-reduction treatmentIt carries
All low resolution test feature image blocks are taken, set is denoted asWherein N2For the total number of test feature image block.
6. a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood according to claim 5, special
Sign is that step 3 specifically includes：
Block-by-block input setIn low resolution test feature image block, select anchor point to carry out a super-resolution
It rebuilds, obtains primary reconstruction high-definition picture.
7. a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood according to claim 6, special
Sign is that the specific steps that an anchor point selects in step 3 include：
Step 3.1, by calculating i-th piece of low resolution test feature image blockWith anchor pointBetween Euclidean distance, choosing
It is fixed withThe nearest anchor point of Euclidean distanceAs an anchor point；
Step 3.2, according to the subscript m of the anchor point obtained in step 3.11, obtain the corresponding positive mapping square of an anchor point
Battle arrayPass through calculation formula：
Obtain the corresponding high-definition picture block of i-th piece of low resolution test feature image block
Step 3.3, block-by-block has handled all low resolution test feature image blocks, by all high-definition picture blocks of acquisitionIt is put into entire image on corresponding position, lap takes mean value, obtains the high-definition picture of width entirety
8. a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood according to claim 7, special
Sign is that step 4 specifically includes：
Step 4.1, by the high-definition picture of acquisitionPiecemeal is carried out, high-definition picture set of blocks is obtained
Step 4.2, all inverse mapping matrixes are utilizedHandle high-definition picture blockObtain different low point of N blocks
Resolution characteristic image block calculates original low-resolution test feature image blockThe low resolution characteristic pattern different from the N blocks
As the Euclidean distance between block, the subscript of the corresponding inverse mapping matrix of low resolution characteristic image block of Euclidean distance minimum is selected
m2, the index as secondary anchor point.
9. a kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood according to claim 8, special
Sign is that step 5 specifically includes：
Step 5.1, according to the index of the secondary anchor point of acquisition, corresponding positive mapping matrix is obtainedCalculate secondary super-resolution
The high-definition picture block of reconstructionCalculation formula is：
Step 5.2, block-by-block rebuilds high-definition picture block, obtains high-definition picture set of blocks
Step 5.3, all high-definition picture blocks obtained in step 5.2 are put on suitable position and the overlay region that is averaged
Domain obtains the radio-frequency component of full resolution pricture
Step 5.4, by the low-resolution image containing low-frequency componentWith the full resolution pricture containing radio-frequency componentIt is added,
Obtain high-definition picture Xh。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810236623.7A CN108416736B (en) | 2018-03-21 | 2018-03-21 | Image super-resolution reconstruction method based on secondary anchor point neighborhood regression |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810236623.7A CN108416736B (en) | 2018-03-21 | 2018-03-21 | Image super-resolution reconstruction method based on secondary anchor point neighborhood regression |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108416736A true CN108416736A (en) | 2018-08-17 |
| CN108416736B CN108416736B (en) | 2021-11-16 |
Family
ID=63133325
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810236623.7A Active CN108416736B (en) | 2018-03-21 | 2018-03-21 | Image super-resolution reconstruction method based on secondary anchor point neighborhood regression |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108416736B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110084750A (en) * | 2019-04-12 | 2019-08-02 | 浙江师范大学 | Single image super-resolution method based on multilayer ridge regression |
| CN111402141A (en) * | 2020-03-25 | 2020-07-10 | 中国计量大学 | Tandem single-image super-resolution reconstruction method |
| CN111724303A (en) * | 2020-05-12 | 2020-09-29 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Super-resolution image processing method and system with image type adaptability |
| CN112990013A (en) * | 2021-03-15 | 2021-06-18 | 西安邮电大学 | Time sequence behavior detection method based on dense boundary space-time network |
| US12008732B2 (en) | 2021-03-30 | 2024-06-11 | Sony Group Corporation | Super-resolution of block-compressed texture for texture mapping applications |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008067363A2 (en) * | 2006-11-27 | 2008-06-05 | Nec Laboratories America, Inc. | Soft edge smoothness prior and application on alpha channel super resolution |
| CN107358575A (en) * | 2017-06-08 | 2017-11-17 | 清华大学 | A kind of single image super resolution ratio reconstruction method based on depth residual error network |
| CN107481189A (en) * | 2017-06-28 | 2017-12-15 | 西安邮电大学 | A kind of super-resolution image reconstruction method of the rarefaction representation based on study |
-
2018
- 2018-03-21 CN CN201810236623.7A patent/CN108416736B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008067363A2 (en) * | 2006-11-27 | 2008-06-05 | Nec Laboratories America, Inc. | Soft edge smoothness prior and application on alpha channel super resolution |
| CN107358575A (en) * | 2017-06-08 | 2017-11-17 | 清华大学 | A kind of single image super resolution ratio reconstruction method based on depth residual error network |
| CN107481189A (en) * | 2017-06-28 | 2017-12-15 | 西安邮电大学 | A kind of super-resolution image reconstruction method of the rarefaction representation based on study |
Non-Patent Citations (3)
| Title |
|---|
| JIAN XU 等: "Image superresolution by midfrequency sparse representation and total variation regularization", 《JOURNAL OF ELECTRONIC IMAGING》 * |
| RADU TIMOFTE 等: "Anchored Neighborhood Regression for Fast Example-Based Super-Resolution", 《COMPUTER VISION FOUNDATION》 * |
| 范九伦 等: "基于多锚点邻域回归的图像超分辨率算法", 《西安邮电大学学报》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110084750A (en) * | 2019-04-12 | 2019-08-02 | 浙江师范大学 | Single image super-resolution method based on multilayer ridge regression |
| CN110084750B (en) * | 2019-04-12 | 2023-06-02 | 浙江师范大学 | Single image super-resolution method based on multi-layer ridge regression |
| CN111402141A (en) * | 2020-03-25 | 2020-07-10 | 中国计量大学 | Tandem single-image super-resolution reconstruction method |
| CN111402141B (en) * | 2020-03-25 | 2024-02-09 | 中国计量大学 | Tandem single-image super-resolution reconstruction method |
| CN111724303A (en) * | 2020-05-12 | 2020-09-29 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Super-resolution image processing method and system with image type adaptability |
| CN112990013A (en) * | 2021-03-15 | 2021-06-18 | 西安邮电大学 | Time sequence behavior detection method based on dense boundary space-time network |
| CN112990013B (en) * | 2021-03-15 | 2024-01-12 | 西安邮电大学 | Time sequence behavior detection method based on dense boundary space-time network |
| US12008732B2 (en) | 2021-03-30 | 2024-06-11 | Sony Group Corporation | Super-resolution of block-compressed texture for texture mapping applications |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108416736B (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110119780B (en) | Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network | |
| CN111192200A (en) | Image super-resolution reconstruction method based on fusion attention mechanism residual error network | |
| CN104778671B (en) | A kind of image super-resolution method based on SAE and rarefaction representation | |
| CN110033410A (en) | Image reconstruction model training method, image super-resolution rebuilding method and device | |
| CN108416736A (en) | A kind of image super-resolution rebuilding method returned based on secondary anchor point neighborhood | |
| CN112037131A (en) | Single-image super-resolution reconstruction method based on generation countermeasure network | |
| CN106952228A (en) | The super resolution ratio reconstruction method of single image based on the non local self-similarity of image | |
| CN108805808A (en) | A method of improving video resolution using convolutional neural networks | |
| CN107341765A (en) | A kind of image super-resolution rebuilding method decomposed based on cartoon texture | |
| CN106157244A (en) | A kind of QR Code Image Super-resolution Reconstruction method based on rarefaction representation | |
| CN112270644A (en) | Face super-resolution method based on spatial feature transformation and cross-scale feature integration | |
| Xiao et al. | A dual-UNet with multistage details injection for hyperspectral image fusion | |
| CN114841856A (en) | Image super-pixel reconstruction method of dense connection network based on depth residual channel space attention | |
| CN112767252B (en) | Image super-resolution reconstruction method based on convolutional neural network | |
| CN110349085A (en) | A kind of single image super-resolution feature Enhancement Method based on generation confrontation network | |
| CN111402138A (en) | Image super-resolution reconstruction method of supervised convolutional neural network based on multi-scale feature extraction fusion | |
| CN113781308A (en) | Image super-resolution reconstruction method and device, storage medium and electronic equipment | |
| CN109559278B (en) | Super resolution image reconstruction method and system based on multiple features study | |
| CN114066729A (en) | Face super-resolution reconstruction method capable of recovering identity information | |
| CN103020940B (en) | Local feature transformation based face super-resolution reconstruction method | |
| CN115880158A (en) | Blind image super-resolution reconstruction method and system based on variational self-coding | |
| CN112070669B (en) | Super-resolution image reconstruction method for arbitrary fuzzy core | |
| CN112508786B (en) | Satellite image-oriented arbitrary-scale super-resolution reconstruction method and system | |
| CN114155171A (en) | Image restoration method and system based on intensive multi-scale fusion | |
| CN109242919A (en) | A kind of image down sampling method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |