CN109447903A - A kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model - Google Patents
A kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model Download PDFInfo
- Publication number
- CN109447903A CN109447903A CN201811213803.XA CN201811213803A CN109447903A CN 109447903 A CN109447903 A CN 109447903A CN 201811213803 A CN201811213803 A CN 201811213803A CN 109447903 A CN109447903 A CN 109447903A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- low
- definition picture
- quality evaluation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 33
- 238000001514 detection method Methods 0.000 claims abstract description 21
- 238000000605 extraction Methods 0.000 claims abstract description 10
- 238000009826 distribution Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 abstract description 4
- 238000004422 calculation algorithm Methods 0.000 description 29
- 238000011156 evaluation Methods 0.000 description 14
- 230000000694 effects Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000013210 evaluation model Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 241000208340 Araliaceae Species 0.000 description 2
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000007850 degeneration Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 235000008434 ginseng Nutrition 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000001303 quality assessment method Methods 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20064—Wavelet transform [DWT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model, conspicuousness detection are carried out to low-resolution image and high-definition picture respectively first, to extract the significant characteristics of image；Secondly original image is multiplied with significant characteristics, and DWT variation is carried out to the gained image that is multiplied, extract the high-frequency information in image, and calculate information gain of the high-definition picture relative to low-resolution image；Then the textural characteristics of LBP operator extraction low-resolution image and high-definition picture are used, and combine image notable feature, by histogram come the texture similarity of lower resolution and high-definition picture；The texture similarity that the final information gain obtained in conjunction with step 2 and step 3 obtain, constructs half reference type super-resolution reconstruction image quality evaluation model.
Description
Technical field
The present invention relates to a kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model, in particular to
A kind of singular value decomposition extraction structure and brightness aspect distortion information using gray level image, utilizes the singular value decomposition of quaternary number
Extract the method for building up and device of half reference type super-resolution reconstruction image quality evaluation model of the whole distortion information of color.
Background technique
Image super-resolution refers to that image super-resolution reconfiguration technique is divided into single image according to the quantity of input picture and surpasses
Differentiate reconstruct and the super-resolution reconstruction of multiple images.Single image super-resolution reconstruction refers to using single image as defeated
Enter to obtain high-definition picture, the reconstruct of multiple image super-resolutions refers to multiple low-resolution image conducts under Same Scene
Input is to reconstruct high-definition picture.High-definition picture possesses more details information, higher pixel density and finer and smoother
Image quality.In order to obtain high-definition picture, most simple direct method is exactly using high-resolution camera.But it is actually answering
In, the considerations of in view of engineering cost and manufacturing process, be all not suitable in many situations using high-resolution camera come
Carry out the acquisition of picture signal.Therefore, high-definition picture is obtained by super-resolution reconstruction algorithm to be needed with practical application
It asks.Image super-resolution reconfiguration technique play the role of in many aspects it is very important, such as: computer vision, remote sensing images,
Web page browsing, medical image, high definition television and video monitoring etc..
A large amount of Super-Resolution Image Restoration was suggested in recent years, but how effectively to go to evaluate these algorithms
Performance be always there are the problem of.A kind of simply and effectively method is that subjective evaluation is carried out to image, that is, looks for different people couple
Super-resolution reconstruction image is observed marking, and obatained score finally obtains a score as the image after processing
Mass fraction.Ultimate recipient due to human eye as information such as images and video, this subjective evaluation method is to oversubscription
The evaluation of resolution reconstructed image is trustworthy.However the method for using subjective assessment comments super-resolution reconstruction image
The cost of valence is too high, can devote a tremendous amount of time.Join in addition, subjective evaluation method is not easy to do Super-Resolution Image Restoration
The work of number optimization.Therefore it is directed to for super-resolution image and corresponding restructing algorithm, objective Environmental Evaluation Model right and wrong
It is often significant.
It is very challenging for objectively evaluate to super-resolution reconstruction image, because in practical applications without high quality
High-definition picture be used as reference picture, so the full reference mass evaluation algorithms of some better performances: Y-PSNR
(Peak Signal-to-Noise Ratio, PSNR), structural similarity algorithm (Structural Similarity Index,
) and the difficulties such as Multi-scale model similarity (Multi-Scale Structural Similarity Index, MS-SSIM) SSIM
To carry out effectively evaluating to the super-resolution reconstruction image generated in practical application.Due in image super-resolution restructuring procedure
In bring distortions, such as fuzzy, ringing effect and texture distortion etc..Therefore existing performance comparative superiority is general
Without reference type quality evaluation algorithm: spatial domain non-reference picture quality estimation method (Blind Referenceless Image
Spatial Quality Evaluator, BRISQUE), based on the robust of free energy without reference evaluation algorithms (No-
Reference Free Energy based Robust Metric, NFERM) and based on code book indicate non-reference picture comment
Valence method (Codebook Representation for No-reference Image Assessment, CORNIA) etc.
It is difficult to effective evaluation super-resolution reconstruction image.Therefore it for this completely new problem of super-resolution reconstruction image quality evaluation, grinds
It is significant for studying carefully the quality evaluation algorithm of effective super-resolution reconstruction image and corresponding restructing algorithm.
Summary of the invention
The purpose of the present invention is in view of the above-mentioned problems, proposing a kind of based on based on information gain and texture paging two
Point, half reference type super-resolution reconstruction image quality evaluation model of foundation.
To achieve the above object, the present invention proposes a kind of building for half reference type super-resolution reconstruction image quality evaluation model
Cube method, this method comprises:
Step 1: conspicuousness detection is carried out to low-resolution image and high-definition picture respectively, to extract image
Significant characteristics；
Step 2: original image is multiplied with significant characteristics, and carries out DWT variation to the gained image that is multiplied, and extracts image
In high-frequency information, and calculate information gain of the high-definition picture relative to low-resolution image；
Step 3: using the textural characteristics of LBP operator extraction low-resolution image and high-definition picture, and image is combined
Notable feature, by histogram come the texture similarity of lower resolution and high-definition picture；
Step 4: the texture similarity that the information gain and step 3 obtained in conjunction with step 2 obtains constructs half reference type
Super-resolution reconstruction image quality evaluation model.
Preferably, the conspicuousness in the step 1 detects, so that extracting Saliency maps picture includes following content:
The progress of vision notable feature is carried out to low-resolution image and high-definition picture based on the detection module watched attentively
The calculation formula of the notable feature M (r) extracted is as follows:
Wherein, r indicates that high-definition picture or low-resolution image, rg indicate image of the r after gaussian filtering, and C is normal
Number, P indicate gradient, PxIndicate the gradient in the direction x, PyIndicating the gradient in the direction y, s indicates picture signal,Indicate convolution.
Preferably, the specific steps of the step 2 are as follows:
A, the notable feature M (r) of extraction is negated to obtain weight matrix, using the weight matrix come to former low resolution
Rate image and high-definition picture are weighted to obtain the specific image information of image: W (r)=1-M (r)；
B, obtained specific image is subjected to DWT transformation, obtains horizontal direction high-frequency sub-band LH, high frequency of vertical direction
Band HL, the high-frequency sub-band HH and low frequency sub-band LL of diagonal；
C, the comentropy E of low-resolution image LR and each subband of high-definition picture HR are calculated separately,
Wherein, M and N indicates the picture size of each subband, and i and j indicate the coordinated indexing of image, and P (i, j) indicates image
The wavelet coefficient of each sub-band images after being decomposed through DWT；
D, increase of the high-definition picture compared to the comentropy of LH, HL and HH sub-band images of low-resolution image is calculated
The summation S of amount1；
Wherein, ELHIndicate the comentropy of high-resolution subband LH, EHLIndicate the comentropy of high-resolution subband HL,
EHHIndicate the comentropy of high-resolution subband HH, ELHThe comentropy of the subband LH of ' expression low resolution, EHL' indicate low resolution
The comentropy of the subband HL of rate, EHHThe comentropy of the subband HH of ' expression low resolution.
Preferably, the specific steps of the step 3 are as follows:
A, the texture structure of low-resolution image LP Yu high-definition picture HP, formula are extracted respectively using LBP operator
It is as follows:
Wherein, P and R indicates the quantity and radius in field, gcAnd giIt is the pixel value of center and neighborhood；
B, obtained texture structure and specific image information are combined, obtain the conspicuousness texture structure figure of image
U=LBP (r) W (r)；
C, the conspicuousness texture structure figure that low-resolution image and high-definition picture obtain is carried out pair using histogram
Than obtaining the similarity degree S of texture structure2,
H=hist (U),
Wherein, H is the histogram distribution of conspicuousness texture structure figure U, and LR is low resolution, and HR is high-resolution, and n is indicated
The quantity of histogram, c are constant.
Preferably, the information flow gain in the step 4 in conjunction with high-definition picture compared to low-resolution image,
Texture similarity degree, in conjunction with the visual quality of rear forecast imageWherein α and β is adjusting parameter.
The invention has the benefit that disclosed half reference type super-resolution reconstruction image quality evaluation of one kind
The method for building up of model has a characteristic that this method, as reference information, is existed using low-resolution image compared with other methods
There is feasibility in practical application, for super-resolution reconstruction image in image super-resolution restructuring procedure, picture structure hair
Degeneration is given birth to, distinctive ringing effect and fuzzy etc. is distorted, and goes out Saliency maps picture using the conspicuousness Detection and Extraction of image, and tie
The textural characteristics of image are closed, which meets the detail portion that human eye sensory perceptual system observation super-resolution reconstruction image is more concerned about image
Point, so method is more agreed in the quality of evaluation super-resolution reconstruction image with subjective quality assessment, than universal image
Quality evaluating method is more targeted, and prediction result is more acurrate.
Detailed description of the invention
Fig. 1 is the method for building up process of half reference type super-resolution reconstruction image quality evaluation model proposed by the present invention
Figure；
Fig. 2 be in the present invention information gain to the quality evaluation diagram of high-resolution reconstruction image；
Fig. 3 is low resolution and high-definition picture and corresponding conspicuousness texture structure figure and histogram；
Fig. 4 is parameter alpha and β optimization comparison diagram.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
As shown in Figure 1, disclosed half reference type super-resolution reconstruction image quality evaluation model of one kind is built
Cube method, this method comprises:
Step 1: conspicuousness detection is carried out to low-resolution image LR and high-definition picture HR respectively, to extract
The significant characteristics of image；
Step 2: original image is multiplied with significant characteristics, and carries out DWT variation to the gained image that is multiplied, and extracts image
In high-frequency information, and calculate information gain of the high-definition picture relative to low-resolution image；
Step 3: using the textural characteristics of LBP operator extraction low-resolution image and high-definition picture, and image is combined
Notable feature, by histogram come the texture similarity of lower resolution and high-definition picture；
Step 4: the texture similarity that the information gain and step 3 obtained in conjunction with step 2 obtains constructs half reference type
Super-resolution reconstruction image quality evaluation model.
Vision significance can be improved the performance of quality assessment modules in practical applications, however measured according to eye tracker
Salient region and most of existing conspicuousness technologies are difficult to useful effect in the quality evaluation of super-resolution image (SR)
Task, so the present invention is the conspicuousness detection method based on simulation human eye movement " watch attentively and sweep ", due to SR image
It is the observation method using " watching attentively " mostly, therefore in the present invention only with the inspection for being directed to " watching attentively " when carrying out subjective testing
It surveys module and carries out conspicuousness detection, specifically include:
The progress of vision notable feature is carried out to low-resolution image and high-definition picture based on the detection module watched attentively
The calculation formula of the notable feature M (r) extracted is as follows:
Wherein, r indicates that high-definition picture or low-resolution image, rg indicate image of the r after gaussian filtering, and C is normal
Number, P indicate gradient, PxIndicate the gradient in the direction x, PyIndicating the gradient in the direction y, s indicates picture signal,Indicate convolution；
It is that r input is low-resolution image LR, and obtained conspicuousness is special carrying out the detection of low-resolution image conspicuousness
Sign should be M (LR), and when carrying out the detection of high-definition picture conspicuousness, r input is high-definition picture HR, and what is obtained is aobvious
Work property feature should be M (HR).
Image super-resolution reconstruct is that LR image is reconstructed to generate HR image, with what is obtained from the HR image of reconstruct
More information.It is relatively easy to due in image super-resolution restructuring procedure low frequency part being reconstructed,
What conventional images super-resolution reconstruction algorithm was designed primarily directed to the reconstruct of high-frequency information.The present invention is also for HR image phase
Information gain module is devised compared with the information incrementss of the high frequency section of LR image, specific as follows:
A, the notable feature M (r) of extraction is negated to obtain weight matrix, using the weight matrix come to former low resolution
Rate image and high-definition picture are weighted to obtain the specific image information of correspondence image: W (r)=1-M (r)；
B, obtained specific image is subjected to DWT transformation, obtains horizontal direction high-frequency sub-band LH, high frequency of vertical direction
Band HL, the high-frequency sub-band HH and low frequency sub-band LL of diagonal；
C, the comentropy E of low-resolution image LR and each subband of high-definition picture HR high frequency section are calculated separately,
Wherein, the picture size of each subband (LH, HL, HH) of M and N expression, the coordinated indexing of i and j expression image, P (i,
J) wavelet coefficient for indicating each sub-band images after image is decomposed through DWT, when carrying out high-definition picture calculating, obtained information
Entropy includes；ELHThe comentropy of high-resolution subband LH, EHLThe comentropy of high-resolution subband HL, EHHIt is high-resolution
The comentropy of subband HH；When carrying out low-resolution image calculating, obtained comentropy includes: ELHThe subband LH of '-low resolution
Comentropy, EHLThe comentropy of the subband HL of '-low resolution, EHHThe comentropy of the subband HH of '-low resolution；
D, increase of the high-definition picture compared to the comentropy of LH, HL and HH sub-band images of low-resolution image is calculated
The summation S of amount1；
For verification information gain model S1Super-resolution reconstruction image can be effectively assessed, using information gain model
S1Quality evaluation is carried out to the HR image for the different quality that same width LR image generates, experimental result is as shown in Figure 2.Wherein
One behavior LR image and corresponding HR image, the corresponding the first row image of the second behavior are generated through the weighting of conspicuousness module
Image.Figure it is seen that the HR image of different quality, the image generated after the weighting of conspicuousness module has brighter
Aobvious difference, this is also information gain module S1Super-resolution reconstruction image can be effectively evaluated to lay a good foundation.With HR
Image MOS is worth increasing, information gain model S1Predicted value also increase with it.It is indicated above that information proposed by the invention
Gain model can effectively evaluate super-resolution reconstruction image.
Furthermore detect whether have facilitation to information gain model to verify conspicuousness.The present invention is in ECCV-2011
Information gain model is tested in super-resolution reconstruction image library respectively and conspicuousness detection is combined with information gain model
Evaluation model experimental result is as shown in table 1:
Table 1
As it can be seen from table 1 being carried out only with information gain detection model to ECCV-2011 super-resolution reconstruction image library
Assessment, KRCC, SRCC, PLCC and RMSE have respectively reached 0.6082,0.8078,0.7880 and 1.1860.However it uses and is based on
When the information gain detection model of conspicuousness detection carries out quality evaluation to super-resolution reconstruction image library, KRCC, SRCC, PLCC
0.6257,0.8180,0.8173 and 1.1100 have been respectively reached with RMSE.It is clear that the information based on conspicuousness detection increases
Beneficial detection model has significant raising compared to only with information gain model in performance.Both illustrating to combine can be more
Add the performance for effectively evaluating super-resolution reconstruction image and corresponding super-resolution reconstruction algorithm.
Image super-resolution reconstructs while obtaining more image informations, is still to make the texture structure of HR image as far as possible
It is consistent with the texture structure of LR image.Since different Super-Resolution Image Restorations and different amplification factors are right
When LR image carries out super-resolution reconstruction, the texture structure of the HR image of generation also has different journeys compared with the texture of LR image
The degeneration of degree.So can be effectively to super-resolution reconstruction image according to the texture structure difference between HR image and LR image
It is assessed.Therefore the texture similarity degree proposition of the HR image generated the present invention is based on LR image and super-resolution reconstruction algorithm
A kind of evaluation module for super-resolution reconstruction image, including following content:
A, the texture structure of low-resolution image LP Yu high-definition picture HP, formula are extracted respectively using LBP operator
It is as follows:
Wherein, P and R indicates the quantity and radius in field, gcAnd giIt is the pixel value of center and neighborhood；
B, obtained texture structure and specific image information are combined, obtain the conspicuousness texture structure figure of image
U=LBP (r) W (r)；
C, the conspicuousness texture structure figure that low-resolution image and high-definition picture obtain is carried out pair using histogram
Than obtaining the similarity degree S of texture structure2,
H=hist (U),
Wherein, H is the histogram distribution of conspicuousness texture structure figure U, and LR is low resolution, and HR is high-resolution, and n is indicated
The quantity of histogram, c are constant, obtained S2Texture similarity degree as between LR and HR image, LR image and HR image
Texture structure it is more similar, S2Value it is bigger, the quality of the HR image reconstructed is also better, and vice versa.
In order to verify texture paging feature S2Validity, Fig. 3 lists an example, Fig. 3 the first behavior LR image
And the HR image of the different quality of super-resolution reconstruction generation is carried out by it.Second behavior the first row LR and HR image it is significant
Property texture structure figure, the histogram distribution of the corresponding second row conspicuousness texture structure figure of third behavior.Not homogeneity as we know from the figure
The conspicuousness texture structure figure of the HR image of amount has apparent difference, in order to more intuitively compare the HR image of different quality
Texture structure and LR image texture structure otherness, the third line lists the histogram distribution of texture structure figure.It is different
The most apparent part of the texture structure histogram difference of HR image and corresponding LR image is irised out in figure with wire frame.Therefrom may be used
Texture structure histogram distribution to find the better HR image of quality is more similar to the texture structure histogram of LR image.It does not enclose
Part variation out is relatively small, but on the whole or the better HR image of quality texture structure figure histogram distribution with
The distribution similarity degree of the texture structure histogram of its corresponding LR image is higher.The mos of 3 width HR images shown in Fig. 3 the first row
Value is followed successively by 5.65,3.85 and 2.00, corresponding S2Value be respectively 0.8656,0.7762 and 0.7217.It can be seen that with
The variation of HR picture quality, S2Value become smaller accordingly, illustrate that LR image is lower with the texture similarity degree of HR image.Pass through
The texture structure similarity feature that above-mentioned description of test this section is mentioned can effectively evaluate super-resolution reconstruction image.
In addition, in order to verify the necessity that the texture structure scale model that this section is proposed is combined with conspicuousness module.
Test texture structure scale model and conspicuousness detection and texture knot respectively in ECCV-2014 super-resolution reconstruction image library
The evaluation model that structure similar modular blocks combine, experimental result are as shown in table 2:
Table 2
From table 2 it can be seen that commenting only with texture scale model ECCV-2014 super-resolution reconstruction image library
Estimate, KRCC, SRCC, PLCC and RMSE have respectively reached 0.4249,0.6067,0.5966 and 1.5461.However it uses based on aobvious
Write property detection texture structure scale model to super-resolution reconstruction image library carry out quality evaluation when, KRCC, SRCC, PLCC and
RMSE has respectively reached 0.4728,0.6602,0.6677 and 1.4342.It is clear that the texture structure based on conspicuousness detection
Scale model has significant raising compared to only with texture structure scale model in performance.Both illustrating to combine can
More efficiently evaluate super-resolution reconstruction image.
Finally, the information flow gain in conjunction with high-definition picture compared to low-resolution image, texture similarity degree, in conjunction with
The visual quality of forecast image afterwardsWherein α and β is adjusting parameter, and in ECCV-2014 image library, we are to ginseng
Number α and β has carried out Verification.α first is set as 1, and parameter optimization, experiment knot are then carried out in section [0,1] to parameter beta
Fruit is as shown in Figure 4.The result shows that the two is of equal importance, i.e. α and β and all it is set as 1.
Experimental verification is carried out to the method in the present invention below
Half reference type algorithm proposed by the invention is tested for the property in ECCV-2014 image library.Image library ECCV-
LR image in 2014 is to be generated by the image of high quality through nine different types of down-samplings and fuzzy combined treatment, then again by
Six kinds of Super-Resolution Image Reconstructions, which are reconstructed, generates HR image, therefore ECCV-2014 image library provides high quality
Image is as reference picture.In order to verify the validity and superiority of half reference mass evaluation algorithms proposed by the invention,
We are by the full reference type algorithm of itself and mainstream in ECCV-2014 image library: PSNR, SSIM, MSSSIM, FSIM, IWSSIM,
VIF, MAD and GSM, general no reference type algorithm: BRISQUE, NFERM, BIQI, DIIVINE, BLLINDSII, NIQE and
DESIQUE, and the quality evaluation algorithm proposed for super-resolution reconstruction image compare, as a result referring to the following table 3
~5, table 3 is performance of the full reference type quality module on the library ECCV-2014, and table 4 is no reference type quality module in ECCV-
Performance on 2014 libraries.
Table 3
| Algorithm | KRCC | SRCC | PLCC | RMSE |
| PSNR | 0.2068 | 0.3020 | 0.3498 | 1.8048 |
| SSIM | 0.3800 | 0.5308 | 0.5817 | 1.5670 |
| FSIM | 0.3928 | 0.5605 | 0.6586 | 1.4496 |
| IWSSIM | 0.5936 | 0.7863 | 0.8538 | 1.0028 |
| VIF | 0.6242 | 0.8130 | 0.8543 | 1.0041 |
| MAD | 0.5056 | 0.6972 | 0.7528 | 1.2682 |
| GSM | 0.3050 | 0.4420 | 0.5494 | 1.6097 |
| MSSSIM | 0.4686 | 0.6403 | 0.6802 | 1.4122 |
| Proposed(RR) | 0.6558 | 0.8503 | 0.8321 | 1.0686 |
Table 4
The performance of various image quality evaluation algorithms is tested using tetra- measurement standards of KRCC, SRCC, PLCC and RMSE.
KRCC and SRCC indicates the monotonicity of image quality evaluation model, by the way that Subjective and objective qualities score is directly calculated.
Two standards of PLCC and RMSE are used to measure the accuracy of image quality evaluation algorithm.Using non-linear transform function to image matter
Amount evaluation algorithms carry out image to predict that resulting objective quality scores are converted, and then calculate subjective quality scores and process
Objective quality scores after conversion obtain two values of PLCC and RMSE.
The performance of full reference type algorithm VIF is most from experimental result as can be seen that on image library ECCV-2014, in table 3
Good, the value of KRCC, SRCC, PLCC and RMSE have respectively reached 0.6242,0.8130,0.8543 and 1.0041, without ginseng in table 4
The value for examining KRCC, SRCC, PLCC and RMSE of type algorithm BRISQUE is respectively 0.6280,0.8045,0.8818 and 0.9221,
Optimal result is achieved in terms of accuracy compared to other algorithms.Obtained by half reference mass evaluation algorithms proposed by the invention
Tetra- indexs of KRCC, SRCC, PLCC and RMSE be respectively 0.6558,0.8503,0.8321 and 1.0686.In conjunction with 3 He of table
The mentioned quality evaluation algorithm of the present invention achieves optimal effect in terms of monotonicity known to table 4, is slightly inferior in terms of accuracy
The evaluation algorithms of the most preferably full reference type of performance, it is just poor much compared with based on trained quality evaluating method BRISQUE.But by
It is worked in the quality evaluation of image primarily to being ranked up and being not only to one accurately to the quality of a series of images
Score, thus compared to accuracy for, the monotonicity of algorithm seems even more important.And the full supervision type quality based on study
There is over-fitting when testing in image library in evaluation algorithms, do not have good generalization ability.
It although an embodiment of the present invention has been shown and described, for the ordinary skill in the art, can be with
A variety of variations, modification, replacement can be carried out to these embodiments without departing from the principles and spirit of the present invention by understanding
And modification, the scope of the present invention is defined by the appended.
Claims (5)
1. a kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model, characterized by comprising:
Step 1: carrying out conspicuousness detection to low-resolution image and high-definition picture respectively, to extract the aobvious of image
Work property feature；
Step 2: original image is multiplied with significant characteristics, and carries out DWT variation to the gained image that is multiplied, and is extracted in image
High-frequency information, and calculate information gain of the high-definition picture relative to low-resolution image；
Step 3: using the textural characteristics of LBP operator extraction low-resolution image and high-definition picture, and combine image significant
Feature, by histogram come the texture similarity of lower resolution and high-definition picture；
Step 4: the texture similarity that the information gain and step 3 obtained in conjunction with step 2 obtains constructs half reference type oversubscription
Resolution reconstructed image quality evaluation model.
2. a kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model according to claim 1,
It is characterized by: the conspicuousness in the step 1 detects, so that extracting Saliency maps picture includes following content:
Vision notable feature is carried out to low-resolution image and high-definition picture based on the detection module watched attentively to extract,
The calculation formula of the notable feature M (r) of extraction is as follows:
Wherein, r indicates that high-definition picture or low-resolution image, rg indicate image of the r after gaussian filtering, and C is constant, P
Indicate gradient, PxIndicate the gradient in the direction x, PyIndicating the gradient in the direction y, s indicates picture signal,Indicate convolution.
3. a kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model according to claim 2,
It is characterized by: the specific steps of the step 2 are as follows:
A, the notable feature M (r) of extraction is negated to obtain weight matrix, using the weight matrix come to former low resolution figure
Picture and high-definition picture are weighted to obtain the specific image information of image: W (r)=1-M (r)；
B, obtained specific image is subjected to DWT transformation, obtains horizontal direction high-frequency sub-band LH, the high-frequency sub-band of vertical direction
HL, the high-frequency sub-band HH and low frequency sub-band LL of diagonal；
C, the comentropy E of low-resolution image LR and each subband of high-definition picture HR are calculated separately,
Wherein, M and N indicates the picture size of each subband, and i and j indicate the coordinated indexing of image, and P (i, j) indicates image warp
The wavelet coefficient of each sub-band images after DWT is decomposed；
D, incrementss of the high-definition picture compared to the comentropies of LH, HL and HH sub-band images of low-resolution image are calculated
Summation S1:
Wherein, ELHIndicate the comentropy of high-resolution subband LH, EHLIndicate the comentropy of high-resolution subband HL, EHHTable
Show the comentropy of high-resolution subband HH, ELHThe comentropy of the subband LH of ' expression low resolution, EHL' indicate low resolution
The comentropy of subband HL, EHHThe comentropy of the subband HH of ' expression low resolution.
4. a kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model according to claim 3,
It is characterized by: the specific steps of the step 3 are as follows:
A, the texture structure of low-resolution image LP Yu high-definition picture HP are extracted respectively using LBP operator, formula is as follows:
Wherein, P and R indicates the quantity and radius in field, gcAnd giIt is the pixel value of center and neighborhood；
B, obtained texture structure and specific image information are combined, obtain the conspicuousness texture structure figure of image
U=LBP (r) W (r)；
C, the conspicuousness texture structure figure that low-resolution image and high-definition picture obtain is compared using histogram, is obtained
To the similarity degree S of texture structure2,
H=hist (U),
Wherein, H is the histogram distribution of conspicuousness texture structure figure U, and LR is low resolution, and HR is high-resolution, and n indicates histogram
The quantity of figure, c are constant.
5. a kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model according to claim 4,
It is characterized by: combining information flow gain of the high-definition picture compared to low-resolution image, texture phase in the step 4
Like degree, in conjunction with the visual quality of rear forecast imageWherein α and β is adjusting parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811213803.XA CN109447903B (en) | 2018-10-17 | 2018-10-17 | Method for establishing semi-reference super-resolution reconstruction image quality evaluation model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811213803.XA CN109447903B (en) | 2018-10-17 | 2018-10-17 | Method for establishing semi-reference super-resolution reconstruction image quality evaluation model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109447903A true CN109447903A (en) | 2019-03-08 |
| CN109447903B CN109447903B (en) | 2022-11-18 |
Family
ID=65546625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811213803.XA Active CN109447903B (en) | 2018-10-17 | 2018-10-17 | Method for establishing semi-reference super-resolution reconstruction image quality evaluation model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109447903B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110415242A (en) * | 2019-08-02 | 2019-11-05 | 中国人民解放军军事科学院国防科技创新研究院 | A kind of super-resolution multiplying power evaluation method based on reference picture |
| CN113298337A (en) * | 2020-10-19 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Quality evaluation method and device |
| CN113436078A (en) * | 2021-08-10 | 2021-09-24 | 诺华视创电影科技(江苏)有限公司 | Self-adaptive image super-resolution reconstruction method and device |
| CN117495854A (en) * | 2023-12-28 | 2024-02-02 | 淘宝（中国）软件有限公司 | Video data processing method, device and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825500A (en) * | 2016-03-10 | 2016-08-03 | 江苏商贸职业学院 | Camera image quality evaluation method and device |
-
2018
- 2018-10-17 CN CN201811213803.XA patent/CN109447903B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825500A (en) * | 2016-03-10 | 2016-08-03 | 江苏商贸职业学院 | Camera image quality evaluation method and device |
Non-Patent Citations (6)
| Title |
|---|
| KE GU: "Saliency-Guided Quality Assessment of Screen Content Images", 《IEEE TRANSACTIONS ON MULTIMEDIA》 * |
| LIJUAN TANG: "A reduced-reference quality assessment metric for super-resolution reconstructed images with information gain and texture similarity", 《SIGNAL PROCESSING: IMAGE COMMUNICATION》 * |
| LIXIONG LIU等: "No-reference image quality assessment based on spatial and spectral entropies", 《SIGNAL PROCESSING: IMAGE COMMUNICATION》 * |
| 王小怡等: "超分辨率重建图像质量评价算法分析", 《信息通信》 * |
| 范赐恩等: "颜色空间统计联合纹理特征的无参考图像质量评价", 《光学精密工程》 * |
| 黄慧娟等: "基于SVD的超分辨率重建图像质量无参考评价方法", 《计算机辅助设计与图形学学报》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110415242A (en) * | 2019-08-02 | 2019-11-05 | 中国人民解放军军事科学院国防科技创新研究院 | A kind of super-resolution multiplying power evaluation method based on reference picture |
| CN110415242B (en) * | 2019-08-02 | 2020-05-19 | 中国人民解放军军事科学院国防科技创新研究院 | Super-resolution magnification evaluation method based on reference image |
| CN113298337A (en) * | 2020-10-19 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Quality evaluation method and device |
| CN113436078A (en) * | 2021-08-10 | 2021-09-24 | 诺华视创电影科技(江苏)有限公司 | Self-adaptive image super-resolution reconstruction method and device |
| CN117495854A (en) * | 2023-12-28 | 2024-02-02 | 淘宝（中国）软件有限公司 | Video data processing method, device and storage medium |
| CN117495854B (en) * | 2023-12-28 | 2024-05-03 | 淘宝（中国）软件有限公司 | Video data processing method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109447903B (en) | 2022-11-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ciancio et al. | No-reference blur assessment of digital pictures based on multifeature classifiers | |
| CN109447903A (en) | A kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model | |
| Li et al. | No-reference quality assessment of deblocked images | |
| Zarić et al. | VCL@ FER image quality assessment database | |
| KR101664913B1 (en) | Method and system for determining a quality measure for an image using multi-level decomposition of images | |
| Liu et al. | A no-reference metric for perceived ringing artifacts in images | |
| Liu et al. | A perceptually relevant no-reference blockiness metric based on local image characteristics | |
| Saha et al. | Utilizing image scales towards totally training free blind image quality assessment | |
| CN101562675B (en) | No-reference image quality evaluation method based on Contourlet transform | |
| CN101482973B (en) | Partial reference image quality appraisement method based on early vision | |
| CN104123705B (en) | A kind of super-resolution rebuilding picture quality Contourlet territory evaluation methodology | |
| Yeganeh et al. | Objective quality assessment for image super-resolution: A natural scene statistics approach | |
| CN105049851A (en) | Channel no-reference image quality evaluation method based on color perception | |
| Tang et al. | A reduced-reference quality assessment metric for super-resolution reconstructed images with information gain and texture similarity | |
| Moorthy et al. | Visual perception and quality assessment | |
| CN102036098B (en) | Full-reference type image quality evaluation method based on visual information amount difference | |
| Li et al. | Perceptual quality evaluation for image defocus deblurring | |
| CN105631890B (en) | Picture quality evaluation method out of focus based on image gradient and phase equalization | |
| CN104021523A (en) | Novel method for image super-resolution amplification based on edge classification | |
| Li et al. | GridSAR: Grid strength and regularity for robust evaluation of blocking artifacts in JPEG images | |
| Li et al. | A natural image quality evaluation metric | |
| Feng et al. | Real-world non-homogeneous haze removal by sliding self-attention wavelet network | |
| Moreno et al. | Towards no-reference of peak signal to noise ratio | |
| Liao et al. | An image quality assessment algorithm based on dual-scale edge structure similarity | |
| Wang et al. | No-Reference Quality Assessment of Super-Resolution Reconstructed Images by Incorporating Domain Knowledge. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231123 Address after: Room 106, Building 30, Jiangjingyuan, No. 52 Yaogang Road, Chongchuan District, Nantong City, Jiangsu Province, 226000 Patentee after: Nantong Daguo Intelligent Technology Co.,Ltd. Address before: No.48 jiangtongdao Road, Nantong City, Jiangsu Province, 226000 Patentee before: JIANGSU VOCATIONAL College OF BUSINESS |