CN105160653A - Quality evaluation method for fog-degraded images - Google Patents

Abstract

The invention discloses a quality evaluation method for fog-degraded images. The method researches fog as a distortion type. A codebook-based design idea is adopted. A codebook suitable for fog-degraded image features is constructed for achieving quality evaluation on fog-degraded images. The method comprises the following steps of extracting features of training images in a block way; using an improved K-means algorithm to cluster extracted features, finishing constructing a codebook, using the well constructed codebook to encode the training image blocks; using a pooling strategy to extract feature vectors of the training images on an encoded system matrix, making the feature vectors of the training images and training image subjective scoring put in an SVR for training and acquiring a regression model; and further acquiring quality scores of test images.

Description

A kind of quality evaluating method for Misty Image

Technical field

The present invention relates to a kind of method of technical field of image processing, specifically a kind of quality evaluating method for Misty Image.

Background technology

Along with the generation equipment of ubiquitous digital picture and the network service of fast development, digital picture has become people breath of life in daily life.Such as: easily can find a large amount of images in Baidu, Google.These images often can be subject to the interference such as noise, fuzzy, JPEG/J2K compression, and can cause the decline of quality, the decline degree how weighing picture quality is a current hot research direction.What therefore image quality evaluation became in image procossing is more and more important.

A lot of colleges and universities and scientific research institution is had to participate in the research of image quality evaluation at present, scholar representative is abroad Canadian University of Waterloo (CA) Waterloo, Ontario, N2L3GI Canada associate professor WangZhou and Texas ,Usa college professor Bovik, 2004, they first by some structural informations (contrast, brightness, structural similarity) of image for full reference image quality appraisement, achieve extraordinary effect, make image quality evaluation enter fast-developing epoch.

Image quality evaluating method focus turns to non-reference picture quality appraisement field day by day in recent years, and to be a class remove without any need for the priori (i.e. reference picture) of image the algorithm evaluating distorted image to non-reference picture quality appraisement.Non-reference picture quality appraisement can be divided into two classes: based on certain distortion with not based on certain distortion.Based on the method for certain distortion, namely this kind of is known without the type of distortion with reference to algorithm evaluation.Such as: WangZhou etc. propose the algorithm for evaluating JPEG distortion.The algorithm that Sheikh etc. propose based on NSS is used for evaluating J2K distortion.But the type of distortion of image is unknown in most cases, is difficult to determine that it is applicable to above any nothing with reference to algorithm, which limits the range of application of these algorithms.Not based on the method for certain distortion, i.e. the type of distortion of this kind of algorithm evaluation is unknown, is general non-reference picture quality appraisement algorithm.Equally also it can be divided into two classes: OA-DU (opinionawaredistortionunaware), OU-DU (opinionunawaredistortionunaware).OA-DU refers to that this kind of algorithm needs the subjective scoring of training image in the training process, such as: BIQI, BRISQUE, CBIQ, CORNIA etc.OU-DU refers to that this kind of algorithm does not need the subjective scoring of training image.Such as: QAC algorithm, NIQE algorithm.Although QAC algorithm does not use the subjective scoring of image, but it utilizes the full objective evaluation result providing training image with reference to algorithm FSIM, this is equivalent to construct " subjective scoring " to training image, the advantage of this algorithm does not need to utilize the image in existing image quality evaluation database to train, and the scope of training image can be expanded.Training image selected by NIQE algorithm is all undistorted reference picture, and be equivalent to construct one " reference picture " by training, its training image also can not be chosen from existing image quality evaluation database.The advantage of this kind of algorithm does not rely on image quality evaluation database, strong adaptability.

In non-reference picture quality appraisement field, Statistical Learning Theory occupies extremely important status, Bovik in 2010 etc. propose BIQI algorithm, first passage uses Statistical Learning Theory to obtain general non-reference picture quality appraisement algorithm, makes non-reference picture quality appraisement obtain significant progress.In the text, propose " two step Frame Theories " namely: first feature is extracted to training image, then trained training image feature and image subjective scoring by Statistical Learning Theory method, the proposition of this " two step Frame Theories " indicates that Statistical Learning Theory obtains successfully in general non-reference picture quality appraisement field just.

This type of algorithm is generally made up of following process:

(1) feature extraction: by converting image, extracts the proper vector that can reflect image information.This step has great importance in this type of image quality evaluation algorithm, and the quality of Feature Selection directly affects the evaluation result of algorithm and algorithm evaluation map as consuming time.

(2) study prediction: by the study such as neural network, SVR, a model is obtained to obtained feature.The quality of the model prediction unknown images of recycling study.This is the core of this type of algorithm, due to the introducing by these Statistical Learning Theory, make non-reference picture quality appraisement algorithm no longer need to focus in a certain concrete distortion, this considerably increases the robustness of non-reference picture quality appraisement algorithm, and practical application.

(3) experimental analysis: to propose algorithm carry out database test and and other algorithms contrast.

Although the image quality evaluation algorithm of the Corpus--based Method theories of learning comparatively early proposed achieves successfully, also there is the problem that this is certain, as: algorithm complex is higher, and the accuracy of algorithm is inadequate.In order to address this problem, 2011, YePeng etc. proposed the non-reference picture quality appraisement algorithm based on code book, and within 2012, YePeng improves this algorithm, propose CORNIA algorithm, semi-supervised learning is incorporated in image quality evaluation by this algorithm first.And employ soft coding in an encoding process, this coded system is better than hard that algorithm employs above and encodes.

China starts late in image quality evaluation field, but also achieves certain achievement.At present, in 2D image quality evaluation comparison of results outstanding be Tongji University, Xian Electronics Science and Technology University and The Hong Kong Polytechnic University.In 3D rendering quality assessment starting comparatively early be University Of Tianjin and the National University of Defense technology.

At present domestic to non-reference picture quality appraisement research be mainly divided into two classes: the first kind is general non-reference picture quality appraisement, rarefaction representation is used for image quality evaluation by the high-new ripples of domestic scholars in 2012 etc. first, propose a kind of general image quality evaluation algorithm, and achieve reasonable effect.The Hong Kong Polytechnic University scholar ZhangLei in 2013 etc. propose QAC algorithm and are incorporated into by code book first in OU-DU (training image subjective scoring is unknown, and type of distortion is unknown).Equations of The Second Kind is to particular type image quality evaluation.Wherein representative is image quality evaluation after mist elimination, and Guo's Fan and Yu Jing have delivered the algorithm for comprehensive evaluation Misty Image recovery effect respectively on robotization journal and Chinese image graphics journal.

Current image quality evaluation mainly focuses on non-reference picture quality appraisement, and achieves larger progress.But there is a lot of local deficiency in current research, is mainly manifested in following three aspects:

(1) for type of distortion limited

Current non-reference picture quality appraisement algorithm is all based on existing image quality evaluation database, but image quality evaluation database only has several types distortion at present.And mist is a kind of special distortion, existing without poor with reference to the evaluation effect of algorithm to it.

(2) existing method can not evaluate non-natural scene image

The image that non-natural scene image mainly refers to by Practical computer teaching or image does not have natural scene statistical property after processing image.Existing non-reference picture quality appraisement method is all for evaluating natural image, and these methods are all much utilize some characteristics of natural image to propose, for non-natural images as: the evaluation effects such as the P figure in Quick Response Code, polarization image are poor.

(3) shortage of image quality evaluation database

Current image quality evaluation database only has several type of distortion, to some special images as: Misty Image, Quick Response Code etc. lack database, make like this algorithm train and test time lack foundation accurately.

Summary of the invention

The present invention relates to the main contents of two aspects:

(1) Misty Image feature extraction and code book build

Less about the research of Misty Image Feature Selection at present, how to choose one or more features, can reflect that the change of mistiness degree and the code book built for Misty Image quality assessment will be the main contents of inventing to greatest extent.

(2) Misty Image quality assessment

For the deficiency of conventional images quality evaluation algorithm in Misty Image quality assessment, invented the quality evaluating method of the applicable Misty Image based on composite character code book, experiment shows the method superior performance of inventing.

Accompanying drawing explanation

Accompanying drawing 1 is a kind of Misty Image quality score forecasting process schematic diagram of the present invention

Accompanying drawing 2 is a kind of Misty Image quality assessment SVR training process schematic diagram of the present invention

Embodiment

A quality evaluating method for applicable Misty Image, comprises following step: feature extraction, code book structure and coding, SVR model training, quality score are predicted.

(1) feature extraction

The feature extracted is the characteristic that composite character can reflect Misty Image preferably.Natural scene statistical nature can well the distortion of picture engraving, in order to obtain the proper vector for building code book, select generalized Gaussian distribution (GGD) and asymmetric generalized Gaussian distribution (AGGD) to Misty Image extremely distorted image effectively portray.Such as formula (1)-(3):

$f(x;\mathrm{\α},{\mathrm{\σ}}^2)=\frac{\mathrm{\α}}{2\mathrm{\β\Γ}(1/\mathrm{\α})}\exp (-{\left(\frac{\left|x\right|}{\mathrm{\β}}\right)}^{\mathrm{\α}})$

$\mathrm{\β}=\mathrm{\σ}\sqrt{\frac{\mathrm{\Γ}(1/\mathrm{\α})}{\mathrm{\Γ}(3/\mathrm{\α})}}---\left(2\right)$

Wherein Γ (.) is with minor function:

$\mathrm{\Γ}\left(a\right)\text{=}{\∫}_0^{\∞}{t}^{a-1}{e}^{-t}\mathrm{dt},a>0$

Wherein form parameter α controls the shape of distribution, simultaneously σ ²control variance.Because the distribution of MSCN (MeanSubtractedContrastNormalized) coefficient is dynamic, so select zero-mean distribution.Two parameters (α, the σ of GGD ²) can effectively be estimated.The MSCN coefficients statistics figure of matching undistorted image and corresponding distorted image is carried out with this parameter model.Above-mentioned Two Variables is two parameters of picture engraving distortion.

A kind of structure of rule is had between adjacent coefficient, this rule changes along with adding distortion, this section describes the structure of this rule between neighbor from four direction: level (H), vertically (V), principal diagonal (D1), auxiliary diagonal (D2).

$H(i,j)=\hat{I}(i,j)\hat{I}(i,j+1)---\left(4\right)$

$V(i,j)=\hat{I}(i,j)\hat{I}(i+1,j)---\left(5\right)$

$D1(i,j)=\hat{I}(i,j)\hat{I}(i+1,j+1)---\left(6\right)$

$D2(i,j)=\hat{I}(i,j)\hat{I}(i+1,j-1)---\left(7\right)$

Wherein i ∈ (1,2 ... M), j ∈ (1,2 ... N).Under gaussian coefficient model, suppose that MSCN coefficient is 0 average unit variance distribution, this result obeys following distribution:

$f(x,p)=\frac{\exp \left(\frac{\left|x\right|\mathrm{\ρ}}{1-{\mathrm{\ρ}}^2}\right)K_0\left(\frac{\left|x\right|}{1-{\mathrm{\ρ}}^2}\right)}{\mathrm{\π}\sqrt{1-{\mathrm{\ρ}}^2}}---\left(8\right)$

F in formula (8) is an asymmetric probability density function, and ρ represents the degree of correlation of adjacent coefficient, K ₀it is the Bessel's function of a distortion.This density function can carry out good modeling to of an adjacent coefficient distribution, only has a parameter in formula (8), so can not carry out a good modeling to the adjacent coefficient of distorted image.In order to better carry out modeling, adopt asymmetric generalized Gaussian distribution (AGGD) model:

$f(x;\mathrm{\γ},{\mathrm{\σ}}_l^2,{\mathrm{\σ}}_r^2)=\left\{\begin{array}{c}\frac{\mathrm{\γ}}{({\mathrm{\β}}_l+{\mathrm{\β}}_i)\mathrm{\Γ}\left(\frac{1}{r}\right)}\exp (-{\left(\frac{-x}{{\mathrm{\β}}_l}\right)}^{\mathrm{\γ}})\∀x\≤0\\ \frac{\mathrm{\γ}}{({\mathrm{\β}}_l+{\mathrm{\β}}_i)\mathrm{\Γ}\left(\frac{1}{\mathrm{\γ}}\right)}\exp (-{\left(\frac{x}{{\mathrm{\β}}_i}\right)}^{\mathrm{\γ}})\∀\≥0\end{array}\right.---\left(9\right)$

${\mathrm{\β}}_l={\mathrm{\σ}}_l\sqrt{\frac{\mathrm{\Γ}\left(\frac{1}{v}\right)}{\mathrm{\Γ}\left(\frac{3}{v}\right)}}---\left(10\right)$

${\mathrm{\β}}_r={\mathrm{\σ}}_r\sqrt{\frac{\mathrm{\Γ}\left(\frac{1}{v}\right)}{\mathrm{\Γ}\left(\frac{3}{v}\right)}}---\left(11\right)$

The shape that form parameter σ control AGGD distributes, when time, it is identical that now AGGD distribution distributes with GGD.The parameter of AGGD effectively can estimate with instantaneous matching method.

The average of distribution is also one of parameter, represents by formula (12):

$\mathrm{\η}=({\mathrm{\β}}_r-{\mathrm{\β}}_l)\frac{\mathrm{\Γ}\left(\frac{2}{\mathrm{\γ}}\right)}{\mathrm{\Γ}\left(\frac{1}{\mathrm{\γ}}\right)}---\left(12\right)$

Formula (4)-(7) give the four direction of image pixel, use formula (9) and formula (12) to extract four features in each direction add two parameters, 18 parameters altogether of GGD.Specific features information is in table 1.

Table 118 dimensional feature is specifically formed

Feature number	Feature interpretation	Extracting method
			f 1-f 2	Shape and variance	GGD fitting formula 1 extracts
f 3-f 6	Average, shape, left is poor, right is poor	H direction AGGD fitting formula (9), (12) are extracted
			f 7-f 10	Average, shape, left is poor, right is poor	V direction AGGD fitting formula (9), (12) are extracted
f 11-f 14	Average, shape, left is poor, right is poor	D1 direction AGGD fitting formula (9), (12) are extracted
			f 15-f 18	Average, shape, left is poor, right is poor	D2 direction AGGD fitting formula (9), (12) are extracted

Natural image has multiple dimensioned property, distortion can change the structural information of image on different scale, when natural image is in the information extraction of two or more yardstick, the quantity of information extracted can not produce significant change, so select 2 yardsticks (yardstick is the original scale of image, another yardstick be down-sampling after graphical rule) the upper feature extracting image.Table 1 can be found out be extracted 18 dimensional features on each yardstick, altogether forms the proper vectors of 36 dimensions.With the natural scene statistical nature of the Characterizations image of this 36 dimension.

The feature used is natural scene statistics and contrast, gradient, luminance mix feature, and the dimension of feature is 39 dimensions.In table 2.By this feature referred to as composite character.

Table 2 composite character

Feature number	Feature interpretation	Extracting method
			f 1-f 36	Natural scene statistical nature	Each yardstick on two yardsticks
f 37	Brightness	Formula 12 extracts
			f 38	Contrast	Formula 14 extracts
f 39	Gradient	Formula 17 extracts

Image is divided into the block of B × B size, in each block, extracts the composite character F=[f of block image ₁, f ₂, f ₃..., f ₃₉] forming a proper vector, entire image forms X=[F ₁, F ₂, F ₃, F _n]. wherein, F ₁f _nrepresent the composite character extracted in each fritter, and be all column vector.

(2) code book builds and coding

It is improvement K-means clustering algorithm that code book builds the clustering algorithm used, traditional K-means clustering algorithm is random to choosing of initial cluster center, provided this shortcoming choosing initial cluster center in last chapter, improving K-means clustering algorithm is fixing when choosing initial cluster center.Lower part provides computation process:

1. need the code word number of cluster to be K, use represent cluster centre, then data set is used represent, find two initial cluster centers with tentation data concentrates N number of dimension to be the vector of M, then all data sets compute vector with between Euclidean distance T _ij, formula (13) and (14) give computation process:

$T_{ij}=\left|{\stackrel{\→}{r}}_i-{\stackrel{\→}{r}}_j\right|(i\≠j)---\left(13\right)$

$\left|\stackrel{\→}{r_i}-\stackrel{\→}{r_j}\right|=\sqrt{{(\stackrel{\→}{r_{i1}}-\stackrel{\→}{r_{j1}})}^2+\mathrm{...}+{(\stackrel{\→}{r_{iM}}-\stackrel{\→}{r_{jM}})}^2}---\left(14\right)$

When having traveled through all data, find maximum T _ij, be expressed as T _u, now will ask for two data of maximal value as initial two class hearts.The remaining cluster centre of following calculating.

(2. suppose now to have determined k (2≤k≤K-1) individual cluster centre, kth+1 cluster centre is exactly ask in a remaining N-k data and Euclidean distance and those maximum data between the k determined (2≤k≤K-1) individual cluster centre.Computing formula is as follows:

$S_i=\underset{j=1}{\overset{k}{\Σ}}\left|\stackrel{\→}{r_i}-\stackrel{\→}{c_i}\right|---\left(15\right)$

$\left|\stackrel{\→}{r_i}-\stackrel{\→}{c_j}\right|=\sqrt{{(\stackrel{\→}{r_{i1}}-\stackrel{\→}{c_{j1}})}^2+\mathrm{...}+{(\stackrel{\→}{r_{iM}}-\stackrel{\→}{c_{jM}})}^2}---\left(16\right)$

3. S is calculated _imaximal value, if now data are then it can be used as the cluster centre that new.Repeat said process until the cluster centre number that obtains equal required by the number of codewords of getting.

In Misty Image quality assessment, the concrete building process of code book is as follows: after training image is all extracted feature by piecemeal, form a matrix Z=[X _l, X ₂..., X _p], wherein X ₁x _p, represent the feature that piecemeal extracts from p training image.With the K-means algorithm improved, cluster is carried out to Z.Generate K cluster centre, each cluster centre is exactly a code word, generates a code book containing K code word.Code book D _{[d × K]}=[D ₁, D ₂d _k] represent, the code word D in code book _{t (i=1,2..K)}it is the cluster centre formed by cluster.The image built for code book is all the image not having label, and namely these images do not have picture quality subjective scoring.

(3) local feature coding

Coded system can produce larger impact to the result of last algorithm, and the coded system of current main flow has four kinds: hard to encode, and soft encodes, SC coding and LLC coding.Wherein first two is the classical coded system comparatively early proposed, and latter two is the coded system of up-to-date proposition.And select classical soft coding, mainly because up-to-date coded system not necessarily has best effect, and soft coding similar to a kind of optimum code strategy of its proposition, after experimental section can provide the contrast choosing this coded system and other coded systems.The detailed process of coding is as follows:

The distance calculated between local feature and code word uses dot product.Use s _ijrepresent i-th local feature vectors x _iwith a jth code word D _jbetween similarity s (i, j)=x _id _j, such image local feature x _iformula (17) can be expressed as:

c _i＝[max(s _il，0)，…，max(s _ik，0)，(17)

max(-s _il，0)]，…，max(-s _ik，0)

Similarity in formula (17) has positive and negative two similar components, makes their resolution increase like this, improves eigenvector recognition.

(4) feature pooling

The part of coding have employed a matrix of coefficients C _{2K × N}=[c ₁, c ₂c _n], c _i=[c _{i, l}, c _{i, 2}c _{i, 2k}] ^t.Conveniently support vector returns, and needs the proper vector obtaining a regular length.The subjective perception impact of region the poorest in well-known image on image is maximum.In Images Classification problem, maximal value pooling has best classifying quality.

Maximal value pooling:

$\widehat{\mathrm{\β}}={\mathrm{\ψ}}_{\max }\left(C\right)---\left(18\right)$

Definition ψ in formula (18) _maxfor asking for the function of matrix of coefficients C maximal value in each row.

${\widehat{\mathrm{\β}}}_i=\max \{c_{1_i},c_{2_i}...c_{N_i}\}---\left(19\right)$

Will for regression training.

(5) SVR model training

Use instrument libsvm trains the proper vector of training image and the subjective scoring of training image.Give the building process of image quality evaluation model: first piecemeal is carried out to training image and extract feature (selecting composite character here), code book is obtained by clustering algorithm (application be improve K-means clustering algorithm), recycling code book is encoded to training image and is obtained the proper vector of training image, finally proper vector is put into regression model together with the mos value of training image to train, just build difficulty action accomplishment evaluation model.

(6) quality score prediction

After building training pattern, proper vector is extracted to test pattern, utilizes the quality of SVR model prediction Misty Image.When providing a width greasy weather test pattern, feature extraction being carried out to it, utilizing code book to carry out coding to it and obtaining a proper vector, vector being returned the quality score just obtaining test pattern by model.

Claims (1)

1. the Misty Image quality evaluating method based on code book, first the method extracts feature to training image piecemeal, improvement K-means algorithm is used to carry out cluster to the feature extracted, complete the structure of code book, the code book built is utilized to encode to training image blocks, pooling strategy is finally utilized the matrix of coefficients after coding to be extracted to the proper vector of training image, after extracting proper vector according to said method, itself and training image subjective scoring are put into SVR to train and obtain a regression model, thus obtain the quality score of test pattern.