CN111598826A - Image objective quality evaluation method and system based on joint multi-scale image characteristics - Google Patents

Abstract

The invention provides a method and a system for evaluating objective quality of a picture based on joint multi-scale picture characteristics, wherein the method comprises the following steps: picture processing: processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾From y₁ ⁽ⁿ⁾Extracting and obtaining edge structure characteristics; side salient feature extraction: group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾And extracting to obtain edge saliency characteristics. The invention has higher accuracy for evaluating the quality of the desktop picture, and the comprehensive performance is more superior to the prior art through the verification of the prior database, and the invention has the following advantages for the distortion type of the desktop picture: the Gaussian blur, motion blur and JPEG2000 compression distortion have prominenceThe performance is excellent.

Description

Image objective quality evaluation method and system based on joint multi-scale image characteristics

Technical Field

The invention relates to the field of image processing, in particular to a method and a system for evaluating objective quality of a picture based on joint multi-scale picture characteristics.

Background

With the wide application of intelligent terminals, such as smart phones, tablets and notebook computers, desktop content pictures have replaced natural pictures and become the most common pictures with the highest consumption in people's daily life. Desktop content pictures are computer-generated pictures formed by combining graphics, text and natural pictures, and are largely used in applications such as desktop games, desktop collaboration, remote education and the like. For these applications, picture quality is of particular importance. However, since the desktop content picture and the natural picture have different characteristics, the distortion condition of the desktop content picture cannot be reflected well by the traditional quality evaluation method designed for the natural picture: compared with the characteristics of rich colors and smooth edges of natural pictures, desktop content pictures are often single in color, sharp in edges and full of a large number of repeated graphs; and the distortion of natural pictures is generally caused by the limited capability of physical sensors, but the distortion of desktop content pictures is generally caused by the computer itself. Therefore, an accurate and efficient objective quality evaluation method for the inner content pictures is urgently needed.

Patent document CN108335289A (application number: 201810049789.8) discloses an objective quality evaluation method for a full-reference fused image, which includes: selecting a picture database as input of model training, grouping pictures according to distortion types, wherein pictures with different degrees of distortion under each distortion type respectively obtain a file name and a label of each group of pictures; extracting characteristics, namely selecting various full reference measurement algorithms, respectively scoring the pictures in each distortion type, obtaining a characteristic vector by each group of pictures through one full reference measurement algorithm operation, and forming a characteristic matrix by the obtained characteristic vectors; data preprocessing, namely respectively normalizing the feature vector scores corresponding to the distorted image label and the distortion type between (1,100) and (0,1), and performing transposition processing to meet the training requirement of the SVM; and (5) performing characteristic training to obtain a quality evaluation model.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide a method and a system for evaluating the objective quality of a picture based on joint multi-scale picture characteristics.

The invention provides a picture objective quality evaluation method based on joint multi-scale picture characteristics, which comprises the following steps:

picture processing: processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾From y₁ ⁽ⁿ⁾Extracting and obtaining edge structure characteristics;

side salient feature extraction: group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾Extracting edge saliency characteristics;

calculating the feature similarity: calculating to obtain edge structure similarity and edge significance similarity according to the obtained edge structure feature and edge significance feature;

a characteristic combination step: calculating to obtain a final local quality map according to the obtained edge structure similarity and the edge significance similarity;

a characteristic pooling step: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

Preferably, the picture processing step:

processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾。

Preferably, the edge salient feature extracting step:

group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾The edge saliency characteristic is obtained by extraction, and the calculation formula is as follows:

wherein,

C_LM ⁽ⁿ⁾representing a brightness mask calculation result, wherein the brightness mask calculation result is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y₁ ⁽ⁿ⁾representing the group of pictures after the Laplacian pyramid processing;

y₀ ⁽ⁿ⁾representing the picture group processed by the Gaussian pyramid;

n represents the number of layers;

layer y₁ ⁽¹⁾The displayed picture characteristics are edge structure characteristics;

denotes a number y₀ ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₁representing a luminance contrast threshold;

| | represents an absolute value operation;

a₁represents a constant that ensures the stability of the equation;

C_LCM ⁽ⁿ⁾representing the results of contrast mask calculations based on C_LM ⁽ⁿ⁾Image characteristics of the processed group of pictures;

n represents the number of layers;

C_LCM ⁽¹⁾the picture feature expressed when n is 1 is an edge saliency feature;

a₂represents a constant that ensures the stability of the equation;

represents that C is_LM ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₂represents a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

denotes convolution;

and ×. 2 represents upsampling.

Preferably, the feature similarity calculation step:

and calculating to obtain the edge structure similarity according to the obtained edge structure characteristic and the edge saliency characteristic, wherein the calculation formula is as follows:

wherein,

S₁(x, y) represents the similarity of the point (x, y) side structure;

subscripts r and d indicate that the feature is taken from a reference picture or a distorted picture, respectively;

y_1r ⁽¹⁾(x, y) represents an edge structure feature when n is 1 at the point (x, y) of the reference picture;

y_1d ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the distorted picture;

T₁the expression is a non-zero constant to ensure the stability of the equation;

and calculating to obtain the edge significance similarity according to the obtained edge structure characteristic and the edge significance characteristic, wherein the calculation formula is as follows:

S₂(x,y)＝MS₁(x,y)^α·MS₂(x,y)

wherein,

S₂(x, y) represents the point (x, y) side saliency similarity;

α denotes S₂M S in (x, y)₁(x, y) is weighted;

MS₁(x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS₂(x, y) represents the edge significant similarity calculated by the similarity calculation function;

w₁(x, y) represents a weighting factor;

G_LMr ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

G_LMd ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

T₂represents a non-zero constant for equation stability;

∑_(x,y)w₁(x, y) denotes w at all points on the picture₁(x, y) accumulation;

C_LCMr ⁽¹⁾(x, y) represents LCM mask characteristics when n is 1 at point (x, y) of the reference picture;

C_LCMd ⁽¹⁾(x, y) represents the LCM mask characteristics when the distortion picture is n-1 at point (x, y);

C_LCMd ⁽²⁾(x, y) represents LCM mask characteristics indicating when the distortion picture has n of 2 at point (x, y);

T₃a non-zero constant is shown to ensure equation stability.

Preferably, the feature combining step:

according to the obtained edge structure similarity and the edge significance similarity, calculating to obtain the local quality similarity, wherein the calculation formula is as follows:

S_QM(x,y)＝(S₁(x,y))^ξ·(S₂(x,y))^ψ

＝(S₁(x,y))^ξ·MS₁(x,y)^μ·MS₂(x,y)^ψ

μ＝ψ·α

wherein,

S_QM(x, y) represents local mass similarity at point (x, y);

ξ denotes S₁(x, y) at local mass S_QM(x, y) is weighted;

psi denotes M S₁(x, y) at local mass S_QM(x, y) is weighted;

μ denotes M S₂(x, y) at local mass S_QM(x, y) is weighted;

α denotes S₂M S in (x, y)₁(x, y) takes weight.

Preferably, the feature pooling step:

and calculating to obtain a final objective evaluation score according to the obtained local quality map similarity, wherein the calculation formula is as follows:

w₂(x,y)＝max(y_1r ⁽²⁾(x,y),y_1d ⁽²⁾(x,y))

wherein,

s represents the final objective evaluation score;

w₂(x, y) represents a weight parameter.

y_1r ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the reference picture.

The invention provides a desktop content picture objective quality evaluation system based on joint multi-scale picture characteristics, which comprises the following steps:

the picture processing module: processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾From y₁ ⁽ⁿ⁾Extracting and obtaining edge structure characteristics;

the edge salient feature extraction module: group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾Extracting edge saliency characteristics;

the feature similarity calculation module: calculating to obtain edge structure similarity and edge significance similarity according to the obtained edge structure feature and edge significance feature;

a characteristic combination module: calculating to obtain a final local quality map according to the obtained edge structure similarity and the edge significance similarity;

a characteristic pooling module: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

Preferably, the picture processing module:

processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾；

The edge salient feature extraction module:

group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾The edge saliency characteristic is obtained by extraction, and the calculation formula is as follows:

wherein,

C_LM ⁽ⁿ⁾representing a brightness mask calculation result, wherein the brightness mask calculation result is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

to representA y function;

y₁ ⁽ⁿ⁾representing the group of pictures after the Laplacian pyramid processing;

y₀ ⁽ⁿ⁾representing the picture group processed by the Gaussian pyramid;

n represents the number of layers;

layer y₁ ⁽¹⁾The displayed picture characteristics are edge structure characteristics;

denotes a number y₀ ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₁representing a luminance contrast threshold;

| | represents an absolute value operation;

a₁represents a constant that ensures the stability of the equation;

C_LCM ⁽ⁿ⁾representing the results of contrast mask calculations based on C_LM ⁽ⁿ⁾Image characteristics of the processed group of pictures;

n represents the number of layers;

C_LCM ⁽¹⁾the picture feature expressed when n is 1 is an edge saliency feature;

a₂represents a constant that ensures the stability of the equation;

represents that C is_LM ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₂represents a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

denotes convolution;

×. 2 represents upsampling;

the feature similarity calculation module:

and calculating to obtain the edge structure similarity according to the obtained edge structure characteristic and the edge saliency characteristic, wherein the calculation formula is as follows:

wherein,

S₁(x, y) represents the similarity of the point (x, y) side structure;

subscripts r and d indicate that the feature is taken from a reference picture or a distorted picture, respectively;

y_1r ⁽¹⁾(x, y) represents an edge structure feature when n is 1 at the point (x, y) of the reference picture;

y_1d ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the distorted picture;

T₁the expression is a non-zero constant to ensure the stability of the equation;

and calculating to obtain the edge significance similarity according to the obtained edge structure characteristic and the edge significance characteristic, wherein the calculation formula is as follows:

S₂(x,y)＝MS₁(x,y)^α·MS₂(x,y)

wherein,

S₂(x, y) represents the point (x, y) side saliency similarity;

α denotes S₂M S in (x, y)₁(x, y) is weighted;

MS₁(x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS₂(x, y) represents the edge significant similarity calculated by the similarity calculation function;

w₁(x, y) represents a weighting factor;

C_LMr ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

C_LMd ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

T₂represents a non-zero constant for equation stability;

∑_(x,y)w₁(x, y) denotes w at all points on the picture₁(x, y) accumulation;

C_LCMr ⁽¹⁾(x, y) represents LCM mask characteristics when n is 1 at point (x, y) of the reference picture;

C_LCMd ⁽¹⁾(x, y) represents the LCM mask characteristics when the distortion picture is n-1 at point (x, y);

C_LCMd ⁽²⁾(x, y) represents LCM mask characteristics indicating when the distortion picture has n of 2 at point (x, y);

T₃a non-zero constant is shown to ensure equation stability.

Preferably, the feature combination module:

according to the obtained edge structure similarity and the edge significance similarity, calculating to obtain the local quality similarity, wherein the calculation formula is as follows:

S_QM(x,y)＝(S₁(x,y))^ξ·(S₂(x,y))^ψ

＝(S₁(x,y))^ξ·MS₁(x,y)^μ·MS₂(x,y)^ψ

μ＝ψ·α

wherein,

S_QM(x, y) represents local mass similarity at point (x, y);

ξ denotes S₁(x, y) at local mass S_QM(x, y) is weighted;

psi denotes M S₁(x, y) at local mass S_QM(x, y) is weighted;

μ denotes M S₂(x, y) at local mass S_QM(x, y) is weighted;

α denotes S₂M S in (x, y)₁(x, y) takes weight.

The feature pooling module:

and calculating to obtain a final objective evaluation score according to the obtained local quality map similarity, wherein the calculation formula is as follows:

w₂(x,y)＝max(y_1r ⁽²⁾(x,y),y_1d ⁽²⁾(x,y))

wherein,

s represents the final objective evaluation score;

w₂(x, y) represents a weight parameter.

y_1r ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the reference picture.

According to the present invention, there is provided a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps of any of the above-mentioned methods for objective quality assessment of pictures based on joint multi-scale picture features.

Compared with the prior art, the invention has the following beneficial effects:

the invention has higher accuracy for evaluating the quality of the desktop picture, and the comprehensive performance is more superior to the prior art through the verification of the prior database, and the invention has the following advantages for the distortion type of the desktop picture: the Gaussian blur, the motion blur and the JPEG2000 compression distortion have outstanding excellent performance.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a schematic view of a process flow provided by the present invention.

Detailed Description

The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.

The invention provides a picture objective quality evaluation method based on joint multi-scale picture characteristics, which comprises the following steps:

picture processing: processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾From y₁ ⁽ⁿ⁾Extracting and obtaining edge structure characteristics;

side salient feature extraction: group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾Extracting edge saliency characteristics;

calculating the feature similarity: calculating to obtain edge structure similarity and edge significance similarity according to the obtained edge structure feature and edge significance feature;

a characteristic combination step: calculating to obtain a final local quality map according to the obtained edge structure similarity and the edge significance similarity;

a characteristic pooling step: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

Specifically, the picture processing step:

processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾。

Specifically, the edge salient feature extraction step:

group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾The edge saliency characteristic is obtained by extraction, and the calculation formula is as follows:

wherein,

C_LM ⁽ⁿ⁾representing a brightness mask calculation result, wherein the brightness mask calculation result is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y₁ ⁽ⁿ⁾representing the group of pictures after the Laplacian pyramid processing;

y₀ ⁽ⁿ⁾representing the picture group processed by the Gaussian pyramid;

n represents the number of layers;

layer y₁ ⁽¹⁾The displayed picture characteristics are edge structure characteristics;

denotes a number y₀ ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₁representing a luminance contrast threshold;

| | represents an absolute value operation;

a₁represents a constant that ensures the stability of the equation;

C_LCM ⁽ⁿ⁾representing the results of contrast mask calculations based on C_LM ⁽ⁿ⁾Image characteristics of the processed group of pictures;

n represents the number of layers;

C_LCM ⁽¹⁾the picture feature expressed when n is 1 is an edge saliency feature;

a₂represents a constant that ensures the stability of the equation;

represents that C is_LM ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₂represents a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

denotes convolution;

and ×. 2 represents upsampling.

Specifically, the feature similarity calculation step:

and calculating to obtain the edge structure similarity according to the obtained edge structure characteristic and the edge saliency characteristic, wherein the calculation formula is as follows:

wherein,

S₁(x, y) represents the similarity of the point (x, y) side structure;

subscripts r and d indicate that the feature is taken from a reference picture or a distorted picture, respectively;

y_1r ⁽¹⁾(x, y) represents an edge structure feature when n is 1 at the point (x, y) of the reference picture;

y_1d ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the distorted picture;

T₁the expression is a non-zero constant to ensure the stability of the equation;

and calculating to obtain the edge significance similarity according to the obtained edge structure characteristic and the edge significance characteristic, wherein the calculation formula is as follows:

S₂(x,y)＝MS₁(x,y)^α·MS₂(x,y)

wherein,

S₂(x, y) represents the point (x, y) side saliency similarity;

α denotes S₂M S in (x, y)₁(x, y) is weighted;

MS₁(x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS₂(x, y) represents the edge significant similarity calculated by the similarity calculation function;

w₁(x, y) represents a weighting factor;

C_LMr ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

C_LMd ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

T₂represents a non-zero constant for equation stability;

∑_(x,y)w₁(x, y) denotes w at all points on the picture₁(x, y) accumulation;

C_LCMr ⁽¹⁾(x, y) represents LCM mask characteristics when n is 1 at point (x, y) of the reference picture;

C_LCMd ⁽¹⁾(x, y) represents the LCM mask characteristics when the distortion picture is n-1 at point (x, y);

C_LCMd ⁽²⁾(x, y) represents LCM mask characteristics indicating when the distortion picture has n of 2 at point (x, y);

T₃a non-zero constant is shown to ensure equation stability.

Specifically, the feature combining step:

according to the obtained edge structure similarity and the edge significance similarity, calculating to obtain the local quality similarity, wherein the calculation formula is as follows:

S_QM(x,y)＝(S₁(x,y))^ξ·(S₂(x,y))^ψ

＝(S₁(x,y))^ξ·MS₁(x,y)^μ·MS₂(x,y)^ψ

μ＝ψ·α

wherein,

S_QM(x, y) represents local mass similarity at point (x, y);

ξ denotes S₁(x, y) at local mass S_QM(x, y) is weighted;

psi denotes M S₁(x, y) at local mass S_QM(x, y) is weighted;

μ denotes M S₂(x, y) at local mass S_QM(x, y) is weighted;

α denotes S₂M S in (x, y)₁(x, y) takes weight.

Specifically, the feature pooling step:

and calculating to obtain a final objective evaluation score according to the obtained local quality map similarity, wherein the calculation formula is as follows:

w₂(x,y)＝max(y_1r ⁽²⁾(x,y),y_1d ⁽²⁾(x,y))

wherein,

s represents the final objective evaluation score;

w₂(x, y) represents a weight parameter.

y_1r ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the reference picture.

The desktop content picture objective quality evaluation system based on the joint multi-scale picture characteristics can be realized through the step flow of the picture objective quality evaluation method based on the joint multi-scale picture characteristics. The image objective quality evaluation method based on the joint multi-scale image feature can be understood as a preferred example of the desktop content image objective quality evaluation system based on the joint multi-scale image feature by those skilled in the art.

The invention provides a desktop content picture objective quality evaluation system based on joint multi-scale picture characteristics, which comprises the following steps:

the picture processing module: processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾From y₁ ⁽ⁿ⁾Extracting and obtaining edge structure characteristics;

the edge salient feature extraction module: group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾Extracting edge saliency characteristics;

the feature similarity calculation module: calculating to obtain edge structure similarity and edge significance similarity according to the obtained edge structure feature and edge significance feature;

a characteristic combination module: calculating to obtain a final local quality map according to the obtained edge structure similarity and the edge significance similarity;

a characteristic pooling module: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

Specifically, the image processing module:

processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾；

The edge salient feature extraction module:

group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾The edge saliency characteristic is obtained by extraction, and the calculation formula is as follows:

wherein,

C_LM ⁽ⁿ⁾representing a brightness mask calculation result, wherein the brightness mask calculation result is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y₁ ⁽ⁿ⁾representing the group of pictures after the Laplacian pyramid processing;

y₀ ⁽ⁿ⁾representing the picture group processed by the Gaussian pyramid;

n represents the number of layers;

layer y₁ ⁽¹⁾The displayed picture characteristics are edge structure characteristics;

denotes a number y₀ ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₁representing a luminance contrast threshold;

| | represents an absolute value operation;

a₁represents a constant that ensures the stability of the equation;

C_LCM ⁽ⁿ⁾representing the results of contrast mask calculations based on C_LM ⁽ⁿ⁾Image characteristics of the processed group of pictures;

n represents the number of layers;

C_LCM ⁽¹⁾the picture feature expressed when n is 1 is an edge saliency feature;

a₂represents a constant that ensures the stability of the equation;

represents that C is_LM ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₂represents a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

denotes convolution;

×. 2 represents upsampling;

the feature similarity calculation module:

and calculating to obtain the edge structure similarity according to the obtained edge structure characteristic and the edge saliency characteristic, wherein the calculation formula is as follows:

wherein,

S₁(x, y) represents the similarity of the point (x, y) side structure;

subscripts r and d indicate that the feature is taken from a reference picture or a distorted picture, respectively;

y_1r ⁽¹⁾(x, y) represents an edge structure feature when n is 1 at the point (x, y) of the reference picture;

y_1d ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the distorted picture;

T₁the expression is a non-zero constant to ensure the stability of the equation;

and calculating to obtain the edge significance similarity according to the obtained edge structure characteristic and the edge significance characteristic, wherein the calculation formula is as follows:

S₂(x,y)＝MS₁(x,y)^α·MS₂(x,y)

wherein,

S₂(x, y) represents the point (x, y) side saliency similarity;

α denotes S₂M S in (x, y)₁(x, y) is weighted;

MS₁(x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS₂(x, y) represents the edge significant similarity calculated by the similarity calculation function;

w₁(x, y) represents a weighting factor;

C_LMr ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

C_LMd ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

T₂represents a non-zero constant for equation stability;

∑_(x,y)w₁(x, y) denotes w at all points on the picture₁(x, y) accumulation;

C_LCMr ⁽¹⁾(x, y) represents LCM mask characteristics when n is 1 at point (x, y) of the reference picture;

C_LCMd ⁽¹⁾(x, y) represents the LCM mask characteristics when the distortion picture is n-1 at point (x, y);

C_LCMd ⁽²⁾(x, y) represents LCM mask characteristics indicating when the distortion picture has n of 2 at point (x, y);

T₃a non-zero constant is shown to ensure equation stability.

Specifically, the feature combination module:

according to the obtained edge structure similarity and the edge significance similarity, calculating to obtain the local quality similarity, wherein the calculation formula is as follows:

S_QM(x,y)＝(S₁(x,y))^ξ·(S₂(x,y))^ψ

＝(S₁(x,y))^ξ·MS₁(x,y)^μ·MS₂(x,y)^ψ

μ＝ψ·α

wherein,

S_QM(x, y) represents local mass similarity at point (x, y);

ξ denotes S₁(x, y) at local mass S_QM(x, y) is weighted;

psi denotes M S₁(x, y) at local mass S_QM(x, y) is weighted;

μ denotes M S₂(x, y) at local mass S_QM(x, y) is weighted;

α denotes S₂M S in (x, y)₁(x, y) takes weight.

The feature pooling module:

and calculating to obtain a final objective evaluation score according to the obtained local quality map similarity, wherein the calculation formula is as follows:

w₂(x,y)＝max(y_1r ⁽²⁾(x,y),y_1d ⁽²⁾(x,y))

wherein,

s represents the final objective evaluation score;

w₂(x, y) represents a weight parameter.

y_1r ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the reference picture.

According to the present invention, there is provided a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps of any of the above-mentioned methods for objective quality assessment of pictures based on joint multi-scale picture features.

The present invention will be described more specifically below with reference to preferred examples.

Preferred example 1:

the invention provides an objective quality evaluation method for a desktop content picture based on joint multi-scale, which is used for extracting edge structure characteristics and edge significance characteristics of the picture to evaluate the distortion degree. Specifically, a picture feature extraction mode is designed by combining the characteristics of a human visual system, and the extracted picture features comprise two features:

1. the characteristics of the side structure are as follows,

2. edge saliency characteristics.

In order to achieve the purpose, the invention adopts the following technical scheme:

1. processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾(ii) a Let n equal 1 from y₁ ⁽ⁿ⁾Extracting edge structure characteristics;

2. edge saliency features are extracted from the two pyramids by using a Luminance Mask (LM) and a contrast mask (LCM), and the specific calculation method is as follows:

wherein

G (x, y; σ) represents a Gaussian kernel, x represents convolution, and ≈ 2 represents upsampling.

Luminance mask calculation result C_LM ⁽ⁿ⁾Can detect the brightness change recognizable by human eyes, and converts gamma according to a Buchsbaum curve₁Is set to 1; contrast mask calculation result C_LCM ⁽ⁿ⁾Can detect the change of the contrast recognizable by human eyes, and can detect the gamma according to the contrast detectable threshold value₂Set to 0.62.

Let n be 1 from C_LCM ⁽ⁿ⁾And extracting edge saliency features.

3. Feature similarity calculation

Side structure similarity:

S₂(x,y)＝MS₁(x,y)^α·MS₂(x,y),

edge saliency similarity:

where the subscripts r and d denote that the feature is taken from a reference picture or a distorted picture, w₁(x,y)＝y_1r ⁽²⁾，T₁,T₂,T₃Respectively taking 0.07,1 × 10-50 and 0.01.

4. Feature combination

The final local mass map is:

S_QM(x,y)＝(S₁(x,y))^ξ·(S₂(x,y))^ψ

＝(S₁(x,y))^ξ·MS₁(x,y)^μ·MS₂(x,y)^ψ,

wherein, mu is psi- α,

set to 1.8,0.02,0.9, respectively.

5. Feature pooling

Final objective evaluation score:

wherein, w₂＝max(y_1r ⁽²⁾(x,y),y_1d ⁽²⁾(x,y))。

To illustrate the effectiveness of the above model, a test was performed on the desktop content picture authority database SIQAD. The SIQAD database includes 20 reference pictures, each corresponding to 7 orders of magnitude of 7 distortion pictures of 980 distortion. The 7 kinds of distortion include Gaussian Noise (GN), Gaussian Blur (GB), Motion Blur (MB), contrast variation (CC), JPEG compression (JPEG), JPEG2000 compression (J2K), layer efficient coding (LSC).

Three indicators proposed by VQEG experts group and specifically used to measure the consistency between subjective score and objective evaluation score are used to determine the superiority of the model, and these three indicators are Pearson Linear Correlation Coefficient (PLCC), Root Mean Square Error (RMSE) and Spearman rank-order correlation coefficient (SROCC), which are calculated as follows:

wherein m and Q represent subjective scores and objective scores respectively,

mean values representing the subjective score and the objective score, respectively, d_iRepresenting the difference between the subjective score sorting sequence and the objective score sorting sequence of the ith picture. The values of PLCC and SROCC are between 0 and 1, and the closer to 1, the better the consistency between the subjective score and the objective score is; the smaller the RMSE value, the smaller the difference between the subjective score and the objective score, and the better the model performance.

Table 1 shows the test results on the database SIQAD, where PSNR, SSIM, MSSIM, IWSSIM, VIF, IFC, FSIM, and SCQ are quality evaluation methods designed for natural pictures, SIQM, SQI, ESIM, MDOGS, and GFM are objective quality evaluation methods designed for desktop content pictures in recent years, and it can be seen by comparing the data of each method that:

aiming at the overall performance, the bit array first name and the SROCC bit array second name are evaluated in PLCC and RMSE indexes;

for the single distortion type, the method obtains 9 first names and 1 third name, is obviously superior to other methods, and has remarkable superiority in evaluating the distortion types GB, MB and J2K.

Table one SIQAD database test results:

those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.

The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims (10)

1. A picture objective quality evaluation method based on joint multi-scale picture features is characterized by comprising the following steps:

picture processing: processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾From y₁ ⁽ⁿ⁾Extracting and obtaining edge structure characteristics;

side salient feature extraction: group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾Extracting edge saliency characteristics;

calculating the feature similarity: calculating to obtain edge structure similarity and edge significance similarity according to the obtained edge structure feature and edge significance feature;

a characteristic combination step: calculating to obtain a final local quality map according to the obtained edge structure similarity and the edge significance similarity;

a characteristic pooling step: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

2. The picture objective quality evaluation method based on the joint multi-scale picture features according to claim 1, wherein the picture processing step comprises:

processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾。

3. The image objective quality evaluation method based on the joint multi-scale image features according to claim 2, wherein the edge salient feature extraction step comprises:

group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾The edge saliency characteristic is obtained by extraction, and the calculation formula is as follows:

wherein,

C_LM ⁽ⁿ⁾representing a brightness mask calculation result, wherein the brightness mask calculation result is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y₁ ⁽ⁿ⁾representing the group of pictures after the Laplacian pyramid processing;

y₀ ⁽ⁿ⁾representing the picture group processed by the Gaussian pyramid;

n represents the number of layers;

layer y₁ ⁽¹⁾The displayed picture characteristics are edge structure characteristics;

denotes a number y₀ ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₁representing a luminance contrast threshold;

| | represents an absolute value operation;

a₁represents a constant that ensures the stability of the equation;

C_LCM ⁽ⁿ⁾representing the results of contrast mask calculations based on C_LM ⁽ⁿ⁾Image characteristics of the processed group of pictures;

n represents the number of layers;

C_LCM ⁽¹⁾the picture feature expressed when n is 1 is an edge saliency feature;

a₂represents a constant that ensures the stability of the equation;

represents that C is_LM ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₂represents a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

denotes convolution;

and ×. 2 represents upsampling.

4. The picture objective quality evaluation method based on the joint multi-scale picture features according to claim 3, wherein the feature similarity calculation step comprises:

and calculating to obtain the edge structure similarity according to the obtained edge structure characteristic and the edge saliency characteristic, wherein the calculation formula is as follows:

wherein,

S₁(x, y) represents the similarity of the point (x, y) side structure;

subscripts r and d indicate that the feature is taken from a reference picture or a distorted picture, respectively;

y_1r ⁽¹⁾(x, y) represents an edge structure feature when n is 1 at the point (x, y) of the reference picture;

y_1d ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the distorted picture;

T₁the expression is a non-zero constant to ensure the stability of the equation;

and calculating to obtain the edge significance similarity according to the obtained edge structure characteristic and the edge significance characteristic, wherein the calculation formula is as follows:

S₂(x,y)＝MS₁(x,y)^α·MS₂(x,y)

wherein,

S₂(x, y) represents the point (x, y) side saliency similarity;

α denotes S₂(x, y) in MS₁(x, y) is weighted;

MS₁(x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS₂(x, y) represents the edge significant similarity calculated by the similarity calculation function;

w₁(x, y) represents a weighting factor;

C_LMr ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

C_LMd ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

T₂represents a non-zero constant for equation stability;

∑_(x,y)w₁(x, y) denotes w at all points on the picture₁(x, y) accumulation;

C_LCMr ⁽¹⁾(x, y) represents LCM mask characteristics when n is 1 at point (x, y) of the reference picture;

C_LCMd ⁽¹⁾(x, y) represents the LCM mask characteristics when the distortion picture is n-1 at point (x, y);

C_LCMd ⁽²⁾(x, y) represents LCM mask characteristics indicating when the distortion picture has n of 2 at point (x, y);

T₃a non-zero constant is shown to ensure equation stability.

5. The picture objective quality evaluation method based on the joint multi-scale picture features according to claim 4, wherein the feature combination step comprises:

according to the obtained edge structure similarity and the edge significance similarity, calculating to obtain the local quality similarity, wherein the calculation formula is as follows:

S_QM(x,y)＝(S₁(x,y))^ξ·(S₂(x,y))^ψ

＝(S₁(x,y))^ξ·MS₁(x,y)^μ·MS₂(x,y)^ψ

μ＝ψ·α

wherein,

S_QM(x, y) represents local mass similarity at point (x, y);

ξ denotes S₁(x, y) at local mass S_QM(x, y) is weighted;

psi denotes MS₁(x, y) at local mass S_QM(x, y) is weighted;

mu denotes MS₂(x, y) at local mass S_QM(x, y) is weighted;

α denotes S₂(x, y) in MS₁(x, y) takes weight.

6. The picture objective quality evaluation method based on the joint multi-scale picture features according to claim 5, wherein the feature pooling step comprises:

and calculating to obtain a final objective evaluation score according to the obtained local quality map similarity, wherein the calculation formula is as follows:

w₂(x,y)＝max(y_1r ⁽²⁾(x,y),y_1d ⁽²⁾(x,y))

wherein,

s represents the final objective evaluation score;

w₂(x, y) represents a weight parameter.

y_1r ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the reference picture.

7. A desktop content picture objective quality evaluation system based on joint multi-scale picture features is characterized by comprising the following steps:

the picture processing module: processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾From y₁ ⁽ⁿ⁾Extracting and obtaining edge structure characteristics;

the edge salient feature extraction module: group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾Extracting edge saliency characteristics;

the feature similarity calculation module: calculating to obtain edge structure similarity and edge significance similarity according to the obtained edge structure feature and edge significance feature;

a characteristic combination module: calculating to obtain a final local quality map according to the obtained edge structure similarity and the edge significance similarity;

a characteristic pooling module: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

8. The system of claim 7, wherein the image processing module is configured to:

processing the original picture into picture groups with different scales by using the Gaussian pyramid and the Laplace pyramid, and respectively recording the picture groups as y₀ ⁽ⁿ⁾And y₁ ⁽ⁿ⁾；

The edge salient feature extraction module:

group y of pictures processed from Gaussian pyramid using luminance mask and contrast mask₀ ⁽ⁿ⁾Group y of pictures processed with laplacian pyramid₁ ⁽ⁿ⁾The edge saliency characteristic is obtained by extraction, and the calculation formula is as follows:

wherein,

C_LM ⁽ⁿ⁾representing a brightness mask calculation result, wherein the brightness mask calculation result is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y₁ ⁽ⁿ⁾representing the group of pictures after the Laplacian pyramid processing;

y₀ ⁽ⁿ⁾representing the picture group processed by the Gaussian pyramid;

n represents the number of layers;

layer y₁ ⁽¹⁾The displayed picture characteristics are edge structure characteristics;

denotes a number y₀ ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₁representing a luminance contrast threshold;

| | represents an absolute value operation;

a₁represents a constant that ensures the stability of the equation;

C_LCM ⁽ⁿ⁾representing the results of contrast mask calculations based on C_LM ⁽ⁿ⁾Image characteristics of the processed group of pictures;

n represents the number of layers;

C_LCM ⁽¹⁾the picture feature expressed when n is 1 is an edge saliency feature;

a₂represents a constant that ensures the stability of the equation;

represents that C is_LM ⁽ⁿ⁺¹⁾Substituting the calculated result into the y function;

γ₂represents a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

denotes convolution;

×. 2 represents upsampling;

the feature similarity calculation module:

and calculating to obtain the edge structure similarity according to the obtained edge structure characteristic and the edge saliency characteristic, wherein the calculation formula is as follows:

wherein,

S₁(x, y) represents the similarity of the point (x, y) side structure;

subscripts r and d indicate that the feature is taken from a reference picture or a distorted picture, respectively;

y_1r ⁽¹⁾(x, y) represents an edge structure feature when n is 1 at the point (x, y) of the reference picture;

y_1d ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the distorted picture;

T₁the expression is a non-zero constant to ensure the stability of the equation;

and calculating to obtain the edge significance similarity according to the obtained edge structure characteristic and the edge significance characteristic, wherein the calculation formula is as follows:

S₂(x,y)＝MS₁(x,y)^α·MS₂(x,y)

wherein,

S₂(x, y) represents the point (x, y) side saliency similarity;

α denotes S₂(x, y) in MS₁(x, y) is weighted;

MS₁(x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS₂(x, y) represents the edge significant similarity calculated by the similarity calculation function;

w₁(x, y) represents a weighting factor;

C_LMr ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

C_LMd ⁽¹⁾(x, y) denotes an LM mask feature when n is 1 at point (x, y) of the reference picture;

T₂represents a non-zero constant for equation stability;

∑_(x,y)w₁(x, y) denotes w at all points on the picture₁(x, y) accumulation;

C_LCMr ⁽¹⁾(x, y) represents LCM mask characteristics when n is 1 at point (x, y) of the reference picture;

C_LCMd ⁽¹⁾(x, y) represents the LCM mask characteristics when the distortion picture is n-1 at point (x, y);

C_LCMd ⁽²⁾(x, y) represents LCM mask characteristics indicating when the distortion picture has n of 2 at point (x, y);

T₃a non-zero constant is shown to ensure equation stability.

9. The system according to claim 8, wherein the feature combination module:

according to the obtained edge structure similarity and the edge significance similarity, calculating to obtain the local quality similarity, wherein the calculation formula is as follows:

S_QM(x,y)＝(S₁(x,y))^ξ·(S₂(x,y))^ψ

＝(S₁(x,y))^ξ·MS₁(x,y)^μ·MS₂(x,y)^ψ

μ＝ψ·α

wherein,

S_QM(x, y) represents local mass similarity at point (x, y);

ξ denotes S₁(x, y) at local mass S_QM(x, y) is weighted;

psi denotes MS₁(x, y) at local mass S_QM(x, y) is weighted;

mu denotes MS₂(x, y) at local mass S_QM(x, y) is weighted;

α denotes S₂(x, y) in MS₁(x, y) takes weight.

The feature pooling module:

and calculating to obtain a final objective evaluation score according to the obtained local quality map similarity, wherein the calculation formula is as follows:

w₂(x,y)＝max(y_1r ⁽²⁾(x,y),y_1d ⁽²⁾(x,y))

wherein,

s represents the final objective evaluation score;

w₂(x, y) represents a weight parameter.

y_1r ⁽²⁾(x, y) represents an edge structure feature when n is 2 at the point (x, y) of the reference picture.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for objective quality assessment of pictures based on joint multi-scale picture features according to any one of claims 1 to 6.

Images