CN111598826A - Image objective quality evaluation method and system based on joint multi-scale image characteristics - Google Patents

Abstract

The present invention provides a method and system for evaluating the objective quality of pictures based on joint multi-scale picture features. y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ , extract edge structure features from y ₁ ⁽ⁿ⁾ ; edge saliency feature extraction step: use brightness mask and contrast mask, from the image group y ₀ ^{( n)} The edge saliency feature is obtained by extracting the image group y ₁ ⁽ⁿ⁾ after processing with the Laplacian pyramid. The present invention has higher accuracy in evaluating the quality of desktop pictures. It is found through the verification of the existing database that the comprehensive performance is more superior than the prior art, and the distortion types of desktop pictures: Gaussian blur, motion blur and JPEG2000 compression distortion have outstanding performance. .

Description

Method and system for objective image quality evaluation based on joint multi-scale image features

technical field

The present invention relates to the field of image processing, in particular, to a method and system for evaluating objective quality of pictures based on joint multi-scale picture features.

Background technique

With the wide application of smart terminals, such as smart phones, tablets and laptops, desktop content pictures have replaced natural pictures and become the most common and consumed pictures in people's daily life. Desktop content pictures are computer-generated images composed of graphics, text, and natural pictures. They are widely used in desktop games, desktop collaboration, and distance education. For these applications, image quality is particularly important. However, due to the different characteristics of desktop content pictures and natural pictures, the traditional quality evaluation methods designed for natural pictures cannot well reflect the distortion of desktop content pictures: compared with the characteristics of rich colors and smooth edges of natural pictures, Desktop content images tend to have relatively single colors, sharp edges, and a lot of repetitive graphics; and the distortion of natural images is generally caused by the limited capabilities of physical sensors, but the distortion of desktop content images is generally caused by the computer itself. Therefore, there is an urgent need for an accurate and efficient objective quality evaluation method for inner content pictures.

Patent document CN108335289A (application number: 201810049789.8) discloses a full-reference fusion image objective quality evaluation method, including: selecting a picture database as an input for model training, grouping pictures according to distortion types, and each distortion type has different degrees of distortion For each group of pictures, the file name and label of each group of pictures are obtained respectively; for feature extraction, a variety of full-reference measurement algorithms are selected to score the pictures in each distortion type. eigenvectors, and the eigenvectors will be obtained to form a feature matrix; data preprocessing, normalize the eigenvector scores corresponding to the distorted image labels and distortion types to be between (1,100) and (0,1) respectively, and transpose to satisfy the SVM Training needs; feature training to obtain a quality evaluation model.

SUMMARY OF THE INVENTION

In view of the defects in the prior art, the purpose of the present invention is to provide a method and system for evaluating the objective quality of pictures based on joint multi-scale picture features.

According to a method for evaluating objective quality of pictures based on joint multi-scale picture features, the method includes:

Image processing steps: use Gaussian pyramid and Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively, and extract edge structure features from y ₁ ⁽ⁿ⁾ ;

Edge saliency feature extraction step: Using the brightness mask and the contrast mask, extract the edge saliency from the image group y ₀ ⁽ⁿ⁾ processed by the Gaussian pyramid and the image group y ₁ ⁽ n) processed by the Laplacian pyramid. feature;

Feature similarity calculation step: According to the obtained edge structure feature and edge saliency feature, calculate and obtain the edge structure similarity and edge saliency similarity;

Feature combination step: Calculate and obtain the final local quality map according to the obtained edge structure similarity and edge saliency similarity;

Feature pooling step: Calculate the final objective evaluation score according to the obtained final local quality map.

Preferably, the picture processing steps:

Use the Gaussian pyramid and the Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively.

Preferably, the edge salient feature extraction step:

Using the luminance mask and the contrast mask, the edge saliency features are extracted from the image group y ₀ ⁽ⁿ⁾ processed by the Gaussian pyramid and the image group y ₁ ⁽ n) processed by the Laplacian pyramid, and the calculation formula is as follows:

in,

C _LM ⁽ⁿ⁾ represents the calculation result of the luminance mask, which is the image feature of the image group processed based on the Gaussian pyramid and the Laplacian pyramid;

表示y函数；

represents the y function;

y ₁ ⁽ⁿ⁾ represents the image group after Laplacian pyramid processing;

y ₀ ⁽ⁿ⁾ represents the image group after Gaussian pyramid processing;

n represents the number of layers;

The image features represented by layer y ₁ ⁽¹⁾ are edge structure features;

表示将y₀ ⁽ⁿ⁺¹⁾带入y函数的计算结果；

Indicates the calculation result of bringing y ₀ ⁽ⁿ⁺¹⁾ into the y function;

γ ₁ represents the brightness contrast threshold;

|| means taking the absolute value operation;

a ₁ represents a constant that guarantees the stability of the equation;

C _LCM ⁽ⁿ⁾ represents the calculation result of the contrast mask, which is based on the image features of the image group processed by C _LM ⁽ⁿ⁾ ;

n represents the number of layers;

C _LCM ⁽¹⁾ indicates that the image features displayed when n=1 are edge saliency features;

a ₂ represents a constant that guarantees the stability of the equation;

表示将C_LM ⁽ⁿ⁺¹⁾带入y函数的计算结果；

Represents the calculation result of bringing C _LM ⁽ⁿ⁺¹⁾ into the y function;

γ ₂ represents the contrast detectable threshold;

G(x, y; σ) represents the Gaussian kernel function;

* means convolution;

↑2 means upsampling.

Preferably, the feature similarity calculation step:

According to the obtained edge structure features and edge saliency features, the edge structure similarity is calculated and obtained, and the calculation formula is as follows:

in,

S ₁ (x, y) represents the structural similarity of the point (x, y) edge;

The subscripts r and d respectively indicate that the feature is taken from the reference picture or the distorted picture;

y _1r ⁽¹⁾ (x, y) represents the edge structure feature of the reference picture when n=1 at point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ represents a non-zero constant to ensure the stability of the equation;

According to the obtained edge structure features and edge saliency features, the similarity of the obtained edge saliency is calculated, and the calculation formula is as follows:

S ₂ (x,y)=MS ₁ (x,y) ^α ·MS ₂ (x,y)

in,

S ₂ (x, y) represents the point (x, y) edge saliency similarity;

α represents the weight of MS ₁ (x, y) in S ₂ (x, y);

MS ₁ (x,y) represents the edge structure similarity calculated by the similarity calculation function;

MS ₂ (x,y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents the weight factor;

G _LMr ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y);

G _LMd ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y);

T ₂ represents a non-zero constant to ensure the stability of the equation;

∑ _(x,y) w ₁ (x,y) represents the accumulation of w ₁ (x,y) at all points on the picture;

C _LCMr ⁽¹⁾ (x, y) represents the LCM mask feature of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents the LCM mask feature of the distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents the LCM mask feature of the distorted picture when n=2 at point (x, y);

T ₃ represents a non-zero constant to ensure the stability of the equation.

Preferably, the feature combining step:

According to the obtained edge structure similarity and edge saliency similarity, the local quality similarity is calculated, and the calculation formula is as follows:

S _QM (x,y)=(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

=(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ

μ=ψ·α

in,

S _QM (x, y) represents the local quality similarity at point (x, y);

ξ represents the weight of S ₁ (x, y) in the local mass S _QM (x, y);

ψ represents the weight of MS ₁ (x, y) in the local mass S _QM (x, y);

μ represents the weight of MS ₂ (x,y) in the local mass S _QM (x,y);

α represents the weight occupied by MS ₁ (x, y) in S ₂ (x, y).

Preferably, the feature pooling step:

According to the obtained local quality map similarity, the final objective evaluation score is calculated and obtained, and the calculation formula is as follows:

w ₂ (x,y)=max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y))

in,

S represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter.

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at point (x, y).

An objective quality evaluation system for desktop content pictures based on joint multi-scale picture features provided according to the present invention includes:

Image processing module: use Gaussian pyramid and Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively, and extract edge structure features from y ₁ ⁽ⁿ⁾ ;

Edge saliency feature extraction module: Using brightness mask and contrast mask, extract edge saliency from Gaussian pyramid processed image group y ₀ ⁽ⁿ⁾ and Laplacian pyramid processed image group y ₁ ⁽ⁿ⁾ feature;

Feature similarity calculation module: According to the obtained edge structure features and edge saliency features, calculate the edge structure similarity and edge saliency similarity;

Feature combination module: Calculate and obtain the final local quality map according to the obtained edge structure similarity and edge saliency similarity;

Feature pooling module: Calculate the final objective evaluation score according to the obtained final local quality map.

Preferably, the picture processing module:

Use the Gaussian pyramid and the Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively;

The edge salient feature extraction module:

Using the luminance mask and the contrast mask, the edge saliency features are extracted from the image group y ₀ ⁽ⁿ⁾ processed by the Gaussian pyramid and the image group y ₁ ⁽ n) processed by the Laplacian pyramid, and the calculation formula is as follows:

in,

C _LM ⁽ⁿ⁾ represents the calculation result of the luminance mask, which is the image feature of the image group processed based on the Gaussian pyramid and the Laplacian pyramid;

表示y函数；

represents the y function;

y ₁ ⁽ⁿ⁾ represents the image group after Laplacian pyramid processing;

y ₀ ⁽ⁿ⁾ represents the image group after Gaussian pyramid processing;

n represents the number of layers;

The image features represented by layer y ₁ ⁽¹⁾ are edge structure features;

表示将y₀ ⁽ⁿ⁺¹⁾带入y函数的计算结果；

Indicates the calculation result of bringing y ₀ ⁽ⁿ⁺¹⁾ into the y function;

γ ₁ represents the brightness contrast threshold;

|| means taking the absolute value operation;

a ₁ represents a constant that guarantees the stability of the equation;

C _LCM ⁽ⁿ⁾ represents the calculation result of the contrast mask, which is based on the image features of the image group processed by C _LM ⁽ⁿ⁾ ;

n represents the number of layers;

C _LCM ⁽¹⁾ indicates that the image features displayed when n=1 are edge saliency features;

a ₂ represents a constant that guarantees the stability of the equation;

表示将C_LM ⁽ⁿ⁺¹⁾带入y函数的计算结果；

Represents the calculation result of bringing C _LM ⁽ⁿ⁺¹⁾ into the y function;

γ ₂ represents the contrast detectable threshold;

G(x, y; σ) represents the Gaussian kernel function;

* means convolution;

↑2 means upsampling;

The feature similarity calculation module:

According to the obtained edge structure features and edge saliency features, the edge structure similarity is calculated and obtained, and the calculation formula is as follows:

in,

S ₁ (x, y) represents the structural similarity of the point (x, y) edge;

The subscripts r and d respectively indicate that the feature is taken from the reference picture or the distorted picture;

y _1r ⁽¹⁾ (x, y) represents the edge structure feature of the reference picture when n=1 at point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ represents a non-zero constant to ensure the stability of the equation;

According to the obtained edge structure features and edge saliency features, the similarity of the obtained edge saliency is calculated, and the calculation formula is as follows:

S ₂ (x,y)=MS ₁ (x,y) ^α ·MS ₂ (x,y)

in,

S ₂ (x, y) represents the point (x, y) edge saliency similarity;

α represents the weight of MS ₁ (x, y) in S ₂ (x, y);

MS ₁ (x,y) represents the edge structure similarity calculated by the similarity calculation function;

MS ₂ (x,y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents the weight factor;

C _LMr ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y);

C _LMd ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y);

T ₂ represents a non-zero constant to ensure the stability of the equation;

∑ _(x,y) w ₁ (x,y) represents the accumulation of w ₁ (x,y) at all points on the picture;

C _LCMr ⁽¹⁾ (x, y) represents the LCM mask feature of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents the LCM mask feature of the distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents the LCM mask feature of the distorted picture when n=2 at point (x, y);

T ₃ represents a non-zero constant to ensure the stability of the equation.

Preferably, the feature combination module:

According to the obtained edge structure similarity and edge saliency similarity, the local quality similarity is calculated, and the calculation formula is as follows:

S _QM (x,y)=(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

=(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ

μ=ψ·α

in,

S _QM (x, y) represents the local quality similarity at point (x, y);

ξ represents the weight of S ₁ (x, y) in the local mass S _QM (x, y);

ψ represents the weight of MS ₁ (x, y) in the local mass S _QM (x, y);

μ represents the weight of MS ₂ (x,y) in the local mass S _QM (x,y);

α represents the weight occupied by MS ₁ (x, y) in S ₂ (x, y).

The feature pooling module:

According to the obtained local quality map similarity, the final objective evaluation score is calculated and obtained, and the calculation formula is as follows:

w ₂ (x,y)=max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y))

in,

S represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter.

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at point (x, y).

According to a computer-readable storage medium storing a computer program provided by the present invention, when the computer program is executed by a processor, the steps of any one of the above-mentioned methods for evaluating objective picture quality based on joint multi-scale picture features are realized. .

Compared with the prior art, the present invention has the following beneficial effects:

The present invention has higher accuracy in evaluating the quality of desktop pictures. It is found through the verification of the existing database that the comprehensive performance is more superior than the prior art, and the distortion types of desktop pictures: Gaussian blur, motion blur and JPEG2000 compression distortion have outstanding performance. .

Description of drawings

Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a schematic diagram of a processing flow provided by the present invention.

Detailed ways

The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the inventive concept. These all belong to the protection scope of the present invention.

According to a method for evaluating objective quality of pictures based on joint multi-scale picture features, the method includes:

Image processing steps: use Gaussian pyramid and Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively, and extract edge structure features from y ₁ ⁽ⁿ⁾ ;

Edge saliency feature extraction step: Using the brightness mask and the contrast mask, extract the edge saliency from the image group y ₀ ⁽ⁿ⁾ processed by the Gaussian pyramid and the image group y ₁ ⁽ n) processed by the Laplacian pyramid. feature;

Feature similarity calculation step: According to the obtained edge structure feature and edge saliency feature, calculate and obtain the edge structure similarity and edge saliency similarity;

Feature combination step: Calculate and obtain the final local quality map according to the obtained edge structure similarity and edge saliency similarity;

Feature pooling step: Calculate the final objective evaluation score according to the obtained final local quality map.

Specifically, the picture processing steps:

Use the Gaussian pyramid and the Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively.

Specifically, the edge salient feature extraction step:

Using the luminance mask and the contrast mask, the edge saliency features are extracted from the image group y ₀ ⁽ⁿ⁾ processed by the Gaussian pyramid and the image group y ₁ ⁽ n) processed by the Laplacian pyramid, and the calculation formula is as follows:

in,

C _LM ⁽ⁿ⁾ represents the calculation result of the luminance mask, which is the image feature of the image group processed based on the Gaussian pyramid and the Laplacian pyramid;

表示y函数；

represents the y function;

y ₁ ⁽ⁿ⁾ represents the image group after Laplacian pyramid processing;

y ₀ ⁽ⁿ⁾ represents the image group after Gaussian pyramid processing;

n represents the number of layers;

The image features represented by layer y ₁ ⁽¹⁾ are edge structure features;

表示将y₀ ⁽ⁿ⁺¹⁾带入y函数的计算结果；

Indicates the calculation result of bringing y ₀ ⁽ⁿ⁺¹⁾ into the y function;

γ ₁ represents the brightness contrast threshold;

|| means taking the absolute value operation;

a ₁ represents a constant that guarantees the stability of the equation;

C _LCM ⁽ⁿ⁾ represents the calculation result of the contrast mask, which is based on the image features of the image group processed by C _LM ⁽ⁿ⁾ ;

n represents the number of layers;

C _LCM ⁽¹⁾ indicates that the image features displayed when n=1 are edge saliency features;

a ₂ represents a constant that guarantees the stability of the equation;

表示将C_LM ⁽ⁿ⁺¹⁾带入y函数的计算结果；

Represents the calculation result of bringing C _LM ⁽ⁿ⁺¹⁾ into the y function;

γ ₂ represents the contrast detectable threshold;

G(x, y; σ) represents the Gaussian kernel function;

* means convolution;

↑2 means upsampling.

Specifically, the feature similarity calculation steps:

According to the obtained edge structure features and edge saliency features, the edge structure similarity is calculated and obtained, and the calculation formula is as follows:

in,

S ₁ (x, y) represents the structural similarity of the point (x, y) edge;

The subscripts r and d respectively indicate that the feature is taken from the reference picture or the distorted picture;

y _1r ⁽¹⁾ (x, y) represents the edge structure feature of the reference picture when n=1 at point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ represents a non-zero constant to ensure the stability of the equation;

According to the obtained edge structure features and edge saliency features, the similarity of the obtained edge saliency is calculated, and the calculation formula is as follows:

S ₂ (x,y)=MS ₁ (x,y) ^α ·MS ₂ (x,y)

in,

S ₂ (x, y) represents the point (x, y) edge saliency similarity;

α represents the weight of MS ₁ (x, y) in S ₂ (x, y);

MS ₁ (x,y) represents the edge structure similarity calculated by the similarity calculation function;

MS ₂ (x,y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents the weight factor;

C _LMr ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y);

C _LMd ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y);

T ₂ represents a non-zero constant to ensure the stability of the equation;

∑ _(x,y) w ₁ (x,y) represents the accumulation of w ₁ (x,y) at all points on the picture;

C _LCMr ⁽¹⁾ (x, y) represents the LCM mask feature of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents the LCM mask feature of the distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents the LCM mask feature of the distorted picture when n=2 at point (x, y);

T ₃ represents a non-zero constant to ensure the stability of the equation.

Specifically, the feature combining step:

According to the obtained edge structure similarity and edge saliency similarity, the local quality similarity is calculated, and the calculation formula is as follows:

S _QM (x,y)=(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

=(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ

μ=ψ·α

in,

S _QM (x, y) represents the local quality similarity at point (x, y);

ξ represents the weight of S ₁ (x, y) in the local mass S _QM (x, y);

ψ represents the weight of MS ₁ (x, y) in the local mass S _QM (x, y);

μ represents the weight of MS ₂ (x,y) in the local mass S _QM (x,y);

α represents the weight occupied by MS ₁ (x, y) in S ₂ (x, y).

Specifically, the feature pooling steps:

According to the obtained local quality map similarity, the final objective evaluation score is calculated and obtained, and the calculation formula is as follows:

w ₂ (x,y)=max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y))

in,

S represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter.

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at point (x, y).

The objective quality evaluation system for desktop content pictures based on joint multi-scale picture features provided by the present invention can be implemented through the steps of the method for objective picture quality evaluation based on joint multi-scale picture features provided by the present invention. Those skilled in the art can understand the method for evaluating the objective quality of pictures based on joint multi-scale picture features as a preferred example of the system for evaluating objective quality of desktop content pictures based on joint multi-scale picture features.

An objective quality evaluation system for desktop content pictures based on joint multi-scale picture features provided according to the present invention includes:

Image processing module: use Gaussian pyramid and Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively, and extract edge structure features from y ₁ ⁽ⁿ⁾ ;

Edge saliency feature extraction module: Using brightness mask and contrast mask, extract edge saliency from Gaussian pyramid processed image group y ₀ ⁽ⁿ⁾ and Laplacian pyramid processed image group y ₁ ⁽ⁿ⁾ feature;

Feature similarity calculation module: According to the obtained edge structure features and edge saliency features, calculate the edge structure similarity and edge saliency similarity;

Feature combination module: Calculate and obtain the final local quality map according to the obtained edge structure similarity and edge saliency similarity;

Feature pooling module: Calculate the final objective evaluation score according to the obtained final local quality map.

Specifically, the picture processing module:

Use the Gaussian pyramid and the Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively;

The edge salient feature extraction module:

Using the luminance mask and the contrast mask, the edge saliency features are extracted from the image group y ₀ ⁽ⁿ⁾ processed by the Gaussian pyramid and the image group y ₁ ⁽ n) processed by the Laplacian pyramid. The calculation formula is as follows:

in,

C _LM ⁽ⁿ⁾ represents the calculation result of the luminance mask, which is the image feature of the image group processed based on the Gaussian pyramid and the Laplacian pyramid;

表示y函数；

represents the y function;

y ₁ ⁽ⁿ⁾ represents the image group after Laplacian pyramid processing;

y ₀ ⁽ⁿ⁾ represents the image group after Gaussian pyramid processing;

n represents the number of layers;

The image features represented by layer y ₁ ⁽¹⁾ are edge structure features;

表示将y₀ ⁽ⁿ⁺¹⁾带入y函数的计算结果；

Indicates the calculation result of bringing y ₀ ⁽ⁿ⁺¹⁾ into the y function;

γ ₁ represents the brightness contrast threshold;

|| means taking the absolute value operation;

a ₁ represents a constant that guarantees the stability of the equation;

C _LCM ⁽ⁿ⁾ represents the calculation result of the contrast mask, which is based on the image features of the image group processed by C _LM ⁽ⁿ⁾ ;

n represents the number of layers;

C _LCM ⁽¹⁾ indicates that the image features displayed when n=1 are edge saliency features;

a ₂ represents a constant that guarantees the stability of the equation;

表示将C_LM ⁽ⁿ⁺¹⁾带入y函数的计算结果；

Represents the calculation result of bringing C _LM ⁽ⁿ⁺¹⁾ into the y function;

γ ₂ represents the contrast detectable threshold;

G(x, y; σ) represents the Gaussian kernel function;

* means convolution;

↑2 means upsampling;

The feature similarity calculation module:

According to the obtained edge structure features and edge saliency features, the edge structure similarity is calculated and obtained, and the calculation formula is as follows:

in,

S ₁ (x, y) represents the structural similarity of the point (x, y) edge;

The subscripts r and d respectively indicate that the feature is taken from the reference picture or the distorted picture;

y _1r ⁽¹⁾ (x, y) represents the edge structure feature of the reference picture when n=1 at point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ represents a non-zero constant to ensure the stability of the equation;

According to the obtained edge structure features and edge saliency features, the similarity of the obtained edge saliency is calculated, and the calculation formula is as follows:

S ₂ (x,y)=MS ₁ (x,y) ^α ·MS ₂ (x,y)

in,

S ₂ (x, y) represents the point (x, y) edge saliency similarity;

α represents the weight of MS ₁ (x, y) in S ₂ (x, y);

MS ₁ (x,y) represents the edge structure similarity calculated by the similarity calculation function;

MS ₂ (x,y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents the weight factor;

C _LMr ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y);

C _LMd ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y);

T ₂ represents a non-zero constant to ensure the stability of the equation;

∑ _(x,y) w ₁ (x,y) represents the accumulation of w ₁ (x,y) at all points on the picture;

C _LCMr ⁽¹⁾ (x, y) represents the LCM mask feature of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents the LCM mask feature of the distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents the LCM mask feature of the distorted picture when n=2 at point (x, y);

T ₃ represents a non-zero constant to ensure the stability of the equation.

Specifically, the feature combination module:

According to the obtained edge structure similarity and edge saliency similarity, the local quality similarity is calculated, and the calculation formula is as follows:

S _QM (x,y)=(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

=(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ

μ=ψ·α

in,

S _QM (x, y) represents the local quality similarity at point (x, y);

ξ represents the weight of S ₁ (x, y) in the local mass S _QM (x, y);

ψ represents the weight of MS ₁ (x, y) in the local mass S _QM (x, y);

μ represents the weight of MS ₂ (x,y) in the local mass S _QM (x,y);

α represents the weight occupied by MS ₁ (x, y) in S ₂ (x, y).

The feature pooling module:

According to the obtained local quality map similarity, the final objective evaluation score is calculated and obtained, and the calculation formula is as follows:

w ₂ (x,y)=max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y))

in,

S represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter.

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at point (x, y).

According to a computer-readable storage medium storing a computer program provided by the present invention, when the computer program is executed by a processor, the steps of any one of the above-mentioned methods for evaluating objective picture quality based on joint multi-scale picture features are realized. .

Hereinafter, the present invention will be described in more detail through preferred examples.

Preferred Example 1:

The invention proposes an objective quality evaluation method based on a joint multi-scale desktop content picture, which extracts the edge structure feature and edge saliency feature of the picture to evaluate the distortion degree. Specifically, combined with the characteristics of the human visual system, a picture feature extraction method is designed, and the extracted picture features include two:

1. Edge structure characteristics,

2. Edge saliency features.

To achieve the above object, the present invention has adopted the following technical solutions:

1. Use the Gaussian pyramid and the Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively; let n=1 extract the edge structure from y ₁ ⁽ⁿ⁾ feature;

2. Use Luminance Masking (LM) and Contrast Masking (LCM) to extract edge saliency features from the two pyramids. The specific calculation methods are as follows:

G(x,y；σ)表示高斯核函数，*表示卷积，↑2表示上采样。in

G(x, y; σ) represents the Gaussian kernel function, * represents convolution, and ↑2 represents upsampling.

The brightness mask calculation result C _LM ⁽ⁿ⁾ can detect the brightness change recognizable by the human eye, and γ ₁ is set to 1 according to the Buchsbaum curve; the contrast mask calculation result C _LCM ⁽ⁿ⁾ can detect the contrast change recognizable by the human eye. , and γ ₂ is set to 0.62 according to the contrast detectable threshold.

Let n=1 extract edge saliency features from C _LCM ⁽ⁿ⁾ .

3. Feature similarity calculation

Edge structure similarity:

S ₂ (x,y)=MS ₁ (x,y) ^α ·MS ₂ (x,y),

Edge saliency similarity:

Among them, the subscripts r and d respectively indicate that the feature is taken from the reference picture or the distorted picture, w ₁ (x, y)=y _1r ⁽²⁾ , T ₁ , T ₂ , T ₃ are respectively 0.07, 1×10-50 , 0.01.

4. Feature combination

The final local mass map is:

S _QM (x,y)=(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

=(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ ,

分别设为1.8,0.02,0.9。where μ=ψ·α,

Set to 1.8, 0.02, 0.9 respectively.

5. Feature pooling

Final objective evaluation score:

where w ₂ =max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y)).

In order to illustrate the effectiveness of the above models, tests are performed on SIQAD, an authoritative database of desktop content images. The SIQAD database includes 20 reference pictures, each of which corresponds to 980 distorted pictures of 7 magnitudes of 7 kinds of distortion. The 7 kinds of distortion include Gaussian Noise (GN), Gaussian Blur (GB), Motion Blur (MB), Contrast Change (CC), JPEG Compression (JPEG), JPEG2000 Compression (J2K), Layer Valid Coding (LSC).

Three indicators proposed by the VQEG expert group to measure the consistency of subjective scores and objective evaluation scores are used to judge the superiority of the model. These three indicators are the Pearson linear correlation coefficient (PLCC). , Root mean squared error (RMSE) and Spearman rank-order correlation coefficient (SROCC) are calculated as follows:

分别代表主观分数与客观分数的平均值，d_i代表第i张图片主观分数排序的顺序与客观分数排序的顺序的差值。PLCC与SROCC的值位于0到1之间，越接近1说明主观分数与客观分数一致性越好；RMSE值越小，说明主观分数与客观分数之前的差别越小，模型的性能也就越好。Among them, m and Q represent the subjective score and objective score, respectively,

represent the average of subjective scores and objective scores, respectively, and d _i represents the difference between the order of subjective scores and the order of objective scores for the i-th picture. The values of PLCC and SROCC are between 0 and 1. The closer to 1, the better the consistency between the subjective score and the objective score; the smaller the RMSE value, the smaller the difference between the subjective score and the objective score, and the better the performance of the model. .

Table 1 shows the test results on the database SIQAD, where PSNR, SSIM, MSSIM, IWSSIM, VIF, IFC, FSIM, SCQ are the quality evaluation methods designed for natural pictures, SIQM, SQI, ESIM, MDOGS, GFM are In recent years, the objective quality evaluation methods designed for desktop content pictures can be seen by comparing the data of each method:

For overall performance, the present invention ranks first in PLCC, RMSE evaluation index, and SROCC ranks second;

For a single distortion type, the present invention has won 9 first places and 1 third place, which is obviously better than other methods, and has significant advantages in evaluating the distortion types GB, MB, J2K.

Table 1 SIQAD database test results:

Those skilled in the art know that, in addition to implementing the system, device and each module provided by the present invention in the form of pure computer readable program code, the system, device and each module provided by the present invention can be completely implemented by logically programming the method steps. The same program is implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, and embedded microcontrollers, among others. Therefore, the system, device and each module provided by the present invention can be regarded as a kind of hardware component, and the modules used for realizing various programs included in it can also be regarded as the structure in the hardware component; A module for realizing various functions can be regarded as either a software program for realizing a method or a structure within a hardware component.

Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essential content of the present invention. The embodiments of the present application and features in the embodiments may be combined with each other arbitrarily, provided that there is no conflict.

Claims (10)

1. a picture objective quality evaluation method based on joint multi-scale picture features, is characterized in that, comprises: Image processing steps: use Gaussian pyramid and Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively, and extract edge structure features from y ₁ ⁽ⁿ⁾ ; Edge saliency feature extraction step: Using the brightness mask and the contrast mask, extract the edge saliency from the image group y ₀ ⁽ⁿ⁾ processed by the Gaussian pyramid and the image group y ₁ ⁽ n) processed by the Laplacian pyramid. feature; Feature similarity calculation step: According to the obtained edge structure feature and edge saliency feature, calculate and obtain the edge structure similarity and edge saliency similarity; Feature combination step: Calculate and obtain the final local quality map according to the obtained edge structure similarity and edge saliency similarity; Feature pooling step: Calculate the final objective evaluation score according to the obtained final local quality map. 2. the picture objective quality evaluation method based on joint multi-scale picture feature according to claim 1, is characterized in that, described picture processing step: Use the Gaussian pyramid and the Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively. 3. the picture objective quality evaluation method based on joint multi-scale picture feature according to claim 2, is characterized in that, described edge salient feature extraction step: Using the luminance mask and the contrast mask, the edge saliency features are extracted from the image group y ₀ ⁽ⁿ⁾ processed by the Gaussian pyramid and the image group y ₁ ⁽ n) processed by the Laplacian pyramid. The calculation formula is as follows:

in, C _LM ⁽ⁿ⁾ represents the calculation result of the luminance mask, which is the image feature of the image group processed based on the Gaussian pyramid and the Laplacian pyramid;

represents the y function; y ₁ ⁽ⁿ⁾ represents the image group after Laplacian pyramid processing; y ₀ ⁽ⁿ⁾ represents the image group after Gaussian pyramid processing; n represents the number of layers; The image features represented by layer y ₁ ⁽¹⁾ are edge structure features;

Indicates the calculation result of bringing y ₀ ⁽ⁿ⁺¹⁾ into the y function; γ ₁ represents the brightness contrast threshold; || means taking the absolute value operation; a ₁ represents a constant that guarantees the stability of the equation; C _LCM ⁽ⁿ⁾ represents the calculation result of the contrast mask, which is based on the image features of the image group processed by C _LM ⁽ⁿ⁾ ; n represents the number of layers; C _LCM ⁽¹⁾ indicates that the image features displayed when n=1 are edge saliency features; a ₂ represents a constant that guarantees the stability of the equation;

Represents the calculation result of bringing C _LM ⁽ⁿ⁺¹⁾ into the y function; γ ₂ represents the contrast detectable threshold; G(x, y; σ) represents the Gaussian kernel function; * means convolution; ↑2 means upsampling. 4. the picture objective quality evaluation method based on joint multi-scale picture feature according to claim 3, is characterized in that, described feature similarity calculation step: According to the obtained edge structure features and edge saliency features, the edge structure similarity is calculated and obtained, and the calculation formula is as follows:

in, S ₁ (x, y) represents the structural similarity of the point (x, y) edge; The subscripts r and d respectively indicate that the feature is taken from the reference picture or the distorted picture; y _1r ⁽¹⁾ (x, y) represents the edge structure feature of the reference picture when n=1 at point (x, y); y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y); T ₁ represents a non-zero constant to ensure the stability of the equation; According to the obtained edge structure features and edge saliency features, the similarity of the obtained edge saliency is calculated, and the calculation formula is as follows: S ₂ (x,y)=MS ₁ (x,y) ^α ·MS ₂ (x,y)

in, S ₂ (x, y) represents the point (x, y) edge saliency similarity; α represents the weight of MS ₁ (x, y) in S ₂ (x, y); MS ₁ (x,y) represents the edge structure similarity calculated by the similarity calculation function; MS ₂ (x,y) represents the edge significant similarity calculated by the similarity calculation function; w ₁ (x, y) represents the weight factor; C _LMr ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y); C _LMd ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y); T ₂ represents a non-zero constant to ensure the stability of the equation; ∑ _(x,y) w ₁ (x,y) represents the accumulation of w ₁ (x,y) at all points on the picture; C _LCMr ⁽¹⁾ (x, y) represents the LCM mask feature of the reference picture when n=1 at point (x, y); C _LCMd ⁽¹⁾ (x, y) represents the LCM mask feature of the distorted picture when n=1 at point (x, y); C _LCMd ⁽²⁾ (x, y) represents the LCM mask feature of the distorted picture when n=2 at point (x, y); T ₃ represents a non-zero constant to ensure the stability of the equation. 5. the picture objective quality evaluation method based on joint multi-scale picture feature according to claim 4, is characterized in that, described feature combining step: According to the obtained edge structure similarity and edge saliency similarity, the local quality similarity is calculated, and the calculation formula is as follows: S _QM (x,y)=(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ =(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ μ=ψ·α in, S _QM (x, y) represents the local quality similarity at point (x, y); ξ represents the weight of S ₁ (x, y) in the local mass S _QM (x, y); ψ represents the weight of MS ₁ (x, y) in the local mass S _QM (x, y); μ represents the weight of MS ₂ (x,y) in the local mass S _QM (x,y); α represents the weight occupied by MS ₁ (x, y) in S ₂ (x, y). 6. the picture objective quality evaluation method based on joint multi-scale picture feature according to claim 5, is characterized in that, described feature pooling step: According to the obtained local quality map similarity, the final objective evaluation score is calculated and obtained, and the calculation formula is as follows:

w ₂ (x,y)=max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y)) in, S represents the final objective evaluation score; w ₂ (x, y) represents a weight parameter. y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at point (x, y). 7. A desktop content picture objective quality evaluation system based on joint multi-scale picture features, characterized in that, comprising: Image processing module: use Gaussian pyramid and Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively, and extract edge structure features from y ₁ ⁽ⁿ⁾ ; Edge saliency feature extraction module: Using brightness mask and contrast mask, extract edge saliency from Gaussian pyramid processed image group y ₀ ⁽ⁿ⁾ and Laplacian pyramid processed image group y ₁ ⁽ⁿ⁾ feature; Feature similarity calculation module: According to the obtained edge structure features and edge saliency features, calculate the edge structure similarity and edge saliency similarity; Feature combination module: Calculate and obtain the final local quality map according to the obtained edge structure similarity and edge saliency similarity; Feature pooling module: Calculate the final objective evaluation score according to the obtained final local quality map. 8. The system for evaluating the objective quality of desktop content pictures based on joint multi-scale picture features according to claim 7, wherein the picture processing module: Use the Gaussian pyramid and the Laplacian pyramid to process the original image into image groups of different scales, denoted as y ₀ ⁽ⁿ⁾ and y ₁ ⁽ⁿ⁾ respectively; The edge salient feature extraction module: Using the luminance mask and the contrast mask, the edge saliency features are extracted from the image group y ₀ ⁽ⁿ⁾ processed by the Gaussian pyramid and the image group y ₁ ⁽ n) processed by the Laplacian pyramid. The calculation formula is as follows:

in, C _LM ⁽ⁿ⁾ represents the calculation result of the luminance mask, which is the image feature of the image group processed based on the Gaussian pyramid and the Laplacian pyramid;

represents the y function; y ₁ ⁽ⁿ⁾ represents the image group after Laplacian pyramid processing; y ₀ ⁽ⁿ⁾ represents the image group after Gaussian pyramid processing; n represents the number of layers; The image features represented by layer y ₁ ⁽¹⁾ are edge structure features;

Indicates the calculation result of bringing y ₀ ⁽ⁿ⁺¹⁾ into the y function; γ ₁ represents the brightness contrast threshold; || means taking the absolute value operation; a ₁ represents a constant that guarantees the stability of the equation; C _LCM ⁽ⁿ⁾ represents the calculation result of the contrast mask, which is based on the image features of the image group processed by C _LM ⁽ⁿ⁾ ; n represents the number of layers; C _LCM ⁽¹⁾ indicates that the image features displayed when n=1 are edge saliency features; a ₂ represents a constant that guarantees the stability of the equation;

Represents the calculation result of bringing C _LM ⁽ⁿ⁺¹⁾ into the y function; γ ₂ represents the contrast detectable threshold; G(x, y; σ) represents the Gaussian kernel function; * means convolution; ↑2 means upsampling; The feature similarity calculation module: According to the obtained edge structure features and edge saliency features, the edge structure similarity is calculated and obtained, and the calculation formula is as follows:

in, S ₁ (x, y) represents the structural similarity of the point (x, y) edge; The subscripts r and d respectively indicate that the feature is taken from the reference picture or the distorted picture; y _1r ⁽¹⁾ (x, y) represents the edge structure feature of the reference picture when n=1 at point (x, y); y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y); T ₁ represents a non-zero constant to ensure the stability of the equation; According to the obtained edge structure features and edge saliency features, the similarity of the obtained edge saliency is calculated, and the calculation formula is as follows: S ₂ (x,y)=MS ₁ (x,y) ^α ·MS ₂ (x,y)

in, S ₂ (x, y) represents the point (x, y) edge saliency similarity; α represents the weight of MS ₁ (x, y) in S ₂ (x, y); MS ₁ (x,y) represents the edge structure similarity calculated by the similarity calculation function; MS ₂ (x,y) represents the edge significant similarity calculated by the similarity calculation function; w ₁ (x, y) represents the weight factor; C _LMr ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y); C _LMd ⁽¹⁾ (x, y) represents the LM mask feature of the reference picture when n=1 at point (x, y); T ₂ represents a non-zero constant to ensure the stability of the equation; ∑ _(x,y) w ₁ (x,y) represents the accumulation of w ₁ (x,y) at all points on the picture; C _LCMr ⁽¹⁾ (x, y) represents the LCM mask feature of the reference picture when n=1 at point (x, y); C _LCMd ⁽¹⁾ (x, y) represents the LCM mask feature of the distorted picture when n=1 at point (x, y); C _LCMd ⁽²⁾ (x, y) represents the LCM mask feature of the distorted picture when n=2 at point (x, y); T ₃ represents a non-zero constant to ensure the stability of the equation. 9. the desktop content picture objective quality evaluation system based on joint multi-scale picture feature according to claim 8, is characterized in that, described feature combination module: According to the obtained edge structure similarity and edge saliency similarity, the local quality similarity is calculated, and the calculation formula is as follows: S _QM (x,y)=(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ =(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ μ=ψ·α in, S _QM (x, y) represents the local quality similarity at point (x, y); ξ represents the weight of S ₁ (x, y) in the local mass S _QM (x, y); ψ represents the weight of MS ₁ (x, y) in the local mass S _QM (x, y); μ represents the weight of MS ₂ (x,y) in the local mass S _QM (x,y); α represents the weight occupied by MS ₁ (x, y) in S ₂ (x, y). The feature pooling module: According to the obtained local quality map similarity, the final objective evaluation score is calculated and obtained, and the calculation formula is as follows:

w ₂ (x,y)=max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y)) in, S represents the final objective evaluation score; w ₂ (x, y) represents a weight parameter. y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at point (x, y). 10. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the picture objective based on joint multi-scale picture features according to any one of claims 1 to 6 is realized The steps of the quality assessment method.

Images