WO2021135391A1

WO2021135391A1 - Image quality evaluation method and apparatus

Info

Publication number: WO2021135391A1
Application number: PCT/CN2020/115341
Authority: WO
Inventors: 王员根; 区富炤; 李进
Original assignee: 广州大学
Priority date: 2020-01-02
Filing date: 2020-09-15
Publication date: 2021-07-08
Also published as: JP2022519469A; CN111192258A

Abstract

An image quality evaluation method and apparatus. The method comprises: first obtaining a sample image, and then adjusting a brightness component of the sample image according to a shape parameter and different distortion levels to obtain a first training image set (S101); processing a first training image according to a preset image processor to obtain a second training image set (S102); performing image processing on the sample image according to a preset distortion type and different distortion levels to obtain a composite distortion training image set (S103); constructing an image quality evaluation model by taking the training image set, the distortion levels of images in the training image set, and a standard quality evaluation score of the sample image as inputs (S104); and obtaining an image to be evaluated, and inputting the image to be evaluated into an image quality evaluation model to generate a prediction quality evaluation score of the image to be evaluated (S105). By implementing this method, quality score evaluation can be performed on an image distorted by a single distortion factor, and quality score evaluation can also be performed on an image distorted by fusion of multiple distortion factors.

Description

Image quality evaluation method and device

Technical field

The present invention relates to the technical field of image processing, in particular to an image quality evaluation method and device.

Background technique

A large part of social media interaction involves sharing pictures. However, the image will always introduce more or less distortion in the process of acquisition, post-processing, transmission and storage. Therefore, image quality assessment has become an important research topic. In real applications, reference images are not always available. For example, on social platforms, users share pictures, and their friends browse the pictures without any reference. This promotes the reference-free image quality assessment to become the most extensive and deepest research field in the field of machine perception. Usually the image suffers from two different types of distortion. One is composite distortion with a specific type of distortion caused by a single factor such as fast fading, white noise, pink Gaussian noise, JPEG, JPEG2000, Gaussian blur, global contrast reduction, and so on. The other is the real distortion introduced by the camera in the process of capturing, processing and storing. The true distortion caused by the fusion of multiple distortion factors such as overexposure, underexposure, motion-induced blur, low-light noise, and compression error during the shooting process has no specific type of distortion.

However, the existing technology for image quality evaluation is limited to the evaluation of images with specific distortion types, that is, the above-mentioned synthetic distortion types. For example, the application number "CN201910364614.0" "an image quality based on the image distortion type" Evaluation method", which first uses Discrete Cosine Transform (DCT) to express the information of the high-dimensional image on the low-dimensional image, and extract the distortion feature value. Then, use support vector machine (SVM) to build an SVM classifier model with labels 1, 2, 3, 4,..., n, divide the distortion types into n types, and then input the distortion feature values of the image, and pass The decision function classifies the selected image into distortion types. Finally, according to the distortion type, the image quality prediction score is calculated in the corresponding regression evaluation model. This method requires the classification of distortion types, and for those without a clear definition of distortion types, which are formed by the fusion of various distortion factors, the real distortion caused by it is not applicable, and it cannot be performed on the real distortion caused by the fusion of multiple distortion factors. Image evaluation.

Summary of the invention

The embodiment of the present invention provides an image quality evaluation method and device, which can not only evaluate the quality score of a distorted image with a clear distortion type caused by a single distortion factor, but also can determine the lack of clarity caused by the fusion of multiple distortion factors. The quality score of the distorted image of the distortion type is evaluated.

An embodiment of the present invention provides an image quality evaluation method, including:

Acquiring a sample image, and then adjusting the brightness component of the sample image according to preset shape parameters and different distortion levels to obtain a first training image set; wherein the first training image set includes images with different distortion levels;

Each image in the first training image set is processed by a preset image processor to obtain a second training image set; wherein, the preset image processor includes any one or more of the following combinations: motion filtering Detector, Gaussian low-pass filter, chromatic aberration transform processor and global contrast degradation image processor;

Perform image processing on the sample images according to preset distortion types and different distortion levels to obtain synthetic distortion training image sets; wherein, the preset distortion types include: Gaussian blur, JPEG compression distortion, JPEG2000 compression distortion, fast decay Distortion or Gaussian white noise; the synthetic distortion training image set contains synthetic distortion images with different distortion levels;

Taking a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image as input, construct an image quality evaluation model through a neural network; wherein, the training image set includes a second training image Set or synthetic distortion training image set;

The image to be evaluated is acquired, and the image to be evaluated is input into the image quality evaluation model to obtain the predicted quality evaluation score of the image to be evaluated.

Further, after the preset image processor processes each image in the first training image set to obtain a second training image set, the method further includes: performing the second training on the second training image set with a preset probability Each image in the image set is compressed.

Further, the adjusting the brightness component of the sample image according to preset shape parameters and different distortion levels to obtain the first training image set specifically includes:

The brightness component of the sample image is adjusted by the following overexposure processing function:

Wherein, λ ₁ , δ ₁ , γ ₁ and ν ₁ are shape parameters, k is the distortion level, L is the luminance component, i is the row coordinate corresponding to the luminance component, and j is the ordinate corresponding to the luminance component.

The brightness component of the sample image is adjusted by the following underexposure processing function:

Wherein, λ ₂ , δ ₂ , γ ₂ and ν ₂ are shape parameters, k is the distortion level, and L is the brightness component.

Further, said taking a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image as input, and constructing an image quality evaluation model through a neural network, specifically includes:

Take a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image as input, and input it into the neural network for pre-training until the total loss function of the neural network converges; The total loss function of the divine path network during training is:

among them,

Is the adaptive sorting loss function,

Is the upper bound control loss function,

Is the lower bound control loss function,

The training image set can be expressed as

Is the sample image, y ₀ is the standard quality evaluation score of the sample image, θ is the network parameter, k is the distortion level,

Is the dynamic change compensation operator, τ _w is the preset quality score upper limit, τ _b is the preset quality score lower limit, n and m are the index numbers of the descending image column, λ _r is the weight value of the adaptive sorting loss function, λ _b is the weight value of the upper bound control loss function, and λ _w is the weight value of the lower bound control loss function;

Input a set of sample images into the initial image quality assessment model, and perform fine-tuning training until the corresponding loss function converges during fine-tuning training to obtain the image quality assessment model; wherein, the sample image set includes a number of the samples Image; the corresponding loss function during the fine-tuning training is:

N is the number of images in the sample image set, π is the second network parameter, I _i is the sample image in the sample image set, and _yi is the standard quality evaluation score of the sample image in the sample image.

On the basis of the foregoing method item embodiments, the present invention provides corresponding device item embodiments;

An embodiment of the present invention provides an image evaluation device, including a first image processing module, a second image processing module, a third image processing module, an image quality evaluation model construction module, and an image evaluation module;

The first image processing module is configured to obtain a sample image, and then adjust the brightness component of the sample image according to preset shape parameters and different distortion levels to obtain a first training image set; wherein, the first training The image set contains images with different distortion levels;

The second image processing module is configured to process each image in the first training image set according to a preset image processor to obtain a second training image set; wherein, the preset image processor includes the following Any one or more combinations: motion filter, Gaussian low-pass filter, color difference transform processor and global contrast degradation image processor;

The third image processing module is configured to perform image processing on the sample images according to preset distortion types and different distortion levels to obtain a synthetic distortion training image set; wherein, the preset distortion types include: Gaussian blur, JPEG compression distortion, JPEG2000 compression distortion, fast decay distortion or Gaussian white noise; the synthetic distortion training image set includes synthetic distortion images with different distortion levels;

The image quality evaluation model building module is configured to take a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image as input, and construct an image quality evaluation model through a neural network; wherein , The training image set includes a second training image set or a synthetic distortion training image set;

The image evaluation module is configured to obtain an image to be evaluated, and input the image to be evaluated into the image quality evaluation model to obtain a predicted quality evaluation score of the image to be evaluated.

Further, it also includes a fourth image processing module, which is configured to perform compression processing on each image in the second training image set according to a preset probability.

The following beneficial effects are achieved by implementing the embodiments of the present invention:

The embodiments of the present invention provide an image quality evaluation method and device. The method first adjusts the brightness component of a sample image to simulate over/under exposure distortion of the image, and obtains a first training image set through a preset image processor Process the images of the first training image set, and fuse the distortion factors of over/under exposure distortion with the distortion factors corresponding to the preset image processor to generate a true distortion image set caused by the fusion of multiple distortion factors, namely The second training image set mentioned above, in addition, image processing is performed on the sample images through preset distortion types and distortion levels, so as to generate a synthetic distortion image set that is distorted by a specific distortion factor, that is, the aforementioned synthetic distortion training image set Then, according to the above-mentioned second training image set or synthetic distortion training image set, an image quality evaluation model is constructed through a neural network, and finally the image to be evaluated is input into the image quality evaluation model to obtain the corresponding predicted quality evaluation score. Compared with the prior art, the image quality evaluation method provided by the embodiments of the present invention can realize the quality evaluation of the image with a specific type of distortion (ie, the composite distorted image) caused by a single distortion factor. It can realize the quality evaluation of images without specific distortion types (ie true distortion images) that are distorted by the fusion of multiple distortion factors.

Description of the drawings

FIG. 1 is a schematic flowchart of an image quality evaluation method provided by an embodiment of the present invention.

Fig. 2 is a schematic structural diagram of an image quality evaluation method provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

As shown in FIG. 1, an image quality evaluation method provided by an embodiment of the present invention includes:

Step S101: Obtain a sample image, and then adjust the brightness component of the sample image according to preset shape parameters and different distortion levels to obtain a first training image set; wherein, the first training image set contains different distortion levels image;

Step S102: Process each image in the first training image set according to a preset image processor to obtain a second training image set; wherein, the preset image processor includes any one or more of the following combinations : Motion filter, Gaussian low-pass filter, color difference transform processor and global contrast degradation image processor;

Step S103: Perform image processing on the sample image according to preset distortion types and different distortion levels to obtain a synthetic distortion training image set; wherein the preset distortion types include Gaussian blur, JPEG compression distortion, and JPEG2000 compression distortion , Fast decay distortion or Gaussian white noise; the synthetic distortion training image set contains synthetic distortion images with different distortion levels;

Step S104: Taking a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image as input, construct an image quality evaluation model through a neural network; wherein, the training image set includes the first 2. Training image set or synthetic distortion training image set;

Step S105: Obtain an image to be evaluated, and input the image to be evaluated into the image quality evaluation model to obtain a predicted quality evaluation score of the image to be evaluated.

For step S101: first obtain a sample image, if the sample image is an RGB image, convert it to an HSV image, then extract the brightness component of the HSV image and adjust it to simulate the image distortion caused by over/under exposure distortion , Specifically including the following two adjustment methods:

In a preferred embodiment, the adjusting the brightness component of the sample image according to preset shape parameters and different distortion levels to obtain the first training image set specifically includes:

Wherein, λ ₁ , δ ₁ , γ ₁ and ν ₁ are shape parameters, k is the distortion level, L is the luminance component, i is the row coordinate corresponding to the luminance component, and j is the ordinate corresponding to the luminance component. An embodiment is used to simulate overexposure distortion.

Wherein, λ ₂ , δ ₂ , γ ₂ and ν ₂ are shape parameters, k is the distortion level, and L is the brightness component. This embodiment is used to simulate under-exposure distortion.

The above two brightness component adjustment methods are selected according to actual needs, so as to simulate the simulation of over/under exposure distortion of the sample image; finally, the HSV image with the changed brightness component is converted back to the RGB three-channel image to generate the first training image The first training image set includes images with different distortion levels. It should be noted that when the distortion level is 0, it represents the original sample image.

For step S102: First, the preset image processor is explained: in order to simulate the image distortion of the sample image caused by a single factor or a combination of multiple factors such as jitter, focus error, halo, and contrast distortion, the present invention adopts different image processors. The combination is performed to obtain the above-mentioned preset image processor, and then image processing is performed on the images of the first training image set. Preferably, the preset image processor includes any one or more combinations of the following: a motion filter, a Gaussian low-pass filter, a color difference transformation processor, and a global contrast degradation image processor; that is, the aforementioned preset image processor may It is a single image processor or a combination of several image processors; therefore, this embodiment will be further described.

First we use an image processor to process _It1 :

I _t1 represents the image in the above-mentioned first training image set. l = 1, 2, 3, 4 represent the index numbers of the motion filter, Gaussian low-pass filter, color difference transform, and global contrast degradation image processor, respectively. All combinations of l are defined as a set Ω = {{1}, {2},{3},{4},{1,2},{1,3},...,{1,2,3,4}}. Then execute the fusion strategy to obtain the fusion image _It2 :

i=1,2,...,15, the above-mentioned second training image set is finally generated. When only one image processor is used for processing, such as a motion filter, it is used to simulate the sample image, subject to over/under exposure distortion factors And the factor of image jitter, the distortion caused by the fusion of two distortion factors. When using a combination of two image processors, such as a motion filter and a global contrast degradation processor, it is used to simulate sample images, subject to over/under exposure distortion factors, image jitter factors, and contrast distortion; Distortion caused by the fusion of distortion factors. Through this embodiment, it is possible to simulate the true distortion of the sample image due to the fusion of multiple distortion factors (at least two).

In a preferred embodiment, the method further includes: compressing each image in the second training image set according to a preset probability. In this embodiment, it is preferable to perform JPEG compression processing with a probability of 1/2 on the second training image set. Simulate the situation of compression distortion.

For step S103, it should be noted that this step is named step S103, which does not mean that this step is executed after step S101 and step S102, and it can also be executed before step S101. The naming of the steps is only for the convenience of presentation, not as a limitation of the computer's execution order. Through step S101 and step S102, the simulation of the true distortion of the sample image (distortion caused by the fusion of multiple distortion factors, without a clear type of distortion) is completed, and step S103 is to synthesize the distortion of the sample image (by a single The distortion caused by the distortion factor has a clear type of distortion) simulation. This step is mainly to perform image processing according to the preset distortion type and distortion level. The distortion type includes, but is not limited to, St. Blur, JPEG compression distortion, JPEG2000 compression distortion, fast decay distortion, or Gaussian white noise, one of which is to finally generate a synthetic distortion training image set.

For step S104, a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image are input, and the predicted quality evaluation score of each image in the training image set is the output, which is constructed by a neural network The quality evaluation model is constructed as follows:

among them,

Is the adaptive sorting loss function,

Is the upper bound control loss function,

Is the lower bound control loss function,

The training image set can be expressed as

N is the number of images in the sample image set, π is the second network parameter, I _i is the sample image in the sample image set, and _yi is the standard quality evaluation score of the sample image in the sample image. The traditional sorting learning function, because the sorting interval is constant, and the upper and lower bounds are not limited, so the output result of sorting is easy to get out of control. In this embodiment of the present invention, during pre-training, an adaptive ranking loss function and upper and lower bound control loss functions are set so that the pre-training ranking result can better adapt to the standard quality evaluation score of the image. The standard quality evaluation score mentioned in this article refers to the preset standard score for reference. When the selected training image set is the above-mentioned second training image, the constructed image quality evaluation model can be used to evaluate the real distorted images. If the selected training image set is the above-mentioned synthetic distortion training image set, the constructed image quality evaluation model can be used to evaluate the real distorted images. The image quality evaluation model can be used to evaluate synthetic and distorted images.

For step S105, an image to be evaluated is obtained, and the image to be evaluated is input into the image quality evaluation model to obtain the predicted quality evaluation score of the image to be evaluated, and the user needs to be evaluated according to the obtained predicted quality evaluation score The image is evaluated.

On the basis of the above method item embodiment, the present invention provides a device item embodiment correspondingly;

As shown in FIG. 2, another embodiment of the present invention provides an image quality evaluation device, including a first image processing module, a second image processing module, a third image processing module, an image quality evaluation model building module, and an image evaluation module ；

In a preferred embodiment, it further includes a fourth image processing module; the fourth image processing module is configured to perform compression processing on each image in the second training image set according to a preset probability.

It is understandable that the foregoing device item embodiment corresponds to the method item embodiment of the present invention, and it can implement the image quality evaluation method provided by any one of the foregoing method item embodiments of the present invention.

It should be noted that the device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physically separate. Modules can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that they have a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement without creative work. The schematic diagram is only an example of the image quality evaluation device, and does not constitute a limitation on the image quality evaluation device. It may include more or less components than those shown in the figure, or a combination of some components, or different components.

Compared with the prior art, the image quality evaluation method provided by the embodiment of the present invention can realize the quality evaluation of the image with a specific type of distortion caused by a single distortion factor, and can also realize the quality evaluation of the image with a specific type of distortion caused by a single distortion factor. The quality evaluation of the image without a specific type of distortion caused by the fusion of distortion factors.

The above are the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications are also considered This is the protection scope of the present invention.

Claims

An image quality evaluation method, characterized in that it comprises:

Acquiring a sample image, and then adjusting the brightness component of the sample image according to preset shape parameters and different distortion levels to obtain a first training image set; wherein the first training image set includes images with different distortion levels;

Each image in the first training image set is processed by a preset image processor to obtain a second training image set; wherein, the preset image processor includes any one or more of the following combinations: motion filtering Detector, Gaussian low-pass filter, chromatic aberration transform processor and global contrast degradation image processor;

Perform image processing on the sample images according to preset distortion types and different distortion levels to obtain synthetic distortion training image sets; wherein, the preset distortion types include: Gaussian blur, JPEG compression distortion, JPEG2000 compression distortion, fast decay Distortion or Gaussian white noise; the synthetic distortion training image set contains synthetic distortion images with different distortion levels;

Taking a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image as input, construct an image quality evaluation model through a neural network; wherein, the training image set includes a second training image Set or synthetic distortion training image set;

The image to be evaluated is acquired, and the image to be evaluated is input into the image quality evaluation model to obtain the predicted quality evaluation score of the image to be evaluated.
The image quality evaluation method of claim 1, wherein after the preset image processor processes each image in the first training image set to obtain the second training image set, further It includes: compressing each image in the second training image set according to a preset probability.
5. The image quality evaluation method of claim 1, wherein the adjusting the brightness component of the sample image according to preset shape parameters and different distortion levels to obtain a first training image set specifically comprises:

The brightness component of the sample image is adjusted by the following overexposure processing function:

Wherein, λ 1 , δ 1 , γ 1 and v 1 are shape parameters, k is the distortion level, L is the luminance component, i is the row coordinate corresponding to the luminance component, and j is the ordinate corresponding to the luminance component.
5. The image quality evaluation method of claim 1, wherein the adjusting the brightness component of the sample image according to preset shape parameters and different distortion levels to obtain a first training image set specifically comprises:

The brightness component of the sample image is adjusted by the following underexposure processing function:

Wherein, λ 2 , δ 2 , γ 2 and v 2 are shape parameters, k is the distortion level, L is the luminance component, i is the row coordinate corresponding to the luminance component, and j is the ordinate corresponding to the luminance component.
2. The image quality evaluation method of claim 1, wherein the input is a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image, and the neural network Build an image quality evaluation model, including:

Take a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image as input, and input it into the neural network for pre-training until the total loss function of the neural network converges; The total loss function of the divine path network during training is:

among them,
Is the adaptive sorting loss function,

Is the upper bound control loss function,

Is the lower bound control loss function,

The training image set can be expressed as
Is the sample image, y 0 is the standard quality evaluation score of the sample image, θ is the network parameter, k is the distortion level,
It is a dynamic change compensation operator, φ θ (·) is the network output value, τ w is the preset quality score upper limit, τ b is the preset quality score lower limit, n and m are the index numbers of the descending image sequence, λ r is the weight value of the adaptive sorting loss function, λ b is the weight value of the upper bound control loss function, and λ w is the weight value of the lower bound control loss function;

Input a set of sample images into the initial image quality assessment model, and perform fine-tuning training until the corresponding loss function converges during fine-tuning training to obtain the image quality assessment model; wherein, the sample image set includes a number of the samples Image; the corresponding loss function during the fine-tuning training is:

N is the number of images in the sample image set, π is the second network parameter, I i is the sample image in the sample image set, and yi is the standard quality evaluation score of the sample image in the sample image.
An image quality evaluation device, which is characterized by comprising a first image processing module, a second image processing module, a third image processing module, an image quality evaluation model building module, and an image evaluation module;

The first image processing module is configured to obtain a sample image, and then adjust the brightness component of the sample image according to preset shape parameters and different distortion levels to obtain a first training image set; wherein, the first training The image set contains images with different distortion levels;

The second image processing module is configured to process each image in the first training image set according to a preset image processor to obtain a second training image set; wherein, the preset image processor includes the following Any one or more combinations: motion filter, Gaussian low-pass filter, color difference transform processor and global contrast degradation image processor;

The third image processing module is configured to perform image processing on the sample images according to preset distortion types and different distortion levels to obtain a synthetic distortion training image set; wherein, the preset distortion types include: Gaussian blur, JPEG compression distortion, JPEG2000 compression distortion, fast decay distortion or Gaussian white noise; the synthetic distortion training image set includes synthetic distortion images with different distortion levels;

The image quality evaluation model building module is configured to take a training image set, the distortion level of each image in the training image set, and the standard quality evaluation score of the sample image as input, and construct an image quality evaluation model through a neural network; wherein , The training image set includes a second training image set or a synthetic distortion training image set;

The image evaluation module is configured to obtain an image to be evaluated, and input the image to be evaluated into the image quality evaluation model to obtain a predicted quality evaluation score of the image to be evaluated.
7. The image quality evaluation device of claim 6, further comprising a fourth image processing module; the fourth image processing module is configured to perform a preset probability on each of the second training image sets. The image is compressed.