WO2022105117A1

WO2022105117A1 - Method and device for image quality assessment, computer device, and storage medium

Info

Publication number: WO2022105117A1
Application number: PCT/CN2021/090416
Authority: WO
Inventors: 陈昊
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-11-17
Filing date: 2021-04-28
Publication date: 2022-05-27
Also published as: CN112418292A

Abstract

The present application relates to computer vision technologies in the technical field of artificial intelligence. Disclosed are a method and device for image quality assessment, a computer device, and a storage medium. The present application, by constructing an image-generating network and by training the image-generating network with a training sample set in a preset database, acquires an image feature extractor; receives an image to be assessed and utilizes the image feature extractor to extract an image feature of said image; performs eigenvector transformation of the image feature of said image to transform the image feature into an eigenvector; constructs a network regression function, utilizes the network regression function to calculate a regression value of the eigenvector, and determines the quality of said image on the basis of the regression value of the eigenvector. In addition, the present application also relates to the blockchain technology in that said image can be stored in a blockchain. The present application constructs an image quality assessment system by means of simplifying a deep-learning network and employing a machine regression scheme. The image quality assessment system is quickly adaptable to various scenarios.

Description

A method, device, computer equipment and storage medium for image quality evaluation

This application claims the priority of the Chinese patent application filed on November 17, 2020 with the application number 202011288901.7 and the invention titled "A method, device, computer equipment and storage medium for image quality evaluation", all of which The contents are incorporated herein by reference.

technical field

The present application belongs to the technical field of artificial intelligence, and specifically relates to a method, device, computer equipment and storage medium for image quality evaluation.

Background technique

At present, in a series of intelligent image application fields, judging the quality of the input image is often the key to start a series of subsequent operations. Generally speaking, the general idea of solving image quality assessment is to simulate the idea of human discrimination as much as possible. There are roughly two main solutions for current image quality assessment:

1. Based on the method of defining important indicators, that is, feature engineering is used to determine image quality. This idea is based on SSIM (Structural SIMilarity, structural similarity), FSIM (feature similarity, image quality measurement standard), NIQE (Natural image quality) Evaluator, image quality evaluation) and other evaluation schemes are represented, and the basic idea is to use the definition of indicators to measure the brightness of the image, the clarity of the edge, etc. to determine.

Second, the method based on the convolutional deep learning network, which often uses a neural network to fit the judgment results of people.

However, in the process of image quality evaluation, the inventor realized that the above two methods both have certain defects. For the image quality evaluation method that defines important indicators, it is often limited by the influence of insufficient definition of indicators, resulting in insufficient generalization ability of the method. Moreover, the design of feature indicators often requires designers to have a high level of mathematics and rich experience, and cannot quickly adapt to various scenarios.

However, the image quality evaluation method of convolutional deep learning network has high requirements on the overhead of computing resources, and in some occasions (such as mobile terminals), there will be large usage restrictions. At present, the computing power and memory of mobile terminals have certain limitations. limitations, making it difficult to deploy convolutional deep learning networks.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to propose a method, device, computer equipment and storage medium for image quality evaluation, so as to solve the problem that the existing image quality evaluation solutions have relatively limited application scenarios and cannot quickly adapt to various scenarios. question.

In order to solve the above technical problems, the embodiments of the present application provide a method for image quality evaluation, which adopts the following technical solutions:

A method of image quality evaluation, comprising:

Build an image generation network, train the image generation network through the training sample set in the preset database, and obtain an image feature extractor;

Receive the image to be evaluated, and use the image feature extractor to extract the image features of the image to be evaluated;

Convert the image features of the image to be evaluated into feature vectors, and convert the image features into feature vectors;

Build a network regression function, use the network regression function to calculate the regression value of the feature vector, and determine the quality of the image to be evaluated according to the regression value of the feature vector.

In order to solve the above technical problems, the embodiment of the present application also provides an image quality evaluation device, which adopts the following technical solutions:

A device for evaluating image quality, comprising:

The building module is used to build an image generation network, train the image generation network through the training sample set in the preset database, and obtain an image feature extractor;

an extraction module for receiving the image to be evaluated, and extracting image features of the image to be evaluated by using an image feature extractor;

The transformation module is used to transform the image features of the image to be evaluated into feature vectors, and convert the image features into feature vectors;

The evaluation module is used to construct a network regression function, use the network regression function to calculate the regression value of the feature vector, and determine the quality of the image to be evaluated according to the regression value of the feature vector.

In order to solve the above-mentioned technical problems, the embodiment of the present application also provides a computer device, which adopts the following technical solutions:

A computer device includes a memory and a processor, wherein computer-readable instructions are stored in the memory, and the processor implements the following image quality evaluation method when executing the computer-readable instructions:

In order to solve the above technical problems, the embodiments of the present application also provide a computer-readable storage medium, which adopts the following technical solutions:

A computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the following image quality evaluation method is implemented:

Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects:

The present application discloses an image quality evaluation method, device, computer equipment and storage medium, which belong to the technical field of artificial intelligence. The present application trains the image generation network by constructing an image generation network and using a training sample set in a preset database. Obtain the image feature extractor; receive the image to be evaluated, and use the image feature extractor to extract the image features of the image to be evaluated; perform feature vector transformation on the image features of the image to be evaluated, and convert the image features into feature vectors; build a network regression function, using The network regression function calculates the regression value of the feature vector, and determines the quality of the image to be evaluated according to the regression value of the feature vector. The present application constructs an image quality evaluation system by simplifying the deep learning network and adopting machine regression. When performing image quality evaluation, the image features are first obtained through the trained deep learning network, and then the image features are calculated based on the network regression function. Regression value, and finally determine the quality of the image to be evaluated based on the regression value of the image features. The image quality evaluation system constructed in this application has a simple structure, does not occupy too many server resources, effectively reduces computing resource consumption, and can meet the deployment and use requirements of mobile terminals . At the same time, the image quality is finally evaluated by the network regression function, and a mathematical explanation can be made for the evaluation result, which is convenient for users to analyze the problem intuitively.

Description of drawings

In order to illustrate the solutions in the present application more clearly, the following will briefly introduce the accompanying drawings used in the description of the embodiments of the present application. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 shows an exemplary system architecture diagram to which the present application can be applied;

FIG. 2 shows a flow chart of an embodiment of a method for image quality evaluation according to the present application;

Fig. 3 shows a flowchart of a specific implementation manner of step S201 in Fig. 2;

Fig. 4 shows a flowchart of a specific implementation manner of step S204 in Fig. 2;

FIG. 5 shows a schematic structural diagram of an embodiment of an apparatus for evaluating image quality according to the present application;

FIG. 6 shows a schematic structural diagram of an embodiment of a computer device according to the present application.

Detailed ways

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field of this application; the terms used herein in the specification of the application are for the purpose of describing specific embodiments only It is not intended to limit the application; the terms "comprising" and "having" and any variations thereof in the description and claims of this application and the above description of the drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings.

As shown in FIG. 1 , the system architecture 100 may include

terminal devices

101 , 102 , and 103 , a network 104 and a server 105 . The network 104 is a medium used to provide a communication link between the

terminal devices

101 , 102 , 103 and the server 105 . The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user can use the

terminal devices

101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages and the like. Various communication client applications may be installed on the

terminal devices

101 , 102 and 103 , such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, and the like.

The

terminal devices

101, 102, and 103 can be various electronic devices that have a display screen and support web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) Players, Laptops and Desktops, etc.

The server 105 may be a server that provides various services, such as a background server that provides support for the pages displayed on the

terminal devices

101 , 102 , and 103 .

It should be noted that the image quality evaluation method provided in the embodiments of the present application is generally performed by a server/terminal device, and accordingly, an image quality evaluation apparatus is generally set in the server/terminal device.

It should be understood that the numbers of terminal devices, networks and servers in FIG. 1 are merely illustrative. There can be any number of terminal devices, networks and servers according to implementation needs.

Continuing to refer to FIG. 2 , a flowchart of an embodiment of the method for image quality evaluation according to the present application is shown. The described image quality evaluation method includes the following steps:

S201: Build an image generation network, train the image generation network through a training sample set in a preset database, and obtain an image feature extractor.

Among them, an image generation network can be constructed based on a deep convolutional neural network model. Convolutional Neural Networks (CNN) is a kind of feedforward neural network (Feedforward Neural Networks) that includes convolution calculation and has a deep structure. One of the representative algorithms of deep learning. Convolutional neural network has the ability of representation learning and can perform shift-invariant classification of input information according to its hierarchical structure, so it is also called "shift-invariant artificial neural network". Convolutional neural network is constructed by imitating the visual perception mechanism of biology, which can perform supervised learning and unsupervised learning. Small computational effort to learn grid-like topology features, such as pixels and audio, with stable results and no additional feature engineering requirements on the data.

Among them, the image generation network includes an encoding layer encoder and a decoding layer decoder. The encoding layer encoder includes several convolution kernels, and the decoding layer decoder includes several deconvolution kernels. The convolution kernels correspond to the deconvolution kernels one by one. The encoding layer encoder A connected channel is established between the convolution kernel of the corresponding decoding layer decoder and the deconvolution kernel of the corresponding decoding layer decoder. After the convolution kernel of the encoding layer encoder extracts the image features, the extracted image features can be directly transmitted through the connected channel to the inverse of the corresponding decoding layer decoder. on the convolution kernel. The encoding layer encoder is a full convolution layer, and the encoding layer encoder is used to extract the image features of the input image, and the image feature extractor is also composed of this part. The decoding layer decoder is a deconvolution layer. The decoding layer decoder is used to decode the extracted image features and restore the image features to the input image. The purpose of the decoding layer decoder to restore the image features is to complete the verification of the encoding layer encoder. It should be noted that when constructing an image generation network, the loss functions L1 and L2 are set for the encoding layer encoder and the decoding layer decoder respectively. When the image generation network is iteratively updated, the image generation can be based on the L1 loss function and the L2 loss function. Iterative update of the network.

Specifically, an image generation network is constructed based on a deep convolutional neural network model, a training sample set is obtained from a preset database, the image generation network is trained through the training sample set, and after the trained image generation network is obtained, the image generation network is The convolution kernels in the encoder layer of the network build image feature extractors.

S202: Receive the image to be evaluated, and use an image feature extractor to extract image features of the image to be evaluated.

Specifically, when an image evaluation requirement is generated, an image evaluation instruction is received, an image to be evaluated is acquired based on the image evaluation instruction, and the image feature of the image to be evaluated is extracted by using the image feature extractor constructed above. It should be noted that the image feature extractor is constructed by the encoder part of the compressed coding layer network, and the image feature extractor will output multi-scale image features during feature extraction, and the image features of the previous layer are the same as the next layer. image input to the layer. In a specific embodiment of the present application, a total of 5 layers of feature extraction convolution layers are constructed. When the input image is 512×512 in size, the image features extracted from these 5 layers are scale feature 0, scale feature 1, and scale feature 2 respectively. , scale feature 3 and scale feature 4, scale feature 0, scale feature 1, scale feature 2, scale feature 3 and scale feature 4 are 512x512, 256x256, 128x128, 64x64, 32x32 respectively.

In this embodiment, the electronic device (for example, the server/terminal device shown in FIG. 1 ) on which the image quality evaluation method runs may receive the image evaluation instruction through a wired connection or a wireless connection. It should be pointed out that the above wireless connection methods may include but are not limited to 3G/4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future .

S203, transform the image features of the image to be evaluated into feature vectors, and convert the image features into feature vectors.

Specifically, after using the image feature extractor to extract the image features of the image to be evaluated, by using the image feature extractor to extract the multiple image features of the image to be evaluated for spatial pyramid pooling (Spatial Pyramid Pooling, SPP), the to-be-evaluated The multiple image features of the image are converted into feature vectors, and the feature vectors are vectors of the same size. Converting multiple image features of the image to be evaluated into feature vectors is beneficial to use the network regression function to calculate the regression value of the image features in the subsequent steps. . Among them, spatial pyramid pooling can convert feature maps of any size into fixed-size feature vectors, and send the fixed-size feature vectors to the fully connected layer.

S204, construct a network regression function, use the network regression function to calculate the regression value of the feature vector, and determine the quality of the image to be evaluated according to the regression value of the feature vector.

In a specific embodiment of the present application, the image evaluation task is divided into image feature extraction and multiple regression evaluation processes. The regression value of the eigenvector is calculated by the multivariate network regression function, the regression value of the eigenvector is calculated based on the construction of the network regression function, and the regression value of the eigenvector is normalized so that the regression value of the eigenvector falls within the range of 0-1. , the regression value can be regarded as a comprehensive score of multiple dimensions of image features, and finally the quality of the image to be evaluated is determined according to the regression value of the feature tensor. Here, the Bayesian kernel regression method is mainly used to construct a multi-dimensional image. The network regression function of the feature.

Specifically, after the image features are converted into feature vectors, a network regression function is constructed based on the Bayesian kernel regression equation, and the network regression function is used to calculate the regression value of the feature vector, and the regression value of the feature vector is normalized. The resulting eigenvector regression value determines the quality of the image to be evaluated. If the regression value is 1, the image quality is excellent, and if the regression value is 0, the image quality is unqualified.

Further, before the steps of constructing the image generation network, training the image generation network through the training sample set in the preset database, and acquiring the image feature extractor, it also includes:

Obtain the image data in the preset database, and preprocess the image data;

Label the preprocessed image data, and randomly combine the labelled image data to obtain a training sample set and a validation data set;

Store training sample sets and validation datasets in a preset database.

Specifically, the image data is acquired from a preset database, the image data is marked, and the quality index of the image data can be marked during marking. Randomly combine the labeled image data to obtain a training sample set and a verification data set. For example, the labeled image data can be randomly divided into 10 equal sample subsets, of which 9 sample subsets are randomly combined as the training sample set , the remaining sample subset is used as the validation data set, and the training sample set and the validation data set are stored in the preset database.

Further, please refer to FIG. 3, which shows a flowchart of a specific implementation of step S201 in FIG. 2, the image generation network includes an encoding layer and a decoding layer, the encoding layer includes several convolution kernels, and the decoding layer includes Several deconvolution kernels, the convolution kernels correspond to the deconvolution kernels one-to-one, construct an image generation network, train the image generation network through the training sample set in the preset database, and obtain the steps of image feature extractor, which specifically includes :

S301, extracting training samples in a training sample set, and sequentially importing each training sample into an encoding layer of an image generation network;

S302, using each training sample to train the coding layer in the image generation network to obtain several trained convolution kernels;

S303, screen several convolution kernels that have been trained based on the deep learning compression algorithm, and remove redundant items in the several convolution kernels;

S304, constructing an image feature extractor by using several convolution kernels after removing redundant items.

Among them, the deep compression algorithm trains the neural network, obtains the weight of each convolutional layer of the trained neural network, sets the weight threshold, and then deletes the convolutional layers below the weight threshold, and then iterates For training, redundant layers are removed again and again through iterative training. Finally, the weights of the convolutional layers retained in the neural network are clustered and the weights are shared, and the value of the cluster center point is used as the value of the ownership value. The model compression effect, and finally the weights are Huffman encoded. This proposal adopts the Deep Compression method to compress the neural network without losing precision, in which the size of the neural network can be compressed to 35 times to 49 times the original size, and the application of storage is more efficient during inference.

Specifically, the training samples in the training sample set are extracted, and each training sample is sequentially imported into the encoding layer encoder of the image generation network. The encoding layer encoder is preset with a number of convolution kernels, and each training sample is used for the image generation network. The convolution kernel of the encoding layer encoder is trained, and several trained convolution kernels are obtained, the weight threshold is set, and several convolution kernels after training are screened based on the deep learning compression algorithm. The convolution layer is removed to remove redundant items in several convolution kernels, and an image feature extractor is constructed by using several convolution kernels after removing redundant items.

Further, after using each training sample to train the coding layer in the convolutional layer of the image generation network to obtain several trained convolution kernels, the method further includes:

Collect the training results of each convolution kernel in the coding layer;

The training result of each convolution kernel is imported into the corresponding deconvolution kernel, and the corresponding deconvolution kernel is trained through the training result of each convolution kernel, and several trained deconvolution kernels are obtained.

Specifically, the training results of each convolution kernel in the encoding layer encoder are collected, the training results of each convolution kernel in the encoding layer encoder are marked, and the training results of each convolution kernel in the marked encoding layer encoder are used for training. Corresponding to the deconvolution kernel in the decoder of the decoding layer, several trained deconvolution kernels are obtained.

Further, import the training result of each convolution kernel into the corresponding deconvolution kernel, train the corresponding deconvolution kernel through the training result of each convolution kernel, and obtain several deconvolution kernels that have been trained. After the steps, also include:

Extract the validation samples in the validation dataset, and import the validation dataset into the image generation network;

Use several trained convolution kernels to perform feature extraction on the verification samples respectively, and obtain the feature extraction results of several verification samples;

The feature extraction results of several verification samples are respectively imported into the corresponding deconvolution kernels for feature restoration, and the feature restoration results are obtained;

Based on the feature restoration results and verification samples, use the back-propagation algorithm for fitting to obtain the prediction error;

The prediction error is compared with the preset threshold, and if the prediction error is greater than the preset threshold, the image generation network is iteratively updated until the prediction error is less than or equal to the preset threshold, and the image generation network is acquired.

Among them, the backpropagation algorithm, that is, the error backpropagation algorithm (Backpropagation algorithm, BP algorithm) is a learning algorithm suitable for multi-layer neuron networks. It is based on the gradient descent method and is used for the error of deep learning networks. calculate. The input and output relationship of BP network is essentially a mapping relationship: the function completed by a BP neural network with n input and m output is a continuous mapping from n-dimensional Euclidean space to a finite field in m-dimensional Euclidean space. A map is highly nonlinear. The learning process of BP algorithm consists of forward propagation process and back propagation process. In the process of forward propagation, the input information is processed layer by layer through the hidden layer through the input layer and transmitted to the output layer, and then transferred to the back propagation, and the partial derivative of the objective function to the weight of each neuron is obtained layer by layer, which constitutes The gradient of the objective function to the weight vector is used as the basis for modifying the weight.

Specifically, the verification samples in the verification data set are extracted, the verification data set is imported into the image generation network, and several trained convolution kernels are used to extract the features of the verification samples respectively, and the corresponding deconvolution kernels are used to restore the features. Then the backpropagation algorithm calculates the prediction error and compares the prediction error with the preset error threshold. If the prediction error is greater than the preset error threshold, the image generation network is iterated based on the loss functions L1 and L2 of the encoder layer encoder and decoder layer decoder. Update until the prediction error is less than or equal to the preset error threshold, and obtain the image generation network that has passed the verification.

Further, the image features of the image to be evaluated are transformed into feature vectors, and the steps of transforming the image features into feature vectors include:

Based on spatial pyramid pooling, the image features of the image to be evaluated are transformed into feature vectors, and the image features are transformed into feature vectors.

Specifically, after using the image feature extractor to extract the image features of the image to be evaluated, multiple image features of the image to be evaluated are extracted by using the image feature extractor to perform spatial pyramid pooling to convert the multiple image features of the image to be evaluated. is a feature vector, and the feature vector is a vector with the same size. Converting multiple image features of the image to be evaluated into a feature vector is beneficial to use the network regression function to calculate the regression value of the image feature in the subsequent steps. Among them, spatial pyramid pooling can convert feature maps of any size into fixed-size feature vectors, and send the fixed-size feature vectors to the fully connected layer.

In the specific embodiment of the present application, the image features are converted into feature vectors in the form of full links through spatial pyramid pooling. The spatial pyramid pooling here refers to performing deformation and convolution operations on the above image features of different scales, and finally obtains feature vectors in the form of full links.

Further, please refer to Fig. 4, Fig. 4 shows a flow chart of a specific implementation of step S204 in Fig. 2, constructing a network regression function, using the network regression function to calculate the regression value of the eigenvector, and according to the regression of the eigenvector. value to determine the quality of the image to be evaluated, including:

S401, constructing an initial regression function based on a Bayesian algorithm;

S402, extract the parameters of the image feature extractor, and calculate the feature weights based on the parameters of the image feature extractor;

S403, import the feature weights into the initial regression function to obtain the network regression function;

S404, import the feature vector into the network regression function, calculate the regression value of the feature vector, and determine the quality of the image to be evaluated according to the regression value of the feature vector.

Specifically, the initial regression function is constructed based on the Bayesian equation. The Bayesian equation is as follows:

Among them, Y here refers to the Bayesian regression value, i refers to the serial number of the input image, h refers to the high-dimensional response function, z refers to the image feature, x refers to the potential factor, β refers to the weight, ε is the modulation factor. Solving the response function h can be derived based on the method of the kernel function, so h can be written in the following form:

Among them, α is the former coefficient of the kernel function, where the kernel function uses a Gaussian kernel function, so here K(z, z') can be rewritten as:

Among them, exp is the e index, and M is the training set capacity, that is, the number of samples. The above K is further rewritten:

Among them, r here is to meet the following conditions:

r _m |δ _m ~δ _m f ₁ (r _m )+(1-δ _m )P ₀

Among them, m=1,...,M; rm is the probability value of the conditional probability of Bayes' theorem, f is the probability density function, δ _m ~ bernouli (π), bernouli refers to the composite Bernoulli distribution, δ is the variance, At this point, the regression process can be transformed into a regression based on a Bayesian Gaussian kernel.

Specifically, after the image features are converted into feature vectors, an initial regression function is constructed based on the Bayesian algorithm, the parameters of the image feature extractor are extracted, and the feature weights are calculated based on the parameters of the image feature extractor, and the feature weights are normalized. To normalize, import the normalized feature weights into the initial regression function to obtain the network regression function, import the feature vector into the network regression function, calculate the regression value of the feature vector, and determine the quality of the image to be evaluated according to the regression value of the feature vector .

It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned images to be evaluated, the above-mentioned images to be evaluated can also be stored in a node of a blockchain.

The blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. , when the computer-readable instructions are executed, the processes of the above-mentioned method embodiments may be included. Wherein, the aforementioned storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

It should be understood that although the various steps in the flowchart of the accompanying drawings are sequentially shown in the order indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order and may be performed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution sequence is also It does not have to be performed sequentially, but may be performed alternately or alternately with other steps or at least a portion of sub-steps or stages of other steps.

With further reference to FIG. 5 , as an implementation of the method shown in FIG. 2 above, the present application provides an embodiment of an apparatus for evaluating image quality. The apparatus embodiment corresponds to the method embodiment shown in FIG. 2 . Specifically, it can be applied to various electronic devices.

As shown in FIG. 5 , the apparatus for image quality evaluation described in this embodiment includes:

The building module 501 is used to build an image generation network, and train the image generation network through a training sample set in a preset database to obtain an image feature extractor;

Extraction module 502, for receiving the image to be evaluated, and extracting image features of the image to be evaluated by using an image feature extractor;

The conversion module 503 is used to convert the image features of the image to be evaluated into feature vectors, and convert the image features into feature vectors;

The evaluation module 504 is configured to construct a network regression function, use the network regression function to calculate the regression value of the feature vector, and determine the quality of the image to be evaluated according to the regression value of the feature vector.

Further, the device for image quality evaluation also includes:

The preprocessing module is used to obtain the image data in the preset database and preprocess the image data;

The labeling module is used to label the preprocessed image data, and randomly combine the labelled image data to obtain a training sample set and a verification data set;

The storage module is used to store the training sample set and the validation data set in the preset database.

Further, the image generation network includes an encoding layer and a decoding layer, the encoding layer includes several convolution kernels, and the decoding layer includes several deconvolution kernels, and the convolution kernels correspond to the deconvolution kernels one-to-one. The building module 501 specifically includes:

The extraction unit is used to extract the training samples in the training sample set, and sequentially import each training sample into the coding layer of the image generation network;

The first training unit is used to train the coding layer in the image generation network by using each training sample to obtain several trained convolution kernels;

The compression unit is used to screen several convolution kernels after training based on the deep learning compression algorithm, and remove redundant items in several convolution kernels;

The construction unit is used to construct an image feature extractor using several convolution kernels after removing redundant items.

Further, the device for image quality evaluation also includes:

The acquisition unit is used to collect the training results of each convolution kernel in the coding layer;

The second training unit is used to import the training result of each convolution kernel into the corresponding deconvolution kernel, train the corresponding deconvolution kernel through the training result of each convolution kernel, and obtain several training completed inverse convolution kernels. convolution kernel.

Further, the device for image quality evaluation also includes:

The verification unit is used to extract the verification samples in the verification data set, and import the verification data set into the image generation network;

The convolution unit is used to extract the features of the verification samples by using several trained convolution kernels to obtain the feature extraction results of several verification samples;

The restoration unit is used to import the feature extraction results of several verification samples into the corresponding deconvolution kernels for feature restoration, and obtain the feature restoration results;

The fitting unit is used to restore the result and the verification sample based on the feature, and use the back-propagation algorithm to perform fitting to obtain the prediction error;

The iterative unit is configured to compare the prediction error with a preset threshold, and if the prediction error is greater than the preset threshold, iteratively update the image generation network until the prediction error is less than or equal to the preset threshold, and acquire the image generation network.

Further, the conversion module specifically includes:

The transformation unit is used to transform the image features of the image to be evaluated based on the spatial pyramid pooling, and convert the image features into feature vectors.

Further, the evaluation module 504 specifically includes:

The function construction unit is used to construct the initial regression function based on the Bayesian algorithm;

a parameter extraction unit for extracting parameters of the image feature extractor, and calculating feature weights based on the parameters of the image feature extractor;

The import unit is used to import the feature weights into the initial regression function to obtain the network regression function;

The evaluation unit is used to import the feature vector into the network regression function, calculate the regression value of the feature vector, and determine the quality of the image to be evaluated according to the regression value of the feature vector.

The application discloses an image quality evaluation device, belonging to the technical field of artificial intelligence. The application constructs an image generation network, trains the image generation network through a training sample set in a preset database, and obtains an image feature extractor; Evaluate the image, and use the image feature extractor to extract the image features of the image to be evaluated; perform feature vector transformation on the image features of the image to be evaluated, and convert the image features into feature vectors; build a network regression function, and use the network regression function to calculate the regression of the feature vector value, and determine the quality of the image to be evaluated according to the regression value of the feature vector. The present application constructs an image quality evaluation system by simplifying the deep learning network and adopting machine regression. When performing image quality evaluation, the image features are first obtained through the trained deep learning network, and then the image features are calculated based on the network regression function. Regression value, and finally determine the quality of the image to be evaluated based on the regression value of the image features. The image quality evaluation system constructed in this application has a simple structure, does not occupy too many server resources, effectively reduces computing resource consumption, and can meet the deployment and use requirements of mobile terminals . At the same time, the image quality is finally evaluated by the network regression function, and a mathematical explanation can be made for the evaluation result, which is convenient for users to analyze the problem intuitively.

To solve the above technical problems, the embodiments of the present application also provide computer equipment. Please refer to FIG. 6 for details. FIG. 6 is a block diagram of the basic structure of a computer device according to this embodiment.

The computer device 6 includes a memory 61 , a processor 62 , and a network interface 63 that communicate with each other through a system bus. It should be pointed out that only the computer device 6 with components 61-63 is shown in the figure, but it should be understood that it is not required to implement all of the shown components, and more or less components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, special-purpose Integrated circuit (Application Specific Integrated Circuit, ASIC), programmable gate array (Field-Programmable Gate Array, FPGA), digital processor (Digital Signal Processor, DSP), embedded devices, etc.

The computer equipment may be a desktop computer, a notebook computer, a palmtop computer, a cloud server and other computing equipment. The computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touch pad or a voice control device.

The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disk, etc. In some embodiments, the memory 61 may be an internal storage unit of the computer device 6 , such as a hard disk or a memory of the computer device 6 . In other embodiments, the memory 61 may also be an external storage device of the computer device 6, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Of course, the memory 61 may also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6 , such as computer-readable instructions of a method for evaluating image quality. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or will be output.

In some embodiments, the processor 62 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. This processor 62 is typically used to control the overall operation of the computer device 6 . In this embodiment, the processor 62 is configured to execute computer-readable instructions stored in the memory 61 or process data, for example, computer-readable instructions for executing the image quality evaluation method.

The network interface 63 may include a wireless network interface or a wired network interface, and the network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

The application discloses a computer device, which belongs to the technical field of artificial intelligence. The application constructs an image generation network, trains the image generation network through a training sample set in a preset database, and obtains an image feature extractor; receives an image to be evaluated, And use the image feature extractor to extract the image features of the image to be evaluated; transform the image features of the image to be evaluated into feature vectors, and convert the image features into feature vectors; build a network regression function, use the network regression function to calculate the regression value of the feature vector, and The quality of the image to be evaluated is determined according to the regression value of the feature vector. The present application constructs an image quality evaluation system by simplifying the deep learning network and adopting machine regression. When performing image quality evaluation, the image features are first obtained through the trained deep learning network, and then the image features are calculated based on the network regression function. Regression value, and finally determine the quality of the image to be evaluated based on the regression value of the image features. The image quality evaluation system constructed in this application has a simple structure, does not occupy too many server resources, effectively reduces computing resource consumption, and can meet the deployment and use requirements of mobile terminals . At the same time, the image quality is finally evaluated by the network regression function, and a mathematical explanation can be made for the evaluation result, which is convenient for users to analyze the problem intuitively.

The present application also provides another implementation manner, that is, to provide a computer-readable storage medium, the computer-readable storage medium may be non-volatile or volatile, and the computer-readable storage medium stores Computer readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the method for image quality assessment as described above.

The application discloses a storage medium, which belongs to the technical field of artificial intelligence. The application constructs an image generation network, trains the image generation network through a training sample set in a preset database, and obtains an image feature extractor; receives an image to be evaluated, And use the image feature extractor to extract the image features of the image to be evaluated; transform the image features of the image to be evaluated into feature vectors, and convert the image features into feature vectors; build a network regression function, use the network regression function to calculate the regression value of the feature vector, and The quality of the image to be evaluated is determined according to the regression value of the feature vector. The present application constructs an image quality evaluation system by simplifying the deep learning network and adopting machine regression. When performing image quality evaluation, the image features are first obtained through the trained deep learning network, and then the image features are calculated based on the network regression function. Regression value, and finally determine the quality of the image to be evaluated based on the regression value of the image features. The image quality evaluation system constructed in this application has a simple structure, does not occupy too many server resources, effectively reduces computing resource consumption, and can meet the deployment and use requirements of mobile terminals . At the same time, the image quality is finally evaluated by the network regression function, and a mathematical explanation can be made for the evaluation result, which is convenient for users to analyze the problem intuitively.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.

Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the scope of the patent of the present application. This application may be embodied in many different forms, rather these embodiments are provided so that a thorough and complete understanding of the disclosure of this application is provided. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements for some of the technical features. . Any equivalent structure made by using the contents of the description and drawings of the present application, which is directly or indirectly used in other related technical fields, is also within the scope of protection of the patent of the present application.

Claims

A method of image quality evaluation, comprising:

constructing an image generation network, and training the image generation network through a training sample set in a preset database to obtain an image feature extractor;

receiving an image to be evaluated, and extracting image features of the image to be evaluated using the image feature extractor;

The image features of the image to be evaluated are transformed into feature vectors, and the image features are transformed into feature vectors;

A network regression function is constructed, the regression value of the feature vector is calculated by using the network regression function, and the quality of the image to be evaluated is determined according to the regression value of the feature vector.
The method for image quality evaluation according to claim 1, wherein, before the step of constructing an image generation network, training the image generation network through a training sample set in a preset database, and acquiring an image feature extractor, Also includes:

acquiring image data in the preset database, and preprocessing the image data;

Labeling the preprocessed image data, and randomly combining the labeled image data to obtain a training sample set and a verification data set;

The training sample set and the verification data set are stored in the preset database.
The method for image quality evaluation according to claim 2, wherein the image generation network includes an encoding layer and a decoding layer, the encoding layer includes several convolution kernels, and the decoding layer includes several deconvolution kernels, The convolution kernels correspond to the deconvolution kernels one-to-one, the image generation network is constructed, the image generation network is trained through the training sample set in the preset database, and the steps of acquiring the image feature extractor are as follows: include:

extracting training samples in the training sample set, and sequentially importing each of the training samples into the coding layer of the image generation network;

Use each of the training samples to train the coding layer in the image generation network to obtain several trained convolution kernels;

Screening several of the convolution kernels that have been trained based on the deep learning compression algorithm, to remove redundant items in several of the convolution kernels;

The image feature extractor is constructed by using several of the convolution kernels after removing the redundant items.
The method for image quality evaluation according to claim 3, wherein, in the coding layer in the convolutional layer of the image generation network is trained by using each of the training samples to obtain several convolutional convolutions that have been trained. After the nuclear step, it also includes:

Collect the training result of each convolution kernel in the coding layer;

Import the training results of each of the convolution kernels into the corresponding deconvolution kernels, train the corresponding deconvolution kernels through the training results of each of the convolution kernels, and obtain several trained deconvolution kernels .
The method for image quality evaluation according to claim 4, wherein, in the process of importing the training result of each of the convolution kernels into the corresponding deconvolution kernel, training is performed by the training results of each of the convolution kernels. The corresponding deconvolution kernel, after obtaining several steps of training completed deconvolution kernels, also includes:

extracting verification samples in the verification data set, and importing the verification data set into the image generation network;

Using several trained convolution kernels to perform feature extraction on the verification samples, respectively, to obtain the feature extraction results of several verification samples;

The feature extraction results of several of the verification samples are respectively imported into the corresponding deconvolution kernels for feature restoration, and feature restoration results are obtained;

Based on the feature restoration result and the verification sample, the back-propagation algorithm is used for fitting to obtain the prediction error;

Compare the prediction error with a preset threshold, and if the prediction error is greater than a preset threshold, iteratively update the image generation network until the prediction error is less than or equal to a preset threshold, and acquire the image Generate a network.
The method for evaluating image quality according to any one of claims 1 to 5, wherein the step of converting the image features of the image to be evaluated into feature vectors, and converting the image features into feature vectors, specifically includes the following steps: :

The image features of the image to be evaluated are transformed into feature vectors based on spatial pyramid pooling, and the image features are transformed into feature vectors.
The method for evaluating image quality according to claim 6, wherein the network regression function is constructed, a regression value of the feature vector is calculated by using the network regression function, and the to-be-to-be-regression value is determined according to the regression value of the feature vector. The steps to assess the quality of the image include:

Construct the initial regression function based on the Bayesian algorithm;

extracting parameters of the image feature extractor, and calculating feature weights based on the parameters of the image feature extractor;

Importing the feature weights into the initial regression function to obtain a network regression function;

The feature vector is imported into the network regression function, the regression value of the feature vector is calculated, and the quality of the image to be evaluated is determined according to the regression value of the feature vector.
A device for evaluating image quality, comprising:

a building module for constructing an image generation network, training the image generation network through a training sample set in a preset database, and obtaining an image feature extractor;

an extraction module for receiving an image to be evaluated, and extracting image features of the image to be evaluated by using the image feature extractor;

a conversion module, for converting the image features of the image to be evaluated into feature vectors, and converting the image features into feature vectors;

An evaluation module, configured to construct a network regression function, use the network regression function to calculate the regression value of the feature vector, and determine the quality of the image to be evaluated according to the regression value of the feature vector.
A computer device, comprising a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the processor executes the computer-readable instructions, the method for evaluating image quality as described below is implemented:

constructing an image generation network, and training the image generation network through a training sample set in a preset database to obtain an image feature extractor;

receiving an image to be evaluated, and extracting image features of the image to be evaluated using the image feature extractor;

The image features of the image to be evaluated are transformed into feature vectors, and the image features are transformed into feature vectors;

A network regression function is constructed, the regression value of the feature vector is calculated by using the network regression function, and the quality of the image to be evaluated is determined according to the regression value of the feature vector.
The computer device according to claim 9, wherein, before the step of constructing an image generation network, training the image generation network through a training sample set in a preset database, and acquiring an image feature extractor, the method further comprises:

acquiring image data in the preset database, and preprocessing the image data;

Labeling the preprocessed image data, and randomly combining the labeled image data to obtain a training sample set and a verification data set;

The training sample set and the verification data set are stored in the preset database.
The computer device of claim 10, wherein the image generation network includes an encoding layer and a decoding layer, the encoding layer includes a number of convolution kernels, the decoding layer includes a number of deconvolution kernels, and the volume The accumulation kernel corresponds to the deconvolution kernel one-to-one, and the image generation network is constructed, and the image generation network is trained through the training sample set in the preset database, and the steps of acquiring the image feature extractor include:

extracting training samples in the training sample set, and sequentially importing each of the training samples into the coding layer of the image generation network;

Use each of the training samples to train the coding layer in the image generation network to obtain several trained convolution kernels;

Screening several of the convolution kernels that have been trained based on the deep learning compression algorithm, to remove redundant items in several of the convolution kernels;

The image feature extractor is constructed by using several of the convolution kernels after removing the redundant items.
The computer device according to claim 11, wherein, in the step of using each of the training samples to train the coding layer in the convolutional layer of the image generation network to obtain several trained convolution kernels After that, also include:

Collect the training result of each convolution kernel in the coding layer;

Import the training results of each of the convolution kernels into the corresponding deconvolution kernels, train the corresponding deconvolution kernels through the training results of each of the convolution kernels, and obtain several trained deconvolution kernels .
The computer device according to claim 12, wherein, in the process of importing the training result of each of the convolution kernels into the corresponding deconvolution kernel, the corresponding deconvolution kernel is trained by the training result of each of the convolution kernels. The convolution kernel, after the steps of obtaining several deconvolution kernels that have been trained, also include:

extracting verification samples in the verification data set, and importing the verification data set into the image generation network;

Using several trained convolution kernels to perform feature extraction on the verification samples, respectively, to obtain the feature extraction results of several verification samples;

The feature extraction results of several of the verification samples are respectively imported into the corresponding deconvolution kernels for feature restoration, and feature restoration results are obtained;

Based on the feature restoration result and the verification sample, the back-propagation algorithm is used for fitting to obtain the prediction error;

Compare the prediction error with a preset threshold, and if the prediction error is greater than a preset threshold, iteratively update the image generation network until the prediction error is less than or equal to a preset threshold, and acquire the image Generate a network.
The computer device according to any one of claims 9 to 13, wherein the step of performing feature vector transformation on the image features of the image to be evaluated, and converting the image features into feature vectors, specifically includes:

The image features of the image to be evaluated are transformed into feature vectors based on spatial pyramid pooling, and the image features are transformed into feature vectors.
A computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the method for evaluating image quality as described below is implemented:

constructing an image generation network, and training the image generation network through a training sample set in a preset database to obtain an image feature extractor;

receiving an image to be evaluated, and extracting image features of the image to be evaluated using the image feature extractor;

The image features of the image to be evaluated are transformed into feature vectors, and the image features are transformed into feature vectors;

A network regression function is constructed, the regression value of the feature vector is calculated by using the network regression function, and the quality of the image to be evaluated is determined according to the regression value of the feature vector.
The computer-readable storage medium according to claim 15, wherein, before the step of constructing an image generation network, training the image generation network through a training sample set in a preset database, and acquiring an image feature extractor, Also includes:

acquiring image data in the preset database, and preprocessing the image data;

Labeling the preprocessed image data, and randomly combining the labeled image data to obtain a training sample set and a verification data set;

The training sample set and the verification data set are stored in the preset database.
The computer-readable storage medium of claim 16, wherein the image generation network includes an encoding layer and a decoding layer, the encoding layer includes several convolution kernels, the decoding layer includes several deconvolution kernels, The convolution kernels correspond to the deconvolution kernels one-to-one, the image generation network is constructed, the image generation network is trained through the training sample set in the preset database, and the steps of acquiring the image feature extractor are as follows: include:

extracting training samples in the training sample set, and sequentially importing each of the training samples into the coding layer of the image generation network;

Use each of the training samples to train the coding layer in the image generation network to obtain several trained convolution kernels;

Screening several of the convolution kernels that have been trained based on the deep learning compression algorithm, to remove redundant items in several of the convolution kernels;

The image feature extractor is constructed by using several of the convolution kernels after removing the redundant items.
The computer-readable storage medium according to claim 17, wherein, in the training of the coding layer in the convolutional layer of the image generation network by using each of the training samples, a plurality of trained convolutional layers are obtained. After the nuclear step, it also includes:

Collect the training result of each convolution kernel in the coding layer;

Import the training results of each of the convolution kernels into the corresponding deconvolution kernels, train the corresponding deconvolution kernels through the training results of each of the convolution kernels, and obtain several trained deconvolution kernels .
The computer-readable storage medium according to claim 18, wherein, in the process of importing the training result of each of the convolution kernels into the corresponding deconvolution kernel, training is performed through the training results of each of the convolution kernels. The corresponding deconvolution kernel, after obtaining several steps of training completed deconvolution kernels, also includes:

extracting verification samples in the verification data set, and importing the verification data set into the image generation network;

Using several trained convolution kernels to perform feature extraction on the verification samples, respectively, to obtain the feature extraction results of several verification samples;

The feature extraction results of several of the verification samples are respectively imported into the corresponding deconvolution kernels for feature restoration, and feature restoration results are obtained;

Based on the feature restoration result and the verification sample, the back-propagation algorithm is used for fitting to obtain the prediction error;

Compare the prediction error with a preset threshold, and if the prediction error is greater than a preset threshold, iteratively update the image generation network until the prediction error is less than or equal to a preset threshold, and acquire the image Generate a network.
The computer-readable storage medium according to any one of claims 15 to 19, wherein the step of transforming the image features of the to-be-evaluated image into feature vectors, and converting the image features into feature vectors, specifically includes :

The image features of the image to be evaluated are transformed into feature vectors based on spatial pyramid pooling, and the image features are transformed into feature vectors.