CN111161238A

CN111161238A - Image quality evaluation method and device, electronic device, and storage medium

Info

Publication number: CN111161238A
Application number: CN201911379470.2A
Authority: CN
Inventors: 彭冬炜
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2019-12-27
Filing date: 2019-12-27
Publication date: 2020-05-15

Abstract

The present disclosure provides an image quality evaluation method, an apparatus, an electronic device and a computer-readable storage medium, which relate to the technical field of image processing, and the image quality evaluation method includes: acquiring an image to be processed; if the judgment operation for representing the authenticity is completed according to the indexes of the sample image, performing prediction processing on the image to be processed to generate a prediction result for representing the grading distribution; and performing aesthetic quality evaluation on the image to be processed according to the prediction result to determine an evaluation result of the image to be processed. The method and the device can improve the accuracy of image quality evaluation.

Description

Image quality evaluation method and device, electronic device, and storage medium

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to an image quality evaluation method, an image quality evaluation device, an electronic device, and a computer-readable storage medium.

Background

With the development of image technology, the quality of images can be scored so as to facilitate the beautification of the images or the subsequent processing.

When the image aesthetic evaluation is carried out in the related technology, the evaluation process is complicated, the robustness is poor, the image aesthetic evaluation cannot be accurately carried out, and the reliability is poor.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

An object of the present disclosure is to provide an image quality evaluation method and apparatus, an electronic device, and a computer-readable storage medium, thereby overcoming, at least to some extent, the problem of inaccurate image quality evaluation due to limitations and disadvantages of the related art.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

According to an aspect of the present disclosure, there is provided an image quality evaluation method including: acquiring an image to be processed; if the judgment operation for representing the authenticity is completed according to the indexes of the sample image, performing prediction processing on the image to be processed to generate a prediction result for representing the grade distribution of the image to be processed; and performing aesthetic quality evaluation on the image to be processed according to the prediction result to determine an evaluation result of the image to be processed.

According to an aspect of the present disclosure, there is provided an image quality evaluation apparatus including: the image acquisition module is used for acquiring an image to be processed; the prediction result determining module is used for performing prediction processing on the image to be processed to generate a prediction result of grade distribution for representing the image to be processed if the judging operation for representing the authenticity is completed according to the index aiming at the sample image; and the image evaluation module is used for performing aesthetic quality evaluation on the image to be processed according to the prediction result so as to determine the evaluation result of the image to be processed.

According to an aspect of the present disclosure, there is provided an electronic device including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any of the image quality assessment methods described above via execution of the executable instructions.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the image quality evaluation method of any one of the above.

In the technical solution provided in the present exemplary embodiment, on one hand, after the determination operation is completed, a generation operation may be performed on the image to be processed, so as to perform a prediction process on the image to be processed according to a generator for performing the generation operation, so as to determine a score distribution of the image to be processed, so as to perform aesthetic quality evaluation on the image to be processed, since an evaluation result may be determined by a high-dimensional score distribution, and the score distribution may be accurately fitted based on the generation operation, a limitation caused when evaluation is performed by a deterministic value is avoided, and evaluation is performed after the determination operation is completed, so as to improve accuracy and reliability of the aesthetic quality evaluation of the image. On the other hand, since the score distribution can be generated after the discrimination operation is completed, so as to perform aesthetic quality evaluation on the image to be processed through the score distribution, compared with the related art, the method simplifies the operation process of image quality evaluation, increases the applicability, simplifies the algorithm design, increases the robustness, and improves the efficiency of image aesthetic quality evaluation.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty.

Fig. 1 schematically shows a schematic diagram of a system architecture for implementing the image quality evaluation method.

Fig. 2 schematically illustrates a schematic diagram of an image quality evaluation method in an exemplary embodiment of the present disclosure.

FIG. 3 schematically illustrates a flow chart for training a model in an exemplary embodiment of the disclosure.

FIG. 4 schematically illustrates a detailed flow chart for training a model according to an index in an exemplary embodiment of the disclosure.

Fig. 5 schematically shows an overall flow diagram of quality evaluation in an exemplary embodiment of the present disclosure.

Fig. 6 schematically shows a block diagram of an image quality evaluation apparatus in an exemplary embodiment of the present disclosure.

Fig. 7 schematically illustrates a schematic view of an electronic device in an exemplary embodiment of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

In the related art, the quality evaluation may be performed based on the probability distribution result. However, most of them use a dozing distance for evaluating the measure of the difference between two probability distributions to determine the distribution of data labeling information given a picture data set. The method for learning the distribution of the labeling information by the algorithms is finished in a batch supervision mode, so that the method has certain limitation and inaccurate results.

In the present exemplary embodiment, first, a system architecture diagram for performing an image quality evaluation method is provided. Referring to fig. 1, a system architecture 100 may include a first end 101, a network 102, and a second end 103. The first end 101 may be a client, and may be, for example, various handheld devices (smart phones) having a photographing function and an image display function, desktop computers, vehicle-mounted devices, wearable devices, and the like. The network 102 is used as a medium for providing a communication link between the first end 101 and the second end 103, the network 102 may include various connection types, such as a wired communication link, a wireless communication link, and the like, and in the embodiment of the present disclosure, the network 102 between the first end 101 and the second end 103 may be a wired communication link, such as a communication link provided by a serial connection line, or a wireless communication link, such as a communication link provided by a wireless network. The second terminal 103 may be a client, for example, a terminal device with a data processing function, such as a portable computer, a desktop computer, a smart phone, and the like, for performing feature extraction and scoring processing on an input image. When the first end and the second end are both clients, the first end and the second end may be the same client.

It should be understood that the number of first ends, networks and second ends in fig. 1 is merely illustrative. There may be any number of clients, networks, and servers, as desired for an implementation.

It should be noted that the image quality evaluation method provided by the embodiment of the present disclosure may be completely executed by the second end or the first end, or may be executed by the first end and the second end, where the execution subject of the image quality evaluation method is not particularly limited. Accordingly, the image quality evaluation device may be disposed in the second end 103 or in the first end 101.

Based on the system architecture, the embodiment of the present disclosure provides an image quality evaluation method, which may be applied to any scene for evaluating and evaluating the quality of a photo, a video, or a picture. Next, the image quality evaluation method in the present exemplary embodiment is explained in detail with reference to fig. 2. The detailed description is as follows:

in step S210, an image to be processed is acquired.

In the embodiment of the present disclosure, the image to be processed may be an image captured by using any camera, or an image downloaded from a network, or an image acquired from another storage device. The image to be processed may be a still image or an image in a moving state, and the like. One or more objects may be included in the image to be processed. The image to be processed may be a color image or a grayscale image, and the like, which is not limited herein.

Continuing to refer to fig. 2, in step S220, if the discrimination operation for characterizing the authenticity is completed according to the index for the sample image, the image to be processed is subjected to prediction processing to generate a prediction result for characterizing the score distribution.

In the embodiment of the present disclosure, the determination operation is used to determine whether the sample image is a real image, i.e., determine whether the sample image is true or false, and the determination operation may be implemented by a discriminator in the countervailing learning model. Furthermore, after the training of the discriminator is finished, the acquired image to be processed can be subjected to prediction processing according to the trained counterstudy model, and then a prediction result is obtained. The prediction process here is used to input the image to be processed to a generator of a trained antagonistic learning model to determine the score distribution of the image to be processed. The prediction result can be the output result of the model. The confrontational learning model may include a generator and a discriminator. Where the generator may act as a sample generator, inputting a noise/sample and then packing it into a realistic sample, i.e. the output. A generator: modeling is carried out on the joint probability, the distribution situation of the data is represented from the statistical angle, and how the data are generated is described, so that the convergence speed is high, such as naive Bayes, Gaussian Discriminant Analysis (GDA), hidden Markov models and the like. The probability distribution of the real data in the training set is continuously learned, and the aim is to convert the input random noise into images which can be falsified (the generated images are more similar to the images used for training, and the better).

The discriminator can be used as a two-classifier to judge whether the input sample is true or false, if the input sample is true, the network output is close to 1, the input sample is false, and the network output is close to 0, so that the discrimination purpose is achieved. In the embodiment of the disclosure, for the discriminator, the input is an image, and the output is a probability value. For the generator, the input is an image and the output is a prediction tag. It should be noted that both the generator and the arbiter can be neural network models, and both can include the same or different structures, for example, each can include an input layer, a convolutional layer, a pooling layer, a fully-connected layer, an output layer, and so on.

To improve the accuracy of the model, the confrontational learning model may be first trained to obtain a trained confrontational learning model, and then used to score the image to be processed. When model training is carried out, the confrontation learning model can be trained according to the sample image and the artificially labeled labeling information of the sample image, and the trained confrontation learning model is obtained. The annotation information refers to information describing the distribution of the sample image, and may be obtained manually or by prediction, and thus may include a true label and a predicted label of the sample image. Wherein, the real label is obtained by manual labeling, and the prediction label is automatically labeled according to the confrontation learning model.

A schematic flow chart of the process of training the antagonistic learning model is schematically shown in fig. 3, and referring to fig. 3, the process mainly includes the following steps:

in step S310, feature extraction is performed on the sample image by a generator in a counterstudy model to determine a prediction label of the sample image.

In the embodiment of the present disclosure, the number of the sample images may be multiple, so as to be used for accurately training the confrontation learning model. After the sample image is obtained, the sample image may be input to a generator in the confrontation learning model to extract image features of the sample image through the convolutional layer, the pooling layer, and the full-link layer of the confrontation learning model, wherein the image features are specifically represented by two-dimensional or multi-dimensional feature data.

After the image features of the sample image are extracted, the image features may be subjected to a prediction process by the generator to obtain a prediction tag of the sample image. The predictive label refers to a label automatically output by the generator to describe the distribution of image quality. The predictive label may specifically be represented by a score distribution rather than a deterministic value. The score distribution refers to an aesthetic probability distribution of a corresponding plurality of levels, which may be represented by, for example, 1-10 points, respectively. Specifically, the score distribution may be represented by a distribution of a histogram of each level, and the score distribution may be a multi-dimensional vector corresponding to a plurality of levels. For any sample image, the predicted score distribution can be obtained as a prediction label of the sample image.

In determining the prediction tag, a forward calculation may be employed by the generator to determine. The forward algorithm is used for calculating the influence of the nodes of the input layer on the nodes of the hidden layer, namely, the network is positively walked: the input layer-hidden layer-output layer calculates the influence of each node on the node of the next layer. Forward computation refers to taking the output of the first layer as the input of the second layer, according to the composition of the network in the generator, and so on until the result of the last layer is obtained.

In step S320, the confrontation learning model is trained according to the prediction label of the sample image and the true label of the sample image, so as to obtain a trained confrontation learning model for evaluating image quality.

In the embodiment of the disclosure, the real label refers to an actual label of the sample image determined manually, and the representation form of the real label corresponds to the predicted label, for example, the predicted label is score distribution, and the real label is also score distribution; the predictive label is the score and the true label is the score. In the embodiment of the present disclosure, the real label refers to a manually determined real score distribution, and the real score distribution may be the same as the determination rule of the distribution of the prediction label, but for the same sample image, the real label and the prediction label may be the same or different, and are not limited herein.

After the prediction label of the sample image and the real label of the sample image are obtained, the confrontation learning model can be trained based on the prediction label of the sample image and the real label of the sample image, so as to obtain a trained confrontation learning model. Specifically, since the confrontation learning model includes two partial models, i.e., a generator and a discriminator, the two included models need to be trained sequentially, so that a trained confrontation learning model is obtained.

Firstly, the discriminators in the confrontation learning model can be trained, and the generator is further trained after the discriminants are trained, so that the trained confrontation learning model is obtained.

Training may be performed according to the input indexes of the true label of the sample image and the predicted label of the sample image. The specific process of training the model may include: determining indexes of a real label of the sample image and a prediction label of the sample image through a discriminator in the counterstudy model; and training the confrontation learning model according to the indexes. The index may be an output of the discriminator, and specifically may be probabilities that the input real tag and the input predicted tag belong to real data respectively. The indicator may be represented by a threshold range of 0, 1. If the input labeling information (prediction label or real label) belongs to real data, the index is 1; if the input marking information does not belong to the real data, the index is 0. The purpose of the discriminator is to make the probability that the prediction labels of all sample images belong to the true data tend to 0, and make the probability that the true labels belong to the true data tend to 1. Specifically, the real label and the predicted label may be input to the discriminator to calculate the probability that the real label and the predicted label belong to the real data through the convolutional layer, the fully-connected layer, and the like in the neural network model corresponding to the discriminator. Further, the discriminators and generators in the antagonistic learning model may be trained according to probabilities.

In the embodiment of the present disclosure, the process of training the arbiter may be: by inputting the real label and the predicted label of the sample image into the discriminator, the difference between the maximized predicted label and the real label can be used as a training target, the discriminator of the confrontation learning model is trained, the efficiency and the accuracy of the training of the confrontation learning model can be improved, and the stability of the model is improved.

Fig. 4 schematically shows a specific flowchart of a model training process according to the index, and referring to fig. 4, the method mainly includes the following steps:

in step S410, the difference between the indices is subjected to a minimization process by a discriminator in the antagonistic learning model to determine an objective function;

in step S420, the fixed arbiter propagates the generator backward according to the objective function, and trains the generator of the antagonistic learning model to obtain the trained antagonistic learning model.

In the embodiment of the present disclosure, since the probability that the true label and the predicted label of the sample image belong to the true data are different, and in order to improve the accuracy, the probability of the predicted label may tend to 0, and the probability of the true label may tend to 1. Thus, the difference between the real tag and the predicted tag can be calculated, where the difference can be the difference between the probabilities that the real tag and the predicted tag belong to real data. After determining the difference, the difference between the two may be minimized to determine the objective function. The objective function may be understood as a loss function for training the generator. The arbiter can be used to maximally distinguish the trained generator from the true distribution, so that the difference between the predicted label and the true label can be minimized through multiple rounds of iterative training of the arbiter, and the difference between the minimized and true distributions can be further used as an objective function of the trained generator, which can be specifically expressed by formula (1):

wherein x represents the samples generated by the generator, x-Pr represents the true tags, x-Pg represents the result of sampling from the generator distribution, g represents the generator, and D represents the discriminator. It should be noted that the training of the arbiter can be performed by performing the maximization process on the arbiter and determining the objective function, so as to obtain a trained arbiter.

After the objective function is obtained, the objective function can be used as a training target, and the generator in the confrontation learning model is trained to obtain a generator with good training performance. After training the arbiter, the parameters of the arbiter may be fixed and the generator may be trained. Specifically, the arbiter may be fixed, and the generator may be propagated backward according to the objective function to train the generator in the antagonistic learning model, so as to obtain a trained antagonistic learning model. The idea of back propagation is: iteratively processing the training tuple data set, comparing the net prediction of each tuple with the actually known class label for learning, and modifying the weight so that the mean square error between the net prediction and the actual class is minimum for each training sample. This modification is done "backwards", i.e. by the output layer, via each hidden layer, to the first hidden layer (hence the name back-propagation). When the objective function is minimal or the model converges, the process of training the generator is stopped. The main steps of back propagation include: the total difference is calculated, the weight update of the hidden layer (before the weight of each edge is to be updated, the influence of the edge on the final output result must be known), the weight is updated, and the back propagation process is completed. Then, only iteration is needed to be carried out, the weight of the edge is continuously adjusted, the deviation between the output of the network and the actual result is corrected, and the weight of the generator of the counterstudy model can be updated according to the index output by the discriminator so as to execute the training process.

In summary, the process of training the antagonistic learning model can include the steps of: firstly, inputting a batch of images into a model, and outputting a prediction label after the model is subjected to forward calculation; secondly, inputting the output prediction label and the real label into a discriminator; thirdly, the discriminator is trained to obtain the discriminator by maximizing the difference between the indexes of the predicted label and the real label through a plurality of rounds of iterative training; fourthly, fixing parameters of the discriminator, carrying out backward propagation on the model according to indexes given by the discriminator, and updating the weight of the generator; and fifthly, repeating the first step to the fourth step until the generator finally converges. Wherein, the discriminator maximally distinguishes the trained generator from the real distribution; the generator is the difference between the minimum and true distributions.

After the trained confrontation learning model is obtained, a discriminator used in a training stage can be omitted, and the feature extraction is carried out on the image to be processed according to the generator of the trained confrontation learning model so as to obtain the image feature for representing the image to be processed. Specifically, the image features of the image to be processed can be extracted by performing convolution operation on the image to be processed through network layers such as a convolution layer, a pooling layer and a full-link layer of a trained generator, the image features are processed through the generator to obtain a prediction label of the image to be processed, and the prediction label is used as a prediction result. The prediction label of the image to be processed can also be a score distribution, so that the aesthetic quality of the image to be processed can be accurately and comprehensively evaluated through the score distribution.

The prediction result in the embodiment of the present disclosure is a score distribution, and the score distribution is used to represent the distribution condition of each level, and may be specifically determined by each level and the probability that the image to be processed belongs to each level. The plurality of ranks may be, for example, a first rank to a tenth rank, and may be represented by 1 to 10 points. Each level may be, for example, 1-10 points, e.g., a 1 point probability of 5%, a 5 point probability of 20%, a 7 point probability of 60%, a 9 point probability of 10%, a 10 point probability of 5%, and other levels of 0. The score distribution may be represented in the form of a distribution histogram, based on which the score distribution may be used to characterize the distribution histogram of the aesthetic quality score of the image at the first to tenth levels (1-10 points). In the embodiment of the disclosure, the aesthetic quality evaluation is performed through the score distribution, so that the limitation of evaluation through a certainty value is avoided, the subjectivity and the individuation of the evaluation of the image to be processed can be highlighted, and the more accurate and richer aesthetic evaluation of the image to be processed can be realized through the score distribution. The prediction result of the image to be processed is determined through the trained counterstudy model, so that the accuracy and the efficiency can be improved, and the reliability of the prediction label can be improved.

With continuing reference to fig. 2, in step S230, the aesthetic quality evaluation is performed on the image to be processed according to the prediction result to determine an evaluation result of the image to be processed.

In the embodiment of the disclosure, after the score distribution of the image to be processed is obtained, the aesthetic quality evaluation may be performed on the image to be processed according to the prediction result formed by the score distribution. The aesthetic quality evaluation refers to the evaluation of the clown degree of an image, and the aesthetic quality evaluation of the image focuses on the evaluation of higher semantic information, but not on the evaluation of the basic image quality such as the definition of the image, the noise pollution degree and the like. The result of the image aesthetic quality evaluation has strong subjectivity, and different users have great difference in the quality evaluation of the same image.

When the prediction result is the score distribution, after the to-be-processed image is given, the system can give aesthetic evaluation with reference significance according to the trained counterstudy model, the evaluation result is obtained according to the score distribution, the evaluation result of the to-be-processed image can also be the score distribution instead of a determined numerical value, and the system has stronger guiding significance in a real scene. The distribution condition of the image to be processed at each level can be determined according to the grading distribution, so that the comprehensive evaluation is performed, the limitation and inaccuracy caused by evaluation only according to a determined numerical value are avoided, the influence of subjective factors can be avoided, the accuracy is improved, and the comprehensive accurate evaluation on the aesthetic quality of the image is realized.

The overall flow chart of the quality evaluation is schematically shown in fig. 5, and referring to the flow chart shown in fig. 5, the overall flow chart mainly includes the following parts: the image 501, the generator 502 in the confrontation learning model, the prediction tag 503, the real tag 504 and the discriminator 505, and the interaction process among the parts may include:

first, an image 501 is obtained, where the image may be a sample image or an image to be processed, and then the image is input into a generator of a countermeasure learning model, and a prediction label of the sample image or the image to be processed, where the prediction label may be a score distribution, is output. Next, the prediction tag 503 and the true tag 504 of the sample image are simultaneously input to the discriminator 505 to obtain an output of the discriminator, which refers to the probability that the prediction tag and the true tag belong to true data. Further, the discriminator and the generator may be trained according to the probability of the sample image to obtain a trained confrontation learning model. It should be noted that, when the image is an image to be processed, only the prediction tag needs to be obtained. When the image is a sample image, all processes need to be performed.

The technical scheme provided by the embodiment of the disclosure provides an aesthetic quality evaluation method based on a counterstudy framework. The image to be processed is input into a trained confrontation learning model for prediction, and the trained confrontation learning model is obtained by inputting a prediction label and a real label of a sample image into a discriminator to train the discriminator and a generator. Based on a generator in the countermeasure learning model, the score distribution of the image to be processed can be accurately and reasonably determined, and then the distribution condition of the image to be processed at each grade or each score can be determined according to the score distribution, so that accurate evaluation can be carried out in an all-around manner, the limitation and inaccuracy of image quality evaluation only according to a determined numerical value are avoided, the influence of subjective factors can be avoided, the efficiency is improved, more reasonable aesthetic evaluation distribution is provided, the accuracy is improved, and all-around accurate evaluation on the image aesthetic quality is realized. When high-dimensional distribution such as score distribution is processed, fitting distribution can be better and better achieved through the confrontation learning model, accuracy and efficiency of determining the fitting distribution are improved, compared with other models, the effect is better, the method is more suitable for image aesthetic quality evaluation according to the score distribution, the effect of image aesthetic quality evaluation is improved, and applicability is improved.

In the present exemplary embodiment, there is also provided an image quality evaluation apparatus, as shown in fig. 6, the apparatus 600 may include:

an image obtaining module 601, configured to obtain an image to be processed; a prediction result determining module 602, configured to perform prediction processing on the image to be processed to generate a prediction result for characterizing score distribution if a determination operation for characterizing authenticity is completed according to an index for a sample image; an image evaluation module 603, configured to perform aesthetic quality evaluation on the image to be processed according to the prediction result, so as to determine an evaluation result of the image to be processed.

In an exemplary embodiment of the present disclosure, the prediction result determining module includes: the characteristic extraction module is used for extracting the characteristics of the image to be processed to obtain image characteristics; and the prediction label determining module is used for acquiring the prediction label of the image to be processed according to the image characteristics and taking the prediction label as the prediction result.

In an exemplary embodiment of the present disclosure, the predictive tag determination module includes: and the forward calculation module is used for performing forward calculation on the image characteristics through a generator in the trained confrontation learning model to determine the prediction label.

In an exemplary embodiment of the present disclosure, the apparatus further includes: and the model training module is used for training the confrontation learning model through the sample image and the artificially labeled labeling information of the sample image to obtain a trained confrontation learning model for performing prediction processing on the image to be processed.

In an exemplary embodiment of the present disclosure, the model training module includes: a label determination module for performing feature extraction on the sample image through a generator in the confrontation learning model to determine a prediction label of the sample image; and the joint training module is used for training the confrontation learning model according to the prediction label of the sample image and the real label of the sample image to obtain the trained confrontation learning model.

In an exemplary embodiment of the disclosure, the joint training module includes: the index calculation module is used for determining indexes of a real label of the sample image and a prediction label of the sample image through a discriminator in the confrontation learning model; and the training control module is used for training the confrontation learning model according to the indexes.

In an exemplary embodiment of the present disclosure, the training control module includes: a difference processing module for minimizing the difference between the indexes by a discriminator of the antagonistic learning model to determine an objective function; and the generator training module is used for fixing the discriminator, carrying out backward propagation on the generator according to the target function, and training the generator of the confrontation learning model so as to obtain the trained confrontation learning model.

It should be noted that, the specific details of each module in the image quality evaluation apparatus have been elaborated in the corresponding image quality evaluation method, and therefore are not described herein again.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Moreover, although the steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that the steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.

In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

An electronic device 700 according to this embodiment of the invention is described below with reference to fig. 7. The electronic device 700 shown in fig. 7 is only an example and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 7, electronic device 700 is embodied in the form of a general purpose computing device. The components of the electronic device 700 may include, but are not limited to: the at least one processing unit 710, the at least one memory unit 720, and a bus 730 that couples various system components including the memory unit 720 and the processing unit 710.

Wherein the storage unit stores program code that is executable by the processing unit 710 such that the processing unit 710 performs the steps according to various exemplary embodiments of the present invention as described in the above section "exemplary method" of the present specification. For example, the processing unit 710 may perform the steps as shown in fig. 2.

The storage unit 720 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)7201 and/or a cache memory unit 7202, and may further include a read only memory unit (ROM) 7203.

The storage unit 720 may also include a program/utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 730 may be any representation of one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The display unit 740 may be a display having a display function to show a processing result obtained by the processing unit 710 performing the method in the present exemplary embodiment through the display. The display includes, but is not limited to, a liquid crystal display or other display.

The electronic device 700 may also communicate with one or more external devices 800 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 700, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 700 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 750. Also, the electronic device 700 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) via the network adapter 760. As shown, the network adapter 760 communicates with the other modules of the electronic device 700 via the bus 730. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above section "exemplary methods" of the present description, when said program product is run on the terminal device.

According to the program product for realizing the method, the portable compact disc read only memory (CD-ROM) can be adopted, the program code is included, and the program product can be operated on terminal equipment, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.

Claims

1. An image quality evaluation method is characterized by comprising:

acquiring an image to be processed;

if the judgment operation for representing the authenticity is completed according to the indexes of the sample image, performing prediction processing on the image to be processed to generate a prediction result for representing the grading distribution;

and performing aesthetic quality evaluation on the image to be processed according to the prediction result to determine an evaluation result of the image to be processed.

2. The image quality evaluation method according to claim 1, wherein the performing prediction processing on the image to be processed to obtain a prediction result for characterizing score distribution comprises:

extracting the features of the image to be processed to obtain image features;

and acquiring a prediction label of the image to be processed according to the image characteristics, and taking the prediction label as the prediction result.

3. The image quality evaluation method according to claim 2, wherein the obtaining of the prediction label of the image to be processed according to the image feature comprises:

and determining the prediction label by forward computing the image characteristics through a generator in a trained confrontation learning model.

4. The image quality evaluation method according to claim 1, characterized in that the method further comprises:

and training the confrontation learning model through the sample image and the manually marked marking information of the sample image to obtain a trained confrontation learning model for performing prediction processing on the image to be processed.

5. The image quality evaluation method according to claim 4, wherein the training of the confrontation learning model through the sample images and the labeled information of the manual labeling of the sample images to obtain the trained confrontation learning model for performing the prediction processing on the image to be processed comprises:

performing feature extraction on the sample image through a generator in the antagonistic learning model to determine a prediction label of the sample image;

and training the confrontation learning model according to the prediction label of the sample image and the real label of the sample image to obtain the trained confrontation learning model.

6. The image quality evaluation method according to claim 5, wherein the training of the confrontation learning model according to the prediction label of the sample image and the true label of the sample image to obtain the trained confrontation learning model comprises:

determining, by a discriminator in the antagonistic learning model, an indicator of a true label of the sample image and a predicted label of the sample image;

and training the confrontation learning model according to the indexes.

7. The image quality evaluation method according to claim 6, wherein the training of the antagonistic learning model according to the index includes:

minimizing, by a discriminator of the antagonistic learning model, a difference between the indices to determine an objective function;

and fixing the discriminator, carrying out backward propagation on the generator according to the objective function, and training the generator of the confrontation learning model to obtain the trained confrontation learning model.

8. An image quality evaluation apparatus, comprising:

the image acquisition module is used for acquiring an image to be processed;

the prediction result determining module is used for performing prediction processing on the image to be processed to generate a prediction result for representing grade distribution if the judging operation for representing authenticity is completed according to the index aiming at the sample image;

and the image evaluation module is used for performing aesthetic quality evaluation on the image to be processed according to the prediction result so as to determine the evaluation result of the image to be processed.

9. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the image quality assessment method of any of claims 1-7 via execution of the executable instructions.

10. A computer-readable storage medium on which a computer program is stored, the computer program, when being executed by a processor, implementing the image quality evaluation method according to any one of claims 1 to 7.