CN113012089B

CN113012089B - Image quality evaluation method and device

Info

Publication number: CN113012089B
Application number: CN201911319997.6A
Authority: CN
Inventors: 成超; 蔡媛; 樊鸿飞; 汪贤; 鲁方波
Original assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Current assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Filing date: 2019-12-19
Publication date: 2024-07-09
Anticipated expiration: 2039-12-19

Abstract

The embodiment of the invention provides an image quality evaluation method and device, wherein the method comprises the following steps: obtaining a plurality of thermodynamic diagrams corresponding to an image to be evaluated, wherein each thermodynamic diagram is used for indicating the significance level of a characteristic part of an object in the image to be evaluated, different thermodynamic diagrams are used for different characteristic parts of the object, and a significance value reflecting the significance level of each characteristic part contained in the image to be evaluated is obtained according to the thermodynamic diagram corresponding to each characteristic part; and generating an evaluation result for the image to be evaluated according to the obtained significant value, wherein the evaluation result is used for indicating the quality of the object in the image to be evaluated. When the scheme provided by the embodiment of the invention is applied to image quality evaluation, the accuracy of image quality evaluation can be improved.

Description

Image quality evaluation method and device

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for evaluating image quality.

Background

In order to obtain an image containing a specific object and having a better quality, the quality of the image may be evaluated, so that it is determined whether the image is an image containing the specific object and having a better quality according to the image quality evaluation result.

In the prior art, when evaluating the quality of an image, the sharpness of the entire image is generally detected, and the detection result is used as the result of evaluating the quality of the image. For example, when the specific object is a human face, if the sharpness of the entire image is high, the image is considered to be a high-quality image including the human face; if the definition of the whole image is poor, the image is considered to be a low-quality image containing a human face.

However, when one image is considered to be a high-quality image because of higher definition, there may be a case where a specific object is not completely displayed or not displayed at all in the image, and in this case, the actual quality of the image is not high from the viewpoint of the specific object. Therefore, when image quality is evaluated by the conventional technique, there is a problem that the accuracy of the evaluation result is low.

Disclosure of Invention

The embodiment of the invention aims to provide an image quality evaluation method and device so as to improve the accuracy of image quality evaluation. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides an image quality evaluation method, including:

obtaining a plurality of thermodynamic diagrams corresponding to an image to be evaluated, wherein each thermodynamic diagram is used for indicating the significance degree of a characteristic part of an object in the image to be evaluated, and different thermodynamic diagrams are used for different characteristic parts of the object;

Obtaining a significance value reflecting the significance degree of each feature part contained in the image to be evaluated according to the thermodynamic diagram corresponding to each feature part;

and generating an evaluation result for the image to be evaluated according to the obtained significant value, wherein the evaluation result is used for indicating the quality of the object in the image to be evaluated.

In an embodiment of the present invention, the obtaining a plurality of thermodynamic diagrams corresponding to the image to be evaluated includes:

Detecting the significance of each pixel point in the image to be evaluated for representing each characteristic part of the object by adopting a pre-trained significance detection model, and obtaining thermodynamic diagrams respectively reflecting the significance of each pixel point in the image to be evaluated for representing each characteristic part, wherein the significance detection model is as follows: and taking a sample image as a model input, taking the significance degree of each pixel point belonging to each characteristic part in the sample image as training supervision information, and training a preset neural network model to obtain a model which is used for obtaining thermodynamic diagrams corresponding to each characteristic part.

In one embodiment of the present invention, the obtaining a saliency value reflecting a saliency degree of each feature included in the image to be evaluated according to the thermodynamic diagram corresponding to each feature includes:

And determining the pixel value of the pixel point with the highest degree of saliency reflected in the thermodynamic diagram aiming at the thermodynamic diagram corresponding to each characteristic part, and taking the determined pixel value as the saliency value reflecting the degree of saliency of the characteristic part contained in the image to be evaluated.

For the thermodynamic diagram corresponding to each feature part, calculating an average pixel value according to the pixel values of all pixel points reflecting the significance degree in the thermodynamic diagram, and taking the calculated average pixel value as the significance value reflecting the significance degree of the feature part in the image to be evaluated.

In one embodiment of the present invention, generating an evaluation result for the image to be evaluated according to the obtained saliency value includes:

and calculating an average significance value according to the obtained significance value, and determining the calculated average significance value as an evaluation result for the image to be evaluated.

In one embodiment of the present invention, when the object is a target face in the image to be evaluated, obtaining a plurality of thermodynamic diagrams corresponding to the image to be evaluated includes:

obtaining first to fifth thermodynamic diagrams, wherein the first thermodynamic diagram is used for indicating the saliency degree of the left eye of the target face in the image to be evaluated, the second thermodynamic diagram is used for indicating the saliency degree of the right eye of the target face in the image to be evaluated, the third thermodynamic diagram is used for indicating the saliency degree of the nose tip of the target face in the image to be evaluated, the fourth thermodynamic diagram is used for indicating the saliency degree of the left mouth corner of the target face in the image to be evaluated, and the fifth thermodynamic diagram is used for indicating the saliency degree of the right mouth corner of the target face in the image to be evaluated.

In a second aspect, an embodiment of the present invention provides an image quality evaluation apparatus, including:

a thermodynamic diagram obtaining module, configured to obtain a plurality of thermodynamic diagrams corresponding to an image to be evaluated, where each thermodynamic diagram is used to indicate a degree of saliency of a feature of an object in the image to be evaluated, and different thermodynamic diagrams are used for different feature of the object;

the salient value obtaining module is used for obtaining salient values reflecting the salient degree of each characteristic part contained in the image to be evaluated according to the thermodynamic diagram corresponding to each characteristic part;

And the evaluation result generation module is used for generating an evaluation result aiming at the image to be evaluated according to the obtained significant value, wherein the evaluation result is used for indicating the quality of the object in the image to be evaluated.

In one embodiment of the present invention, the thermodynamic diagram obtaining module is specifically configured to:

In one embodiment of the present invention, the saliency value obtaining module is specifically configured to:

Determining a pixel value of a pixel point with the highest significance reflected in the thermodynamic diagram aiming at the thermodynamic diagram corresponding to each characteristic part, and taking the determined pixel value as a significance value reflecting the significance of the characteristic part contained in the image to be evaluated;

In one embodiment of the present invention, the evaluation result generating module is specifically configured to:

In one embodiment of the present invention, in the case that the object is a target face in the image to be evaluated, the thermodynamic diagram obtaining module is specifically configured to: obtaining first to fifth thermodynamic diagrams, wherein the first thermodynamic diagram is used for indicating the saliency degree of the left eye of the target face in the image to be evaluated, the second thermodynamic diagram is used for indicating the saliency degree of the right eye of the target face in the image to be evaluated, the third thermodynamic diagram is used for indicating the saliency degree of the nose tip of the target face in the image to be evaluated, the fourth thermodynamic diagram is used for indicating the saliency degree of the left mouth corner of the target face in the image to be evaluated, and the fifth thermodynamic diagram is used for indicating the saliency degree of the right mouth corner of the target face in the image to be evaluated.

In a third aspect, an embodiment of the present invention provides a terminal device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

and a processor, configured to implement the method steps described in the first aspect when executing the program stored in the memory.

In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium, in which a computer program is stored, which computer program, when being executed by a processor, implements the method steps according to the first aspect.

From the above, when the scheme provided by the embodiment of the invention is applied to image quality evaluation, a plurality of thermodynamic diagrams corresponding to an image to be evaluated are obtained, wherein each thermodynamic diagram is used for indicating the saliency degree of a characteristic part of an object in the image to be evaluated, different thermodynamic diagrams are used for different characteristic parts of the object, a saliency value reflecting the saliency degree of each characteristic part included in the image to be evaluated is obtained according to the thermodynamic diagram corresponding to each characteristic part, and an evaluation result aiming at the image to be evaluated is generated according to the saliency value corresponding to each thermodynamic diagram, and is used for indicating the quality of the object in the image to be evaluated. Since the obtained thermodynamic diagram is used for reflecting the significance degree of each pixel point in the image to be evaluated for representing the characteristic part, the significance of the characteristic part reflected by the thermodynamic diagram is weakened when the characteristic part is not fully displayed or is not displayed at all in the image, and conversely, the significance of the characteristic part reflected by the thermodynamic diagram is enhanced when the characteristic part is clearly displayed in the image. In this case, the object in the image can be considered when the image is evaluated based on each thermodynamic diagram, and the accuracy of evaluating the quality of the image can be improved.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of an image quality evaluation method according to an embodiment of the present invention;

fig. 2a is a schematic diagram of an image including a face region according to an embodiment of the present invention;

FIG. 2b is a schematic diagram of a thermodynamic diagram according to an embodiment of the present invention;

FIG. 3a is a schematic diagram of a first thermodynamic diagram for an image to be evaluated according to an embodiment of the present invention;

FIG. 3b is a schematic diagram of a second thermodynamic diagram for an image to be evaluated according to an embodiment of the present invention;

FIG. 3c is a schematic diagram of a third thermodynamic diagram for an image to be evaluated according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of an image quality evaluation device according to an embodiment of the present invention;

Fig. 5 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In some scenes, it is required to perform object recognition on an image, for example, it is required to recognize a face in the image, in which case, the image to be subjected to object recognition needs to have a higher image quality, so that a better object recognition result can be obtained. For this reason, before object recognition is performed, it is necessary to evaluate the quality of an image in order to find an image of higher quality based on the quality evaluation result. In view of the above, the embodiment of the invention provides an image quality evaluation method and device.

The following describes an image quality evaluation method provided by the embodiment of the present invention.

Referring to fig. 1, fig. 1 is a flowchart of an image quality evaluation method according to an embodiment of the present invention, where the method includes S101-S103.

S101: and acquiring a plurality of thermodynamic diagrams corresponding to the image to be evaluated.

The thermodynamic diagrams are used for indicating the degree of salience of the characteristic parts of the object in the image to be evaluated, and different thermodynamic diagrams are used for different characteristic parts of the object.

The object may be a living organism or an inanimate organism. Specifically, the living organisms may include humans, animals, plants, etc., and the non-living organisms may include houses, vehicles, tables, etc.

The characteristic parts are different for different objects.

In one embodiment of the present invention, the object may be a face. In this case, the characteristic parts of the subject may include eyes, nose, mouth, eyebrows, and the like. For example, in an alternative embodiment of the present invention, in a case where the object is a target face in the image to be evaluated, acquiring a plurality of thermodynamic diagrams corresponding to the image to be evaluated includes: obtaining first to fifth thermodynamic diagrams, wherein the first thermodynamic diagram is used for indicating the saliency degree of the left eye of the target face in the image to be evaluated, the second thermodynamic diagram is used for indicating the saliency degree of the right eye of the target face in the image to be evaluated, the third thermodynamic diagram is used for indicating the saliency degree of the nose tip of the target face in the image to be evaluated, the fourth thermodynamic diagram is used for indicating the saliency degree of the left mouth corner of the target face in the image to be evaluated, and the fifth thermodynamic diagram is used for indicating the saliency degree of the right mouth corner of the target face in the image to be evaluated. Correspondingly, the obtained evaluation result is used for indicating the face quality of the image to be evaluated.

When the object is a human body, the characteristic parts of the object may include hands, legs, waists, and the like. When the object is an animal face, the characteristic parts of the object may include eyes, nose, mouth, eyebrows, and the like of the animal. When the object is a plant, the characteristic part of the object may include leaves, stems, and the like; when the object is a house, the characteristic parts of the object may include windows, doors, etc.; when the object is a vehicle, the characteristic parts of the object may include wheels, a vehicle body, and the like; when the object is a table, the characteristic parts of the object may include a table leg, a table top, and the like.

For each characteristic part of the object, the thermodynamic diagram is used for reflecting the significance degree of each pixel point in the image to be evaluated for representing the characteristic part. Therefore, each pixel point in the thermodynamic diagram corresponds to each pixel point in the image to be evaluated one by one, and the size of the thermodynamic diagram is consistent with the size of the image to be evaluated.

For each feature, it may be only a part of the image to be evaluated, so each pixel in the image to be evaluated does not belong to the region where the feature is located in the image to be evaluated. When a pixel point is located in a region where a feature is located in an image to be evaluated, the pixel point is used for representing that the feature is more significant. On the contrary, when a pixel is not located in the region where the feature is located in the image to be evaluated, the pixel is used for representing that the feature is less significant.

Since different feature parts of the object have different characteristics, in one embodiment of the present invention, for each feature part of the object and one pixel point in the image to be evaluated, reflecting the significance of the pixel point for representing the feature part can be understood as follows: reflecting the possibility that the pixel belongs to the region where the feature part is located in the image. The higher the likelihood, the higher the degree of significance; conversely, the smaller the likelihood, the lower the level of significance.

For example: assuming that each pixel point in the image to be evaluated is a pixel point 1, a pixel point 2, … … and a pixel point N respectively, when the characteristic part of the object is the nose tip of the human face, if the possibility that the pixel point 1, the pixel point 2 and the pixel point 3 in the image to be evaluated belong to the region where the nose tip is located is higher, the significance degree that the pixel point 1, the pixel point 2 and the pixel point 3 in the image to be evaluated represent the region where the nose tip is located is higher; if the possibility that the pixel point 4, the pixel points 5, … … and the pixel point N in the image to be evaluated belong to the region where the nose tip is located in the image is smaller, the significance degree that the pixel point 4, the pixel points 5, … … and the pixel point N in the image to be evaluated represent the region where the nose tip is located is lower.

Specifically, for each feature of the object, a numerical value within a certain range may be used to reflect the significance level of each pixel point in the image to be evaluated for representing the feature. For example: the values within the certain range may be pixel values of each pixel point in the thermodynamic diagram. Assuming that the characteristic part of the object is the nose tip of the human face, reflecting that the pixel point in the image to be evaluated represents the significant degree of the nose tip to be higher when the pixel value of the pixel point in the thermodynamic diagram is larger; conversely, when the pixel value of the pixel point in the thermodynamic diagram is smaller, the pixel point in the image to be evaluated is reflected to represent the saliency degree of the nose tip is lower.

Taking fig. 2a and fig. 2b as an example, fig. 2a is a schematic diagram of an image including a face area according to an embodiment of the present invention, and fig. 2a is an image to be evaluated, where feature parts of a face in the image may include a left eye, a right eye, a nose tip, a left mouth corner, and a right mouth corner. Fig. 2b is a schematic diagram of a thermodynamic diagram provided in an embodiment of the present invention, where the thermodynamic diagram is used to reflect the degree to which each pixel point in fig. 2a represents a significant degree of feature being the tip of the nose, and the sizes of the images in fig. 2a and 2b are consistent.

As can be seen from fig. 2b, the brightness of each pixel point in the white area in fig. 2b is higher, that is, the pixel value of each pixel point in the white area is larger, so that the significance degree of each pixel point in the white area for reflecting the tip of the nose in fig. 2a is higher, that is, the probability that each pixel point in the white area belongs to the area where the tip of the nose is in fig. 2a is higher; similarly, in fig. 2b, the brightness of each pixel point in the black area is lower, which means that the pixel value of each pixel point in the black area is smaller, so that the significance degree of each pixel point in the black area for reflecting the nose tip in fig. 2a is lower, that is, the probability that each pixel point in the black area belongs to the area where the nose tip is in fig. 2a is smaller.

Specifically, when the thermodynamic diagram is obtained for each feature part of the object, a possibility that each pixel point in the image to be evaluated belongs to the region where the feature part is located in the image can be obtained, and the value of the pixel value of each pixel point in the thermodynamic diagram is set according to the obtained possibility that each pixel point corresponds to, so that the thermodynamic diagram is obtained.

In one embodiment of the present invention, the likelihood may be represented by a value in a range of [0,1], and in the case where the pixel value of the pixel in the thermodynamic diagram is represented by an 8-bit value, an integer corresponding to a product of the likelihood and 255 may be used as the pixel value of the pixel in the thermodynamic diagram.

For example: when the above possibilities are: 0.5, 0.5×255=127.5, and rounding up the 127.5 to obtain 128, the pixel value of the corresponding pixel point in the thermodynamic diagram may be 128.

For example: assuming that each pixel point in the image to be evaluated is a pixel point 1, a pixel point 2, … … and a pixel point N respectively, when the characteristic part of the object is the nose tip of the human face, if the possibility that the pixel point 1, the pixel point 2 and the pixel point 3 in the image to be evaluated belong to the region where the nose tip is located is high, setting the value of each pixel point corresponding to the pixel point 1, the pixel point 2 and the pixel point 3 in the generated image to be high; if the probability that the pixel point 4, the pixel points 5, … … and the pixel point N in the image to be evaluated belong to the region where the nose tip is located in the image is smaller, setting the value of the pixel value of each pixel point corresponding to the pixel point 4, the pixel point 5, … … and the pixel point N in the generated image to be lower, and determining the set image as a thermodynamic diagram.

S102: and obtaining a significance value reflecting the significance degree of each feature part included in the image to be evaluated according to the thermodynamic diagram corresponding to each feature part.

The above-described salient values are quantized values reflecting the salient degrees of the images to be evaluated including the respective feature portions. The higher the degree of salience of each feature included in the image to be evaluated, the larger the salience value, and the lower the degree of salience of each feature included in the image to be evaluated, the smaller the salience value.

The above significant value may be expressed in the following two ways:

first case: the above significant values are represented by standard fractional values, for example: the standard fraction may be a tenth fraction, a percentile fraction, or the like.

Based on the first situation, in one embodiment of the present invention, when obtaining the saliency value reflecting the saliency degree of each feature part of the image to be evaluated according to the thermodynamic diagram corresponding to each feature part, the pixel value of each pixel point reflecting the saliency degree in the thermodynamic diagram may be statistically analyzed, the pixel value after the statistical analysis is converted into the corresponding standard score value, and the converted standard score value is used as the saliency value reflecting the saliency degree of the feature part of the image to be evaluated.

The statistical analysis may include: calculate the maximum value, calculate the average value, etc.

In one embodiment of the present invention, when the pixel value of the pixel point in the thermodynamic diagram is represented by an 8-bit value and the pixel value of the pixel point after the statistical analysis is converted into the corresponding standard score value, when the standard score value is represented by an N-ary score value, an integer corresponding to a product of a ratio of the pixel value after the statistical analysis and 255 and N may be used as the converted corresponding N-ary score value.

For example: when the pixel value after statistical analysis is: 230, n fraction values are: a tenth score value (230/255) is equal to 10= 9.020, and 9.020 is taken upward to obtain 9.0, and when the pixel value after statistical analysis is 230, the corresponding tenth score value is 9.0.

Second case: the above significant value is represented by a pixel value.

Based on the second aspect, in one embodiment of the present invention, when obtaining the saliency value reflecting the saliency of each feature portion included in the image to be evaluated according to the thermodynamic diagram corresponding to each feature portion, the pixel value of the pixel point with the highest saliency reflected in the thermodynamic diagram may be determined for each thermodynamic diagram corresponding to each feature portion, and the determined pixel value may be used as the saliency value reflecting the saliency of the feature portion included in the image to be evaluated.

Specifically, the pixel value of the pixel point with the highest degree of significance reflected in each thermodynamic diagram can be determined by the calculation formula response=max (hetmap). Wherein heatmap denotes a pixel value of each pixel reflecting a degree of significance in the thermodynamic diagram, max () is used to calculate the maximum value, and Response denotes a pixel value calculated from Max (hetmap).

For example: when the pixel values of the pixels reflecting the degree of saliency in the thermodynamic diagram are 35, 55, 65, 76, 120, 200, 230, respectively, the pixel value of the pixel reflecting the highest degree of saliency is Max (35, 55, 65, 76, 120, 200, 230) =230.

In this way, according to the pixel value of the pixel point with the highest degree of saliency reflected in the thermodynamic diagrams as the saliency value reflecting the degree of saliency of the feature part included in the image to be evaluated, a more accurate saliency value corresponding to each thermodynamic diagram can be obtained.

In another embodiment of the present invention, when obtaining the saliency value reflecting the saliency of each feature portion included in the image to be evaluated according to the thermodynamic diagram corresponding to each feature portion, the average pixel value may be calculated according to the pixel value of each pixel point reflecting the saliency in the thermodynamic diagram corresponding to each feature portion, and the calculated average pixel value may be used as the saliency value reflecting the saliency of each feature portion included in the image to be evaluated.

For example: when the pixel values of the respective pixels reflecting the degree of significance in the thermodynamic diagram are 35, 55, 65, 76, 120, 200, 230, respectively, the calculated average pixel value is (35+55+65+76+120+200+230)/7=114.

In this way, according to the pixel values of the pixel points reflecting the saliency degree in the thermodynamic diagrams, the average pixel value is calculated, and the calculated average pixel value is used as the saliency value reflecting the saliency degree of the characteristic part included in the image to be evaluated, so that a relatively accurate saliency value corresponding to each thermodynamic diagram can be obtained.

S103: and generating an evaluation result for the image to be evaluated according to the obtained significant value.

The obtained significant values are significant values corresponding to each thermodynamic diagram, and when an evaluation result for the image to be evaluated is generated, statistical analysis can be performed on each obtained significant value, and the significant value after the statistical analysis is determined as the evaluation result for the image to be evaluated. The above-described evaluation result is used to indicate the quality of the object in the image to be evaluated.

Wherein the statistical analysis includes: calculate the maximum value, calculate the average value, etc.

The evaluation result may be expressed in two ways:

the first way is: when the above evaluation result is expressed by a standard score value, for example: the standard fraction may be a tenth fraction, a percentile fraction, or the like.

As can be seen from the description in S102, the obtained significant value may be expressed in terms of a pixel value or in terms of a standard score value.

Based on the first manner, in one embodiment of the present invention, when the significant value is represented by a pixel value, the significant value after statistical analysis may be converted into a corresponding standard score value, and the converted standard score value is used as an evaluation result of the image to be evaluated.

In one embodiment of the present invention, when the statistically analyzed significant value is converted into the corresponding standard score value, and the pixel value of the pixel point in the thermodynamic diagram is represented by an 8-bit value, and when the standard score value is represented by an N-ary score value, an integer corresponding to a product of a ratio of the statistically analyzed significant value to 255 and N may be used as the converted corresponding N-ary score value.

For example: the significance values after statistical analysis were: 200, N fraction values are: a ten-system score value (200/255) ×10=7.84, and the 7.84 is taken up to an integer of 8.0, and when the significant value after statistical analysis is 200, the corresponding ten-system score value is converted into 8.0.

Based on the first mode, in another embodiment of the present invention, when the significant value is expressed as a standard score value, the significant value after statistical analysis may be directly used as an evaluation result for the image to be evaluated. In this way, the corresponding score value is adopted as the evaluation result of the image to be evaluated, so that the image quality of the image to be evaluated can be more intuitively determined according to the evaluation result.

In the second mode, when the above evaluation result is expressed by a pixel value.

Based on the second mode, since the saliency value can be represented by a pixel value, the calculated average saliency value is also referred to as a pixel value, and further, the evaluation result of the image to be evaluated is also referred to as a pixel value.

Specifically, in one embodiment of the present invention, the generating the evaluation result for the image to be evaluated according to the obtained saliency value may be calculating an average saliency value according to the obtained saliency value, and determining the calculated average saliency value as the evaluation result for the image to be evaluated.

For example: when the obtained saliency values are 210, 220, 230, 240, and 250, respectively, and the average saliency value is (210+220+230+240+250)/5=230, the average saliency value 230 may be used as the evaluation result of the image to be evaluated.

In this way, an average significant value is calculated according to the obtained significant value, and the calculated average significant value is used as an evaluation result of the image to be evaluated, so that an accurate evaluation result of the image to be evaluated can be obtained.

Fig. 3a is a schematic diagram of a first thermodynamic diagram for an image to be evaluated according to an embodiment of the present invention. The leftmost image in fig. 3a is an image to be evaluated, and the second to sixth images from the leftmost image in fig. 3a are thermodynamic diagrams for each feature part of the face, wherein each pixel point in the image to be evaluated is used for representing the significance degree of the feature part. The second to sixth sheets from the leftmost sheet are in turn: a thermodynamic diagram corresponding to nose tip, a thermodynamic diagram corresponding to right eye, a thermodynamic diagram corresponding to left eye, a thermodynamic diagram corresponding to right mouth corner, and a thermodynamic diagram corresponding to left mouth corner.

Similarly, fig. 3b is a schematic diagram of a second thermodynamic diagram for an image to be evaluated according to an embodiment of the present invention, and fig. 3c is a schematic diagram of a third thermodynamic diagram for an image to be evaluated according to an embodiment of the present invention.

According to the pixel values of the pixels in the thermodynamic diagrams from the second left to the sixth left shown in fig. 3a, a saliency value of each thermodynamic diagram reflecting the saliency degree of the first image containing each feature part at the left side in fig. 3a can be calculated, wherein the calculation result is:

The second thermodynamic diagram from the leftmost side corresponds to a significant value of 0.93;

The third thermodynamic diagram from the leftmost side corresponds to a significant value of 0.95;

the significant value corresponding to the fourth thermodynamic diagram from the leftmost side is 0.99;

The significance value corresponding to the fifth thermodynamic diagram from the leftmost side is 0.89;

the sixth thermodynamic diagram from the leftmost side corresponds to a significant value of 0.93.

According to the corresponding significance value of each thermodynamic diagram, the evaluation result of the image to be evaluated can be calculated, and the evaluation result of the image to be evaluated is 0.94 on the assumption that the evaluation result is represented by an average significance value.

According to the same manner, the evaluation result of the leftmost first image in fig. 3b was 0.53, and the evaluation result of the leftmost first image in fig. 3c was 0.01.

As can be seen from fig. 3a, since each feature of the face in the image to be evaluated in fig. 3a is visible and clear, the evaluation result is higher; as can be seen from fig. 3b, since part of the feature of the face in the image to be evaluated in fig. 3b is not visible, the evaluation result of the image to be evaluated in fig. 3b is lower than that of the image to be evaluated in fig. 3 a. As can be seen from fig. 3c, since the feature of the face is not included in the image to be evaluated in fig. 3c, the evaluation result of the image to be evaluated in fig. 3c is the lowest compared with the evaluation results of the images to be evaluated in fig. 3a and 3 b.

In one embodiment of the present invention, when obtaining, for each feature of the object in S101, a thermodynamic diagram reflecting the significance level of each pixel in the image to be evaluated for representing the feature, a pre-trained significance level detection model may be used to detect the significance level of each pixel in the image to be evaluated for representing each feature of the object, and obtain a thermodynamic diagram respectively reflecting the significance level of each pixel in the image to be evaluated for representing each feature.

Wherein, the significance detection model is as follows: and taking the sample image as a model input, taking the significance degree of each pixel point in the sample image belonging to each characteristic part as training supervision information, and training a preset neural network model to obtain a model which is used for obtaining the thermodynamic diagram corresponding to each characteristic part.

The significance degree of each pixel point belonging to each characteristic part in the sample image can be understood as follows: the probability that each pixel point in the sample image belongs to the region where each characteristic part is located in the image is high.

The above-mentioned training supervision information uses the significant degree that each pixel point belongs to each feature part in the sample image as training supervision information, and is used for using the significant degree that each pixel point belongs to each feature part in the sample image as training reference when detecting the significant degree that each pixel point belongs to each feature part in the sample image, so that the detection result and the training reference tend to be consistent.

Since the thermodynamic diagram is used for reflecting the significance degree of representing the characteristic part by each pixel point in the image to be evaluated, each characteristic part corresponds to one thermodynamic diagram. For example: if the feature parts include two feature parts of the left eye corner and the right eye corner of the face, then the thermodynamic diagram is an image reflecting the significance level of each pixel point in the image to be evaluated including the face for representing the left eye corner and an image reflecting the significance level of each pixel point in the image to be evaluated including the face for representing the right eye corner.

When the saliency detection model is trained, a large number of images can be collected in advance and used as sample images, the sample images are input into an initial model of the saliency detection model, a thermodynamic diagram reflecting the saliency of each pixel point in the sample images belonging to each characteristic part is obtained, the obtained thermodynamic diagram is compared with the training supervision information, and model parameters of the initial model are adjusted according to the difference between the thermodynamic diagram and the training supervision information, so that the initial model tends to be converged, and the saliency detection model is obtained.

In this way, the thermodynamic diagram corresponding to each characteristic part is obtained by adopting the pre-trained saliency detection model, so that the saliency degree of each pixel point in the image to be evaluated for representing each characteristic part of the object can be accurately determined.

In one embodiment of the present invention, when obtaining the thermodynamic diagrams using the above-mentioned saliency detection model, a plurality of thermodynamic diagrams need to be obtained, that is, the above-mentioned saliency detection model needs to output a plurality of thermodynamic diagrams. In order to clearly understand which feature of the object each thermodynamic diagram corresponds to, a sequence flag may be set in advance for each thermodynamic diagram. Thus, the saliency detection model can output thermodynamic diagrams corresponding to different sequence identifications, and further can learn characteristic parts of the thermodynamic diagrams corresponding to the objects according to the sequence identifications.

For example: when the characteristic parts of the object include the right eye corner, the left eye corner and the nose tip of the human face, the sequential marks preset for each thermodynamic diagram are as follows in sequence: the thermodynamic diagram of sequence 1 is the thermodynamic diagram corresponding to the right eye corner of the face, the thermodynamic diagram of sequence 2 is the thermodynamic diagram corresponding to the left eye corner of the face, and the thermodynamic diagram of sequence 3 is the thermodynamic diagram corresponding to the nose tip of the face. When the saliency detection model outputs thermodynamic diagrams of different order identifications, it is possible to determine to which feature of the object the thermodynamic diagram corresponds from the order identifications of the respective thermodynamic diagrams.

Thus, the meaning represented by each thermodynamic diagram can be accurately determined according to the preset sequence identification.

Corresponding to the image quality evaluation method, the embodiment of the invention also provides an image quality evaluation device.

Referring to fig. 4, fig. 4 is a schematic structural diagram of an image quality evaluation apparatus according to an embodiment of the present invention, where the apparatus includes 401 to 403.

A thermodynamic diagram obtaining module 401, configured to obtain a plurality of thermodynamic diagrams corresponding to an image to be evaluated, where each thermodynamic diagram is used to indicate a degree of saliency of a feature of an object in the image to be evaluated, and different thermodynamic diagrams are used for different feature of the object.

A saliency value obtaining module 402, configured to obtain a saliency value reflecting a saliency degree of each feature part included in the image to be evaluated according to a thermodynamic diagram corresponding to each feature part.

An evaluation result generation module 403, configured to generate an evaluation result for the image to be evaluated according to the obtained significant value, where the evaluation result is used to indicate the quality of the object in the image to be evaluated.

In one embodiment of the present invention, the thermodynamic diagram obtaining module 401 is specifically configured to:

In one embodiment of the present invention, the saliency value obtaining module 402 is specifically configured to:

In one embodiment of the present invention, the evaluation result generating module 403 is specifically configured to:

In one embodiment of the present invention, in the case that the object is a target face in the image to be evaluated, the thermodynamic diagram obtaining module 401 is specifically configured to: obtaining first to fifth thermodynamic diagrams, wherein the first thermodynamic diagram is used for indicating the saliency degree of the left eye of the target face in the image to be evaluated, the second thermodynamic diagram is used for indicating the saliency degree of the right eye of the target face in the image to be evaluated, the third thermodynamic diagram is used for indicating the saliency degree of the nose tip of the target face in the image to be evaluated, the fourth thermodynamic diagram is used for indicating the saliency degree of the left mouth corner of the target face in the image to be evaluated, and the fifth thermodynamic diagram is used for indicating the saliency degree of the right mouth corner of the target face in the image to be evaluated.

Corresponding to the image quality method, the embodiment of the invention also provides a terminal device.

Referring to fig. 5, fig. 5 is a schematic structural diagram of a terminal device according to an embodiment of the present invention, which includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 complete communication with each other through the communication bus 504,

A memory 503 for storing a computer program;

the processor 501 is configured to implement the image quality evaluation method provided by the embodiment of the present invention when executing the program stored in the memory 503.

The communication bus mentioned above for the electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal processor (DIGITAL SIGNAL Processing, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.

In still another embodiment of the present invention, there is also provided a computer-readable storage medium having stored therein a computer program which, when executed by a processor, implements the image quality evaluation method provided by the embodiment of the present invention.

In yet another embodiment of the present invention, a computer program product containing instructions that, when run on a computer, cause the computer to perform the image quality evaluation method provided by the embodiment of the present invention is also provided.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus, terminal device, computer readable storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An image quality evaluation method, characterized in that the method comprises:

Generating an evaluation result for the image to be evaluated according to the obtained significant value, wherein the evaluation result is used for indicating the quality of an object in the image to be evaluated;

the generating an evaluation result for the image to be evaluated according to the obtained significant value comprises the following steps:

2. The method according to claim 1, wherein the obtaining a plurality of thermodynamic diagrams corresponding to the image to be evaluated includes:

3. The method according to claim 1 or 2, wherein obtaining a saliency value reflecting a saliency degree of the image to be evaluated including the respective feature portions according to the thermodynamic diagram corresponding to each feature portion comprises:

4. The method according to claim 1 or 2, wherein obtaining a saliency value reflecting a saliency degree of the image to be evaluated including the respective feature portions according to the thermodynamic diagram corresponding to each feature portion comprises:

5. The method according to claim 1, wherein, in the case that the object is a target face in the image to be evaluated, the obtaining a plurality of thermodynamic diagrams corresponding to the image to be evaluated includes:

6. An image quality evaluation device, characterized in that the device comprises:

An evaluation result generation module, configured to generate an evaluation result for the image to be evaluated according to the obtained significant value, where the evaluation result is used to indicate quality of an object in the image to be evaluated;

The evaluation result generation module is specifically configured to calculate an average saliency value according to the obtained saliency value, and determine the calculated average saliency value as an evaluation result for the image to be evaluated.

7. The apparatus of claim 6, wherein the thermodynamic diagram obtaining module is configured to:

8. The terminal equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for carrying out the method steps of any one of claims 1-5 when executing a program stored on a memory.

9. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-5.