CN110807759A

CN110807759A - Method and device for evaluating photo quality, electronic equipment and readable storage medium

Info

Publication number: CN110807759A
Application number: CN201910870110.6A
Authority: CN
Inventors: 汤峰峰
Original assignee: Fantasy Power (shanghai) Culture Communication Co Ltd
Current assignee: Shiyou Beijing Technology Co ltd
Priority date: 2019-09-16
Filing date: 2019-09-16
Publication date: 2020-02-18
Anticipated expiration: 2039-09-16
Also published as: CN110807759B

Abstract

The invention provides a method and a device for evaluating photo quality, an electronic device and a readable storage medium, comprising the following steps: acquiring a photo to be evaluated; judging whether the photo is qualified from the following at least two dimensions: the overall imaging quality, the imaging quality of a shooting target area and the composition quality of a shooting target; when the photo is qualified, carrying out first-class quantitative evaluation on the photo according to a photographic aesthetic evaluation index to obtain a first-class quantitative evaluation result; and obtaining the comprehensive evaluation of the photo according to the quantitative evaluation result. The invention evaluates the quality of the photos from a more comprehensive dimension, thereby enabling the good photos selected by the equipment to be closer to the good photos selected by the photographer.

Description

Method and device for evaluating photo quality, electronic equipment and readable storage medium

Technical Field

The invention relates to the technical field of artificial intelligent photography, in particular to a method and a device for evaluating photo quality, electronic equipment and a readable storage medium.

Background

With the popularization of cameras, especially mobile phones with stronger and stronger photographing capability, people can take a lot of photos with the cameras or the mobile phones in daily life, but the generated photos have the problem of uneven levels due to the influence of imaging environments and imaging time conditions. If the good photos are picked out manually, the whole process is tedious and uninteresting, and the experience of repeated operation is very poor. Compared with data such as photos, mobile devices such as mobile phones have limited storage space, and reduction of unnecessary data storage is beneficial to overall performance maintenance. Therefore, it is possible to improve the above-mentioned problems by allowing a user to evaluate a photograph by image analysis, such as a photographing apparatus or a photograph management apparatus, to leave or give priority to a better photograph, and to delete or leave a less-good photograph.

But commonly used photo evaluation methods are based on some simple rules such as whether a person has a smiling face, whether a photographic subject (i.e., a subject) is in an experienced aesthetic position (e.g., a quartile, a centered position, etc.), whether the subject is appropriately sized, and the like. This method of photo evaluation is still far from the level of a photographer, resulting in a less than satisfactory selection of photos.

Disclosure of Invention

One of the objectives of the present invention is to overcome at least some of the deficiencies in the prior art, and provide a method and an apparatus for evaluating quality of photos, an electronic device, and a readable storage medium, so that evaluation dimensions are more comprehensive, and evaluation efficiency is higher, and therefore, a good photo selected by a device is closer to a good photo selected by a photographer.

The technical scheme provided by the invention is as follows:

a method of evaluating picture quality, comprising: acquiring a photo to be evaluated; judging whether the photo is qualified from the following at least two dimensions: the overall imaging quality, the imaging quality of a shooting target area and the composition quality of a shooting target; when the photo is qualified, carrying out first-class quantitative evaluation on the photo according to a photographic aesthetic evaluation index to obtain a first-class quantitative evaluation result; and obtaining the comprehensive evaluation of the photo according to the quantitative evaluation result.

Further, the determining whether the photo is qualified from at least two of the following dimensions, thereafter, further includes: when the photo is qualified, performing second-class quantitative evaluation according to the maximum similarity of the photo and a good photo database to obtain a second-class quantitative evaluation result; the obtaining of the comprehensive evaluation of the photo according to the quantitative evaluation result of the type includes: and obtaining the comprehensive evaluation of the photo according to the first class quantitative evaluation result and the second class quantitative evaluation result.

Further, obtaining the comprehensive evaluation of the photograph includes: judging whether the photo is a good photo or not according to the comprehensive evaluation; and when the photo is a good photo, adding the good photo into the good photo database.

Further, the determining whether the photo is a good photo according to the comprehensive evaluation includes: and judging whether the picture is a good picture or not according to the comprehensive evaluation and the evaluation of a photographer.

Further, the determining whether the photo is qualified from at least two of the following dimension combinations comprises: judging whether the integral imaging quality of the picture is qualified or not; when the overall imaging quality of the picture is qualified, extracting a shooting target area of the picture; judging whether the imaging quality of the shooting target area is qualified or not; when the imaging quality of the shooting target area is qualified, judging whether the composition quality of the shooting target is qualified; and when the composition quality of the shooting target is qualified, the picture is qualified.

Further, the extracting of the shooting target area of the photo includes: identifying a photo type of the photo; and selecting a corresponding target detection and segmentation algorithm according to the type of the picture, detecting a shooting target of the picture, and extracting a corresponding shooting target area.

Further, the identifying the photo type of the photo includes: identifying the photo type of the photo according to the overall abstract features of the photo extracted by the convolutional neural network; the selecting of the corresponding target detection and segmentation algorithm according to the photo type comprises: when the photo type is a landscape photo, using an image salient region detection algorithm; and when the photo type is a person photo, using a human body detection and segmentation algorithm.

Further, the determining whether the composition quality of the shooting target is qualified includes: and judging whether the composition of the shooting target accords with a preset composition rule, wherein the factors for evaluating the preset composition rule comprise the size, the position and the changeable state of the shooting target.

Further, the performing of the second-class quantitative evaluation according to the maximum similarity between the photo and the good photo database to obtain a second-class quantitative evaluation result includes: according to a similarity evaluation model based on a deep neural network, calculating the similarity between the photo and each photo in a good photo database; obtaining the maximum similarity of the photos according to the similarity calculation result; and obtaining a second class quantitative evaluation result of the photo according to the maximum similarity.

Further, the calculating the similarity between the photo and each photo in the good photo database includes: respectively extracting feature vectors of the photo and a comparison photo based on a deep neural network, wherein the comparison photo is a photo in a good photo database; calculating the Euclidean distance between the two feature vectors according to the feature vectors of the photos and the feature vectors of the comparison photos; and obtaining the similarity between the two photos according to the Euclidean distance between the two feature vectors.

Further, the obtaining of the comprehensive evaluation of the photo according to the first-class quantitative evaluation result and the second-class quantitative evaluation result includes: weighting and summing the first class quantitative evaluation result and the second class quantitative evaluation result, and taking the result after weighting and summing as the comprehensive evaluation of the photo; and the weight of the two types of quantitative evaluation results is dynamically increased along with the increase of the number of the good photos in the good photo database until the maximum preset weight is reached.

Further, the weight of the two types of quantitative evaluation results dynamically increases with the number of good photos in the good photo database until the maximum preset weight is reached includes: when the number of good photos in the good photo database is less than or equal to a first threshold value, the weight of the second-class quantitative evaluation result is a preset initial value; when the number of good photos in the good photo database is greater than or equal to a second threshold value, the weight of the second type of quantitative evaluation result is the maximum preset weight; when the number of good photos in the good photo database is between the first threshold and the second threshold, the weight of the two types of quantitative evaluation results is dynamically increased along with the increase of the number of the good photos.

The present invention also provides an apparatus for evaluating picture quality, comprising: the photo obtaining module is used for obtaining a photo to be evaluated; the qualification judging module is used for judging whether the photo is qualified or not from the following at least two dimensionalities: the overall imaging quality, the imaging quality of a shooting target area and the composition quality of a shooting target; the first-class evaluation module is used for carrying out first-class quantitative evaluation on the photos according to the photo aesthetic evaluation index when the photos are qualified to obtain first-class quantitative evaluation results; and the comprehensive evaluation module is used for obtaining the comprehensive evaluation of the photo according to the quantitative evaluation result.

Further, the method also comprises the following steps: the second-class evaluation module is used for performing second-class quantitative evaluation according to the maximum similarity between the photo and the good photo database when the photo is qualified to obtain a second-class quantitative evaluation result; and the comprehensive evaluation module is further used for obtaining the comprehensive evaluation of the photo according to the first class quantitative evaluation result and the second class quantitative evaluation result.

Further, the method also comprises the following steps: the database updating module is used for judging whether the photo is a good photo or not according to the comprehensive evaluation; and when the photo is a good photo, adding the good photo into the good photo database.

Further, the database updating module is further configured to determine whether the picture is a good picture according to the comprehensive evaluation and the evaluation of the photographer.

Further, the qualification judging module includes: the integral imaging judging unit is used for judging whether the integral imaging quality of the photo is qualified or not; the target area extracting unit is used for extracting a shooting target area of the picture when the overall imaging quality of the picture is qualified; the local imaging judging unit is used for judging whether the imaging quality of the shooting target area is qualified or not; the composition quality judging unit is used for judging whether the composition quality of the shooting target is qualified or not when the imaging quality of the shooting target area is qualified; the qualification judgment module is further used for judging that the picture is qualified when the composition quality of the shooting target is qualified.

Further, the target area extracting unit is further configured to identify a photo type of the photo; and selecting a corresponding target detection and segmentation algorithm according to the type of the picture, detecting a shooting target of the picture, and extracting a corresponding shooting target area.

Further, the target area extracting unit is further configured to identify a photo type of the photo according to the overall abstract feature of the photo extracted by the convolutional neural network; and when the photo type is landscape photo, using an image salient region detection algorithm; and when the photo type is a person photo, using a human body detection and segmentation algorithm.

Further, the composition quality determination unit is further configured to determine whether the composition of the shooting target meets a preset composition rule, and the factors for evaluating the preset composition rule include a size, a position, and a changeable state of the shooting target.

Further, the two types of evaluation modules are further used for calculating the similarity between the photo and each photo in the good photo database according to a similarity evaluation model based on a deep neural network; obtaining the maximum similarity of the photos according to the similarity calculation result; and obtaining a second class quantitative evaluation result of the photo according to the maximum similarity.

Further, the comprehensive evaluation module is further configured to perform weighted summation on the first-class quantitative evaluation result and the second-class quantitative evaluation result, and use the result after weighted summation as the comprehensive evaluation of the photo; and the weight of the two types of quantitative evaluation results is dynamically increased along with the increase of the number of the good photos in the good photo database until the maximum preset weight is reached.

The present invention also provides an electronic device comprising: a memory for storing a computer program; a processor for running the computer program to implement the method for evaluating the quality of a photograph according to any one of the preceding claims.

The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of evaluating the quality of a photograph as set forth in any one of the preceding claims.

The method and the device for evaluating the photo quality, the electronic equipment and the readable storage medium provided by the invention can at least bring the following beneficial effects:

1. according to the invention, unqualified photos are filtered out, and qualified photos are quantitatively evaluated, so that the processing efficiency can be improved when a large number of photos are processed, and the photos can be rapidly selected; when whether the photo is qualified or not is evaluated, the overall imaging quality, the imaging quality of the shooting target area, the composition quality of the shooting target and other dimensions are judged respectively, and the evaluation dimensions are more comprehensive.

2. When the qualified photos are quantitatively evaluated, comprehensive evaluation is carried out on traditional photo aesthetic evaluation indexes and experience accumulated by a good photo database, evaluation dimensionality is more comprehensive, and photo evaluation capability of equipment is further improved, so that the good photos selected by the equipment are closer to the good photos selected by a photographer.

3. The invention provides a set of continuously evolving photo quality evaluation method, which assists the evaluation of the photo evaluation capability of correction equipment by means of the evaluation of a photographer at the initial shooting stage when the number of photos is small; with the more photos taken, the weight of experience evaluation brought by the photo database is gradually increased, the stronger the photo evaluation capability of the equipment is, and the evaluation effect can finally reach the level of a common photographer, so that the photographer can be liberated from heavy photo selection, and meanwhile, the public can conveniently enjoy photo selection results similar to the photographer.

Drawings

The above features, technical features, advantages and implementations of a method and apparatus for evaluating picture quality, an electronic device, and a readable storage medium will be further described in detail below with reference to the accompanying drawings.

FIG. 1 is a flow chart of one embodiment of a method of evaluating picture quality of the present invention;

FIG. 2 is a flow chart of another embodiment of a method of evaluating picture quality of the present invention;

FIG. 3 is a flow chart of another embodiment of a method of evaluating picture quality of the present invention;

FIG. 4 is a schematic structural diagram of an embodiment of an apparatus for evaluating a picture quality according to the present invention;

FIG. 5 is a schematic structural diagram of another embodiment of the apparatus for evaluating a picture quality according to the present invention;

FIG. 6 is a schematic structural diagram of another embodiment of the apparatus for evaluating a picture quality according to the present invention;

FIG. 7 is an example in which the photographing target area position in FIG. 3 is shifted too much from the composition aesthetic experience position;

fig. 8 is an example in which the size of the photographic subject in fig. 3 deviates too much from the empirical value;

fig. 9 is an example of a variable state difference of the photographic subject in fig. 3;

FIG. 10 is a schematic diagram of the qualification module of FIG. 6;

fig. 11 is a schematic structural diagram of an embodiment of an electronic device of the present invention.

The reference numbers illustrate:

100. the device comprises a photo quality evaluation device, a photo acquisition module, a qualified judgment module, a first-class evaluation module, a second-class evaluation module, a comprehensive evaluation module, a database updating module, a global imaging judgment unit, a target area extraction unit, a local imaging judgment unit, a composition quality judgment unit, a picture quality.

Detailed Description

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will be made with reference to the accompanying drawings.

It is understood that the shooting target may be a child or an adult, one or more other persons, other animals, plants, landscapes, buildings, or the like, or any combination of the foregoing objects, as desired.

In addition, "upper, lower", "left, right", "front, rear", "one, another", and the like appearing in the present invention are relative concepts unless otherwise specified. In addition, the terms "first type", "second type", and the like, as used herein, are used for descriptive convenience only and are not to be construed as indicating or implying relative importance or explicitly defining a sequential order.

In one embodiment of the present invention, as shown in fig. 1, a method for evaluating a picture quality includes:

step S100 acquires a photograph to be evaluated.

Step S200 judges whether the photo is qualified from at least two of the following dimensional combinations: overall imaging quality, imaging quality of the shooting target area, and composition quality of the shooting target.

Specifically, a photograph must first be an image that is easily interpreted by a person, and if there are factors that interfere with the person's interpretation of the image, the photograph is not good, nor is it likely to be a good photograph. Therefore, in evaluating the quality of a picture, it is first determined whether the quality of the picture is acceptable. The method can judge one or more optional dimensions from the integral imaging quality, the imaging quality of a shooting target area and the composition quality of a shooting target according to the application scene requirement; when the requirements of all selected dimensions are met, then the picture is qualified in quality. For example, in a photo studio, the first two items can be basically guaranteed, but the expression state of a person is variable and is not easy to control, so that the judgment can be carried out from the dimension of composition quality; the pictures taken in the scene with more characters and uncertain actions of the characters can be comprehensively judged from the three dimensions.

The integral imaging quality is judged according to the basic quality of the image of the whole photo by analyzing whether the photo can be accepted or not on the whole. The basic quality is qualified, namely the image has no obstacle which prevents human eyes from extracting information, and generally relates to correct exposure and correct focusing without overexposure, over darkness, blurring and the like. Overexposure, dullness and blurring of the picture tend to make it impossible for a person to discern details of the object in the image, so the appearance of the above phenomena also indicates that the base quality is not good. And if the basic quality of the whole image is not qualified, the whole imaging quality is not qualified.

The detection of photo overexposure, darkness and blur can be performed using well known algorithms such as image processing and analysis. Optionally, the mean and variance of the brightness of the whole photo image are counted, and whether the photo is over-exposed or over-dark is judged according to the mean and variance of the brightness. The luminance mean and variance can be calculated from all pixel values of the entire photographic image. When the mean is above a large threshold (e.g., 230) and the variance is within a certain range (e.g., 30), it can be confirmed that the photo is overexposed as a whole; when the mean is below another small threshold (e.g., 60) and the variance is within a certain range (e.g., 30), it can be confirmed that the photo as a whole is too dark exposed. And (4) counting the average value of the gradient amplitude information of the whole picture image, and judging whether the picture is fuzzy or not according to the value, for example, when the value is less than a threshold (for example, 5), the picture is considered to be fuzzy. The gradient vector can be obtained by a sobel operator, a robert operator or a prewitt operator, and the gradient magnitude information is the magnitude of the gradient vector.

The threshold values reflect the empirical range of a normal image, and if one of the statistical terms exceeds the empirical range, the image is abnormal in imaging, and the picture is unqualified and should be filtered.

The imaging quality of the shooting target area is judged according to the basic quality of the image of the shooting target area, and generally relates to correct exposure of the shooting target, good focusing of the shooting target, reasonable color tone of the shooting target and the like. The basic quality evaluation method is the same as the method introduced above, namely the brightness mean value and the variance of the image of the shooting target area can be counted, and whether the exposure of the shooting target area is excessive or too dark is judged; and (4) counting the average value of the gradient amplitude information of the image of the shooting target area, and judging whether the image of the shooting target area is fuzzy.

The region of interest of the picture can be extracted using well-known methods of general object detection segmentation (e.g., Mask RCNN example segmentation algorithm), or image saliency region detection (e.g., Qibin Hou, Ming-MingCheng, Xiaoaeei Hu, Ali Borji, Zhuowen Tu, Philip Torr, deep Suspended stereoscopic object detection with short connections, IEEE TPAMI, 2018.). Different methods have different application ranges, for example, when processing a portrait photograph, the method using human detection and segmentation is very effective, and the method using the landscape type photograph salient region detection is simpler and more effective. Preferably, the photo type of the photo is identified; and selecting a corresponding target detection and segmentation algorithm according to the type of the picture, detecting a shooting target of the picture, and extracting a corresponding shooting target area. By photo type is meant a group of photos that are semantically similar and clearly distinct from other photos. The photo types can be determined according to different grouping requirements, and can be distinguished by extracting semantics from different angles, for example, the photo types can be divided into a character photo and a scene photo, can also be divided into a personal photo, a building scene photo and a collective photo, and can also be divided into an indoor photo and an outdoor photo. For example, when the photo is recognized as a person photo, a photographing target region is extracted using a human body detection and segmentation method. When the photo is recognized as a landscape photo, the shooting target area is extracted by using an image salient area detection method.

Since images of different scenes (such as an indoor playground image and a natural scenic spot image) generally have obvious difference on the whole, and the convolutional neural network can extract overall abstract features of the images and can classify the images by using the features, the recognition and classification (corresponding to the type of the picture) of the photo scene can be realized by using the classified convolutional neural network (such as VGG and the like), and then the optimal shooting target area detection method is used according to the recognized type of the picture to obtain the shooting target area.

The quality of the composition of the photographic subject is to detect whether the composition of the photographic subject is reasonable from the composition angle, and generally mainly relates to the reasonable size of the photographic subject in the picture, the reasonable position of the photographic subject, the reasonable variable state of the photographic subject, such as whether the photographic subject is too small or too large, whether the photographic subject is too close to the side, whether the facial expression is proper (such as whether the eyes are closed or white), and the like.

And step S300, when the photo is qualified, performing one-class quantitative evaluation on the photo according to the photo aesthetic evaluation index to obtain one-class quantitative evaluation result.

And step S500, obtaining the comprehensive evaluation of the photo according to the quantitative evaluation result.

Specifically, after the photo is evaluated to be qualified, the photo is further scored according to a traditional evaluation sub-item in the photo aesthetic evaluation, namely a photo aesthetic evaluation index.

The photographic aesthetic evaluation index is a variety of image features extracted from the underlying visual information of the photograph according to conventional photographic aesthetic evaluation experience. Such as the saliency of the shot object, the simplicity of the background color, the harmony of the shot object and the background color, the visual balance of the distribution of the subject, etc., and the sub-items are scored item by item, and the scoring method can adopt the disclosed method (refer to Gallea, robertoandrdizizone, edoardodand Pirrone, roberto. automatic aesthetical photo composition. ici ap 2013,1: 21-30.). And finally, integrating the scores of all the sub-items to obtain a class of quantitative evaluation results.

And taking the quantitative evaluation result as the comprehensive evaluation of the photo. When the comprehensive evaluation exceeds a certain threshold value, the picture is considered to be good.

In the embodiment, unqualified photos (i.e. photos with low obvious quality) are filtered, and then specific photo scoring is completed for the qualified photos, so that the processing efficiency is higher when a large number of photos are processed, and the photos can be quickly selected; when whether the photo is qualified or not is evaluated, the overall imaging quality, the imaging quality of the shooting target area, the composition quality of the shooting target and other dimensions are judged respectively, and the evaluation dimensions are more comprehensive.

In another embodiment of the present invention, as shown in fig. 2, a method for evaluating a picture quality includes:

compared with the method for evaluating the photo quality illustrated in the foregoing fig. 1, in order to further improve the photo evaluation capability of the device, step S400 is added, and step S510 replaces step S500:

step S400, when the photo is qualified, performing second-class quantitative evaluation according to the maximum similarity of the photo and a good photo database to obtain a second-class quantitative evaluation result;

step S510 obtains a comprehensive evaluation of the photograph according to the first-class quantitative evaluation result and the second-class quantitative evaluation result.

Specifically, after the photos are evaluated to be qualified, the photos can be scored according to the accumulated experience of good photos, namely, the photos can be evaluated in a two-class quantitative manner. The execution sequence of the first-class quantitative evaluation and the second-class quantitative evaluation is not limited, and the first-class quantitative evaluation and the second-class quantitative evaluation can be performed simultaneously or in any sequence.

Generally, in a fixed scene, the taken pictures are often in a limited mode. Therefore, preferably, when the good photos are increased to form a database in the scene, the good photo database actually accumulates an experience mode library, and in such a scene, the similarity between the newly-taken photo and each good photo in the library can be obtained, and the maximum value of the similarity is used as another evaluation item for the good photo and the bad photo, namely the second type quantitative evaluation result s 2. And (4) integrating the first-class quantitative evaluation result s1 and the second-class quantitative evaluation result s2 to obtain a final photo quality score s _ final. When s _ final exceeds a certain threshold, then a good photo is considered.

In the embodiment, when the quality of the qualified photos is evaluated, comprehensive evaluation is performed from the traditional photo aesthetic evaluation index and the experience accumulated by the good photo database, so that the evaluation dimensionality is more comprehensive, the photo evaluation capability of the equipment is further improved, and the good photos selected by the equipment are closer to the good photos selected by the photographer.

In another embodiment of the present invention, as shown in fig. 3, a method for evaluating a picture quality includes:

step S100 acquires a photograph to be evaluated.

Step S210 determines whether the overall imaging quality of the photograph is acceptable.

And step S220, when the overall imaging quality of the picture is qualified, extracting a shooting target area of the picture.

Step S230 determines whether the imaging quality of the shooting target area is qualified.

Step S240 determines whether the composition quality of the photographic target is acceptable when the imaging quality of the photographic target area is acceptable.

And step S250, when the composition quality of the shooting target is qualified, the picture is qualified.

Specifically, when the quality of the photo is judged to be qualified, the integral imaging quality of the photo is judged first, and whether the photo can be accepted is analyzed on the whole; if not, the photo evaluation is ended.

When the overall analysis of the picture is acceptable, the image quality of the subject region as the shooting target does not necessarily satisfy the requirement, so that the shooting target region (i.e., the subject region) of the image needs to be extracted and the picture quality of the subject region needs to be analyzed; if not, the photo evaluation is ended.

And when the local image quality condition of the shooting target area is met, judging the composition state by utilizing the detection positioning and segmentation information of the shooting target. In addition to the need for normal imaging, a good picture is also checked for quality from the perspective of the composition, for example, the location of the area of the photographic target cannot be shifted too much from the aesthetic experience of the composition, the size of the photographic target cannot be shifted too much from the experience, and the changeable state of the photographic target cannot be too bad. For example, as shown in fig. 7, the person is too much, and specifically, the center of the outer surrounding frame where the shooting target is located is deviated from the central area of the photograph; as shown in fig. 8, the left-image person is too large, specifically, the face width of the shooting target is larger than a specified number of pixels (e.g., 800), the right-image person is too small, and the face width of the shooting target is smaller than a specified number of pixels (e.g., 40); as shown in fig. 9, the human expression is poor, the human expression is a changeable state which needs to be checked, the expression of the human face can be detected by performing classification recognition processing on the face frame region by using a known classification convolutional neural network (for example, VGG, etc.), the shooting target expression should not be ugly (for example, white eye, red eye, closed eye, pain, etc. are regarded as ugly expressions), otherwise, the photo cannot be a good photo.

And when the photos are judged to be qualified from the dimensions of the integral imaging quality, the imaging quality of the shooting target area and the composition quality of the shooting target in sequence, the photos are considered to be qualified. When a large number of photos to be evaluated exist, the evaluation efficiency can be improved by the gradual progressive evaluation method.

And S310, when the composition quality of the shooting target is qualified, performing one-class quantitative evaluation on the picture according to the shooting aesthetic evaluation index to obtain one-class quantitative evaluation result.

Specifically, the previous steps are based on regular progressive evaluation, a certain poor photo can be judged as a poor photo, the basis of the good photo is only obtained through all condition evaluation, and further, some traditional methods in photographic aesthetic evaluation are used for scoring each photo according to sub-items (such as whether a shooting target is prominent, whether a background is concise, whether the shooting target and background colors are harmonious, whether the main body distribution is balanced visually and the like), and finally, an evaluation score capable of reflecting the good-poor ordering relation is obtained, namely, a quantitative evaluation result s1, such as s 1E [0,1], and the quality is better when the s1 is larger.

And step S410, performing second-class quantitative evaluation according to the maximum similarity of the photos and the good photo database to obtain second-class quantitative evaluation results.

Step S511, carrying out weighted summation on the first-class quantitative evaluation result and the second-class quantitative evaluation result, and taking the result after weighted summation as the comprehensive evaluation of the photo; and the weight of the two types of quantitative evaluation results is dynamically increased along with the increase of the number of the good photos in the good photo database until the maximum preset weight is reached.

And step S600, judging whether the photo is a good photo or not according to the comprehensive evaluation of the photo.

Step S700 adds the good photo to the good photo database.

Specifically, when the photographs are evaluated as acceptable, they can also be scored based on accumulated experience with good photographs. Generally, in a fixed scene, the taken pictures are often in a limited mode (taking indoor dance scene photography as an example, the picture mode can be divided into a 1-person mode, a 2-person mode, a 3-person mode and the like, and can be classified according to people), and the accumulated good picture database actually provides an experience mode library. The similarity degree between the photos is measured by using a module such as a deep neural network, the similarity calculation can be carried out on the current photo and each good photo in the database under the premise that the operation scene is not changed, and the maximum value of the similarity is taken as another evaluation item for the good photo and the bad photo, namely the second-class quantitative evaluation result s2, and the greater the probability that the good photo is represented is.

The deep neural network for measuring similarity can use a network framework such as faceNet face recognition, and is different from the original faceNet in that training data is not a face image of a different person but an image picture of different modes in a fixed scene, and the similarity s between the two images_ij＝exp(-||f(x⁽ⁱ⁾),f(x^(j))||²) Wherein exp is an exponential function with a natural constant e as the base, | |²The Euclidean distance between two points is obtained, and f (eta) is a corresponding feature vector output by the deep neural network to the input photo image.

The sum of s2 and s1 is weighted to obtain a final photo quality score s _ final, for example, s _ final (1-w) s1+ w s2, and the weight w is 0.5 at maximum. s _ final needs to exceed a certain threshold to be considered a good photo, e.g. take a threshold of 0.6. Since the types of good photos within a scene are always limited, the evaluation of good photos is easier and more accurate as the system runs longer and the more good photos are accumulated.

In the initial stage, the good photo database needs to be completed by human assistance: in the initial stage, w is 0, s _ final is s1, s2 is unreliable, so that the photos are not validated to be subjected to final quality scoring, only s1 partial scoring is used for carrying out primary screening on the good photos, all photos which exceed a good photo threshold value and are obtained after the camera is operated are placed into an initial good photo database, a photographer manually selects and screens the photos to remove misjudgment, the manually confirmed good photos are placed into the good photo database, w begins to be validated when the number of the good photos exceeds a certain threshold value t1 (for example 3000), and when the number of the good photos exceeds t2 (for example 10000), w reaches an extreme value of 0.5, specifically, w is 0.5 min (max ((N-t1)/(t2-t1),0),1), and N is the number of the manually confirmed good photos. When the number of good photos in the good photo database is large enough (e.g., a certain threshold is reached), the process of manually selecting the screening can be omitted, and the automatic machine scoring is excellent enough.

This embodiment provides a set of photo evaluation method that constantly evolves, and along with the photo that shoots more, the ability of photo evaluation is stronger, and the effect of evaluation can reach ordinary photographer's level finally to can let the photographer select from heavy photo and liberate, also make masses all can be convenient enjoy the photo of similar photographer and select the result simultaneously.

In one embodiment of the present invention, as shown in fig. 4, an apparatus 100 for evaluating a picture quality includes:

and a photo obtaining module 110, configured to obtain a photo to be evaluated.

A qualification module 120, configured to judge whether the photo is qualified from at least two dimensions: overall imaging quality, imaging quality of the shooting target area, and composition quality of the shooting target.

Specifically, when evaluating the picture quality, it is first determined whether the picture quality is acceptable. The judgment can be carried out in any dimension or several dimensions from the integral imaging quality, the imaging quality of the shooting target area and the composition quality of the shooting target according to the requirement; when the requirements of all selected dimensions are met, then the picture is qualified in quality.

The detection of photo overexposure, darkness and blur can be performed using well known algorithms such as image processing and analysis. Optionally, the mean and variance of the brightness of the whole photo image are counted, and whether the photo is over-exposed or over-dark is judged according to the mean and variance of the brightness. And (5) counting the average value of the gradient amplitude information of the whole picture image, and judging whether the picture is fuzzy or not according to the value.

The shot target area of the picture can be extracted by using a known general target detection segmentation method (such as Mask RCNN example segmentation algorithm), and a known image salient area detection method can also be adopted. Different methods have different application ranges, for example, when processing a portrait photograph, the method using human detection and segmentation is very effective, and the method using the landscape type photograph salient region detection is simpler and more effective. Preferably, the photo type of the photo is identified; and selecting a corresponding target detection and segmentation algorithm according to the type of the picture, detecting a shooting target of the picture, and extracting a corresponding shooting target area. By photo type is meant a group of photos that are semantically similar and clearly distinct from other photos.

The picture composition quality of the shooting target is to detect whether the picture composition of the shooting target is reasonable from the picture composition angle, and generally relates to reasonable size of the shooting target in a picture, reasonable position of the shooting target and reasonable changeable state of the shooting target.

And the first-class evaluation module 130 is used for performing first-class quantitative evaluation on the photos according to the photo aesthetic evaluation index when the photos are qualified, so as to obtain a first-class quantitative evaluation result.

Specifically, when the photos are evaluated to be qualified, the photos need to be scored for specific quality. According to the aesthetic evaluation indexes of photography, such as whether a shooting target (namely a subject) is prominent, whether a background is concise, whether colors of the shooting target and the background are harmonious, whether the subject distribution is balanced visually and other angles, the traditional sub-items are graded item by item, and finally the grades of the sub-items are integrated to obtain a quantitative evaluation result s 1.

And the comprehensive evaluation module 150 is used for obtaining comprehensive evaluation of the photo according to the quantitative evaluation result.

The photographic aesthetic evaluation index is a variety of image features extracted from the underlying visual information of the photograph according to conventional photographic aesthetic evaluation experience. Such as the saliency of a shooting target, the simplicity of background colors, the harmony of the shooting target and the background colors, the visual balance of main body distribution and the like, the sub-items are graded item by item, and the grading method can adopt a disclosed method; and finally, integrating the scores of all the sub-items to obtain a class of quantitative evaluation results. And taking the quantitative evaluation result as the comprehensive evaluation of the photo. When the comprehensive evaluation exceeds a certain threshold value, the picture is considered to be good.

In another embodiment of the present invention, as shown in fig. 5, an apparatus 100 for evaluating a picture quality includes:

on the basis of the evaluation device for photo quality illustrated in fig. 4, in order to further improve the photo evaluation capability of the device, a second-type evaluation module 140 is added.

And the second-class evaluation module 140 is configured to perform second-class quantitative evaluation according to the maximum similarity between the photo and the good photo database when the photo is qualified, so as to obtain a second-class quantitative evaluation result.

The comprehensive evaluation module 150 is further configured to obtain a comprehensive evaluation of the photo according to the first-class quantitative evaluation result and the second-class quantitative evaluation result.

In an embodiment of the present invention, as shown in fig. 6 and 10, an apparatus 100 for evaluating a picture quality includes:

and a photo obtaining module 110, configured to obtain a photo to be evaluated.

The qualification module 120 includes:

an integral imaging judgment unit 121, configured to judge whether the integral imaging quality of the photo is qualified;

a target area extracting unit 122, configured to extract a shooting target area of the photo when the overall imaging quality of the photo is qualified;

a local imaging judgment unit 123, configured to judge whether the imaging quality of the shooting target area is qualified;

and a composition quality judgment unit 124 for judging whether the composition quality of the photographic target is qualified or not when the imaging quality of the photographic target area is qualified.

The qualified judgment module 120 is further configured to judge that the picture is qualified when the composition quality of the shooting target is qualified.

And when the local image quality condition of the shooting target area is met, judging the composition state by utilizing the detection positioning and segmentation information of the shooting target. In addition to the need for normal imaging, a good picture is also required to be checked for quality from the perspective of the composition, for example, the position of the region of the photographic subject cannot be shifted too much from the aesthetic experience of the composition, the size of the photographic subject cannot be shifted too much from the experience, and the changeable state of the photographic subject is poor.

And when the photos are judged to be qualified from the dimensions of the integral imaging quality, the imaging quality of the shooting target area and the composition quality of the shooting target in sequence, the photos are considered to be qualified. When a large number of photographs to be evaluated exist, the evaluation efficiency can be improved by the above method.

And the first-class evaluation module 130 is used for performing first-class quantitative evaluation on the photo according to the photo aesthetic evaluation index to obtain a first-class quantitative evaluation result when the composition quality of the shooting target is qualified.

And the second-class evaluation module 140 is configured to perform second-class quantitative evaluation according to the maximum similarity between the photo and the good photo database to obtain a second-class quantitative evaluation result.

The comprehensive evaluation module 150 is configured to perform weighted summation on the first-class quantitative evaluation result and the second-class quantitative evaluation result, and use a result after the weighted summation as the comprehensive evaluation of the photo; and the weight of the two types of quantitative evaluation results is dynamically increased along with the increase of the number of the good photos in the good photo database until the maximum preset weight is reached.

The database updating module 160 is configured to determine whether the photo is a good photo according to the comprehensive evaluation; and when the photo is a good photo, adding the good photo into the good photo database.

The deep neural network for measuring the similarity can be used for reference of a network framework such as FaceNet face recognition.

The sum of s2 and s1 is weighted to obtain a final photo quality score s _ final, for example, s _ final (1-w) s1+ w s2, and the weight w is 0.5 at maximum. s _ final needs to exceed a certain threshold to be considered a good photo, e.g. take a threshold of 0.6. Since the types of good photos within a scene are always limited, the evaluation of good photos is easier and more accurate as the system runs longer and the more good photos are accumulated. In the initial stage, the good photo database needs to be completed by human assistance: in the initial stage, w is 0, s _ final is s1, s2 is unreliable, so that the photos are not validated to be subjected to final quality scoring, only s1 partial scoring is used for carrying out primary screening on the good photos, all photos which exceed a good photo threshold value and are obtained after the camera is operated are placed into an initial good photo database, a photographer manually selects and screens the photos to remove misjudgment, the manually confirmed good photos are placed into the good photo database, w begins to be validated when the number of the good photos exceeds a certain threshold value t1 (for example 3000), and when the number of the good photos exceeds t2 (for example 10000), w reaches an extreme value of 0.5, specifically, w is 0.5 min (max ((N-t1)/(t2-t1),0),1), and N is the number of the manually confirmed good photos. When the number of good photos in the good photo database is large enough (e.g., a certain threshold is reached), the process of manually selecting the screening can be omitted, and the automatic machine scoring is excellent enough.

This embodiment provides a set of photo evaluation device that constantly evolves, and the photo along with the picture of shooing is more, and the ability of photo evaluation is stronger, and the effect of evaluation can reach ordinary photographer level finally to can let the photographer select from heavy photo and liberate, also make the result is selected to popular enjoyment similar photographer's photo simultaneously.

The embodiment of the apparatus for evaluating the quality of photographs according to the present invention and the embodiment of the method for evaluating the quality of photographs according to the present invention are based on the same inventive concept, and can achieve the same technical effects. Therefore, other specific contents of the embodiment of the apparatus for evaluating picture quality can refer to the description of the embodiment of the method for evaluating picture quality.

In one embodiment of the invention, as shown in FIG. 11, an electronic device 400 includes a memory 410 and a processor 420. The memory 410 is used to store a computer program 430. The processor implements the method for evaluating the quality of a photograph as described above when running the computer program.

As an example, the processor 420 realizes the steps S100 to S500 according to the foregoing description when executing the computer program. The processor 420 realizes the functions of the modules and units in the evaluation apparatus 100 for picture quality described above when executing the computer program. As yet another example, the processor 420, when executing the computer program, implements the functions of the photograph acquisition module 110, the eligibility determination module 120, the one-class evaluation module 130, and the comprehensive evaluation module 150.

Alternatively, the computer program may be divided into one or more modules/units according to the particular needs to accomplish the invention. Each module/unit may be a series of computer program instruction segments capable of performing a particular function. The computer program instruction segment is used to describe the execution process of the computer program in the evaluation apparatus 100 for picture quality. As an example, the computer program may be divided into modules/units in a virtual device, such as a photo acquisition module, a qualification module, a class evaluation module, a comprehensive evaluation module.

The processor is used for realizing the evaluation method of the photo quality by executing the computer program. The processor may be a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), general purpose processor or other logic device, etc., as desired.

The memory may be any internal storage unit and/or external storage device capable of implementing data, program storage. For example, the memory may be a plug-in hard disk, a smart card (SMC), a Secure Digital (SD) card, or a flash card. The memory is used to store a computer program, and other programs and data of the evaluation apparatus 100 for picture quality.

The electronic device 400 may be any computer device, such as a desktop computer (desktop), a laptop computer (laptop), a Personal Digital Assistant (PDA), or a server (server). The electronic device 400 may further include an input/output device, a display device, a network access device, a bus 440, and the like, as needed. The electronic device 400 may also be a single chip computer or a computing device integrating a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU).

It will be understood by those skilled in the art that the above-mentioned units and modules for implementing the corresponding functions are divided for the purpose of convenient illustration and description, and the above-mentioned units and modules are further divided or combined according to the application requirements, that is, the internal structures of the devices/apparatuses are divided and combined again to implement the above-mentioned functions. Each unit and module in the above embodiments may be separate physical units, or two or more units and modules may be integrated into one physical unit. The units and modules in the above embodiments may implement corresponding functions by using hardware and/or software functional units. Direct coupling, indirect coupling or communication connection among a plurality of units, components and modules in the above embodiments can be realized through a bus or an interface; the coupling, connection, etc. between the multiple units or devices may be electrical, mechanical, or the like. Accordingly, the specific names of the units and modules in the above embodiments are only for convenience of description and distinction, and do not limit the scope of protection of the present application.

In one embodiment of the present invention, a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, can implement the method of evaluating the quality of a photograph as described in the preceding embodiments. That is, when part or all of the technical solutions of the embodiments of the present invention contributing to the prior art are embodied by means of a computer software product, the computer software product is stored in a computer-readable storage medium. The computer readable storage medium can be any portable computer program code entity apparatus or device. For example, the computer readable storage medium may be a U disk, a removable magnetic disk, a magnetic diskette, an optical disk, a computer memory, a read-only memory, a random access memory, etc.

It should be noted that the above embodiments can be freely combined as necessary. The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A method for evaluating a picture quality, comprising:

acquiring a photo to be evaluated;

judging whether the photo is qualified from the following at least two dimensions: the overall imaging quality, the imaging quality of a shooting target area and the composition quality of a shooting target;

when the photo is qualified, carrying out first-class quantitative evaluation on the photo according to a photographic aesthetic evaluation index to obtain a first-class quantitative evaluation result;

and obtaining the comprehensive evaluation of the photo according to the quantitative evaluation result.

2. The method for evaluating the quality of a photograph as claimed in claim 1, wherein said determining whether said photograph is acceptable or not is performed from a combination of at least two dimensions selected from the group consisting of:

when the photo is qualified, performing second-class quantitative evaluation according to the maximum similarity of the photo and a good photo database to obtain a second-class quantitative evaluation result;

the obtaining of the comprehensive evaluation of the photo according to the quantitative evaluation result of the type includes:

and obtaining the comprehensive evaluation of the photo according to the first class quantitative evaluation result and the second class quantitative evaluation result.

3. The method for evaluating the quality of a photograph according to claim 1 or 2, wherein said obtaining a comprehensive evaluation of the photograph comprises:

judging whether the photo is a good photo or not according to the comprehensive evaluation;

and when the photo is a good photo, adding the good photo into the good photo database.

4. The method of claim 3, wherein the determining whether the picture is a good picture based on the overall evaluation comprises:

and judging whether the picture is a good picture or not according to the comprehensive evaluation and the evaluation of a photographer.

5. The method for evaluating the quality of a photograph according to claim 1, wherein said determining whether the photograph is acceptable from a combination of at least two dimensions comprising:

judging whether the integral imaging quality of the picture is qualified or not;

when the overall imaging quality of the picture is qualified, extracting a shooting target area of the picture;

judging whether the imaging quality of the shooting target area is qualified or not;

when the imaging quality of the shooting target area is qualified, judging whether the composition quality of the shooting target is qualified;

and when the composition quality of the shooting target is qualified, the picture is qualified.

6. The method according to claim 5, wherein the extracting of the shooting target region of the photograph includes:

identifying a photo type of the photo;

and selecting a corresponding target detection and segmentation algorithm according to the type of the picture, detecting a shooting target of the picture, and extracting a corresponding shooting target area.

7. The method of evaluating the quality of photographs according to claim 6, wherein:

the identifying the photo type of the photo comprises:

identifying the photo type of the photo according to the overall abstract features of the photo extracted by the convolutional neural network;

the selecting of the corresponding target detection and segmentation algorithm according to the photo type comprises:

when the photo type is a landscape photo, using an image salient region detection algorithm;

and when the photo type is a person photo, using a human body detection and segmentation algorithm.

8. The method of claim 5, wherein the determining whether the composition quality of the photographic subject is acceptable comprises:

and judging whether the composition of the shooting target accords with a preset composition rule, wherein the factors for evaluating the preset composition rule comprise the size, the position and the changeable state of the shooting target.

9. The method for evaluating the quality of the photos according to claim 2, wherein the performing of the second-class quantitative evaluation according to the maximum similarity between the photos and the good photo database and the obtaining of the second-class quantitative evaluation result comprise:

according to a similarity evaluation model based on a deep neural network, calculating the similarity between the photo and each photo in a good photo database;

obtaining the maximum similarity of the photos according to the similarity calculation result;

and obtaining a second class quantitative evaluation result of the photo according to the maximum similarity.

10. The method for evaluating the quality of a photograph as claimed in claim 9, wherein said calculating the similarity between the photograph and each photograph in the database of good photographs comprises:

respectively extracting feature vectors of the photo and a comparison photo based on a deep neural network, wherein the comparison photo is a photo in a good photo database;

calculating the Euclidean distance between the two feature vectors according to the feature vectors of the photos and the feature vectors of the comparison photos;

and obtaining the similarity between the two photos according to the Euclidean distance between the two feature vectors.

11. The method for evaluating the quality of a photograph according to claim 2, wherein the obtaining of the comprehensive evaluation of the photograph based on the first-type quantitative evaluation result and the second-type quantitative evaluation result comprises:

weighting and summing the first class quantitative evaluation result and the second class quantitative evaluation result, and taking the result after weighting and summing as the comprehensive evaluation of the photo;

and the weight of the two types of quantitative evaluation results is dynamically increased along with the increase of the number of the good photos in the good photo database until the maximum preset weight is reached.

12. The method for evaluating the quality of photos according to claim 11, wherein the weight of the two types of quantitative evaluation results dynamically increases as the number of good photos in the good photo database increases until reaching the maximum preset weight, and comprises:

when the number of good photos in the good photo database is less than or equal to a first threshold value, the weight of the second-class quantitative evaluation result is a preset initial value;

when the number of good photos in the good photo database is greater than or equal to a second threshold value, the weight of the second type of quantitative evaluation result is the maximum preset weight;

when the number of good photos in the good photo database is between the first threshold and the second threshold, the weight of the two types of quantitative evaluation results is dynamically increased along with the increase of the number of the good photos.

13. An apparatus for evaluating a quality of a photograph, comprising:

the photo obtaining module is used for obtaining a photo to be evaluated;

the qualification judging module is used for judging whether the photo is qualified or not from the following at least two dimensionalities: the overall imaging quality, the imaging quality of a shooting target area and the composition quality of a shooting target;

the first-class evaluation module is used for carrying out first-class quantitative evaluation on the photos according to the photo aesthetic evaluation index when the photos are qualified to obtain first-class quantitative evaluation results;

and the comprehensive evaluation module is used for obtaining the comprehensive evaluation of the photo according to the quantitative evaluation result.

14. The apparatus for evaluating the quality of a photograph according to claim 13, further comprising:

the second-class evaluation module is used for performing second-class quantitative evaluation according to the maximum similarity between the photo and the good photo database when the photo is qualified to obtain a second-class quantitative evaluation result;

and the comprehensive evaluation module is further used for obtaining the comprehensive evaluation of the photo according to the first class quantitative evaluation result and the second class quantitative evaluation result.

15. The apparatus for evaluating the quality of a photograph according to claim 13 or 14, characterized by further comprising:

the database updating module is used for judging whether the photo is a good photo or not according to the comprehensive evaluation; and when the photo is a good photo, adding the good photo into the good photo database.

16. The apparatus for evaluating the quality of a photograph according to claim 15, characterized in that:

and the database updating module is further used for judging whether the picture is a good picture according to the comprehensive evaluation and the evaluation of a photographer.

17. The apparatus for evaluating the quality of a photograph according to claim 13, wherein the qualification judging module includes:

the integral imaging judging unit is used for judging whether the integral imaging quality of the photo is qualified or not;

the target area extracting unit is used for extracting a shooting target area of the picture when the overall imaging quality of the picture is qualified;

the local imaging judging unit is used for judging whether the imaging quality of the shooting target area is qualified or not;

the composition quality judging unit is used for judging whether the composition quality of the shooting target is qualified or not when the imaging quality of the shooting target area is qualified;

the qualification judgment module is further used for judging that the picture is qualified when the composition quality of the shooting target is qualified.

18. The apparatus for evaluating the quality of a photograph according to claim 17, characterized in that:

the target area extracting unit is further used for identifying the photo type of the photo; and selecting a corresponding target detection and segmentation algorithm according to the type of the picture, detecting a shooting target of the picture, and extracting a corresponding shooting target area.

19. The apparatus for evaluating the quality of a photograph according to claim 18, characterized in that:

the target area extracting unit is further used for identifying the photo type of the photo according to the overall abstract features of the photo extracted by the convolutional neural network; and when the photo type is landscape photo, using an image salient region detection algorithm; and when the photo type is a person photo, using a human body detection and segmentation algorithm.

20. The apparatus for evaluating the quality of a photograph according to claim 17, characterized in that:

the composition quality judging unit is further used for judging whether the composition of the shooting target accords with a preset composition rule, and factors for evaluating the preset composition rule comprise the size, the position and the changeable state of the shooting target.

21. The apparatus for evaluating the quality of a photograph according to claim 14, characterized in that:

the second-class evaluation module is further used for calculating the similarity between the photo and each photo in the good photo database according to a similarity evaluation model based on a deep neural network; obtaining the maximum similarity of the photos according to the similarity calculation result; and obtaining a second class quantitative evaluation result of the photo according to the maximum similarity.

22. The apparatus for evaluating the quality of a photograph as claimed in claim 21, wherein:

the comprehensive evaluation module is further used for weighting and summing the first-class quantitative evaluation result and the second-class quantitative evaluation result, and taking the result after weighted and summed as the comprehensive evaluation of the photo; and the weight of the two types of quantitative evaluation results is dynamically increased along with the increase of the number of the good photos in the good photo database until the maximum preset weight is reached.

23. An electronic device, comprising:

a memory for storing a computer program;

a processor for implementing the method of assessing the quality of a photograph as claimed in any one of claims 1 to 12 when said computer program is run.

24. A computer-readable storage medium having stored thereon a computer program, characterized in that:

the computer program, when executed by a processor, implements the method of evaluating photograph quality according to any one of claims 1 to 12.