CN112967358A - Aesthetic quality-based digital photo album screening method and device and electronic equipment - Google Patents
Aesthetic quality-based digital photo album screening method and device and electronic equipment Download PDFInfo
- Publication number
- CN112967358A CN112967358A CN202110250747.2A CN202110250747A CN112967358A CN 112967358 A CN112967358 A CN 112967358A CN 202110250747 A CN202110250747 A CN 202110250747A CN 112967358 A CN112967358 A CN 112967358A
- Authority
- CN
- China
- Prior art keywords
- neural network
- grading
- digital photo
- training
- screening
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012216 screening Methods 0.000 title claims abstract description 65
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012549 training Methods 0.000 claims description 75
- 238000003062 neural network model Methods 0.000 claims description 50
- 238000013528 artificial neural network Methods 0.000 claims description 40
- 238000005286 illumination Methods 0.000 claims description 23
- 238000012545 processing Methods 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 15
- 230000004913 activation Effects 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 239000002699 waste material Substances 0.000 abstract description 2
- 238000001994 activation Methods 0.000 description 12
- 210000002569 neuron Anatomy 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000011176 pooling Methods 0.000 description 4
- 230000002829 reductive effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- WURBVZBTWMNKQT-UHFFFAOYSA-N 1-(4-chlorophenoxy)-3,3-dimethyl-1-(1,2,4-triazol-1-yl)butan-2-one Chemical compound C1=NC=NN1C(C(=O)C(C)(C)C)OC1=CC=C(Cl)C=C1 WURBVZBTWMNKQT-UHFFFAOYSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Library & Information Science (AREA)
- Databases & Information Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a digital photo album screening method, a digital photo album screening device and electronic equipment based on aesthetic quality. The method and the device can make quality judgment similar to subjective thinking of people on the photos in the digital photo album, retain high-quality pictures in the aspect of aesthetic quality, and discard ugly pictures; meanwhile, a large number of images can be rapidly processed, and the problems of time and labor waste are solved, so that the efficiency of screening the photos in the digital photo album is integrally improved.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a digital photo album screening method and device based on aesthetic quality and electronic equipment.
Background
With the popularization of intelligent equipment, the smart phone can take pictures anytime and anywhere, the cost of picture taking is greatly reduced, and a large number of pictures can be accumulated in a digital photo album of a user unlike the conventional case that the storage amount of films is small. The user needs to classify the photos differently, and at present, the user mainly screens the photo album by vision in life, so that a great deal of time and energy are consumed.
Because users mostly take pictures randomly, the quality of pictures in the digital photo album is not uniform due to the shaking degree, the camera type, the camera mode and the like in the shooting process. When the difference degree of the picture quality is not large, the user can not distinguish the pictures even if the user has long manual visual recognition.
Recently, intelligent screening methods through some algorithms are also available in the market, but most screening algorithms are based on some parameters or parameters, such as size, aspect ratio, saturation, histogram information and the like of a limited image, and images which meet expectations and are out of the threshold are retained by using the parameters as the threshold of image set filtering; or based on the screening algorithm of the image content, training the image recognition network, using the output of the network as the label of the image, and then performing screening, such as "keeping the picture containing the portrait", "deleting the picture containing the vehicle", and the like. The image screening algorithm intelligently screens and filters according to specific parameters or image elements, but cannot evaluate the image from an aesthetic perspective and cannot judge whether the image is beautiful or not. Particularly, when a landscape photograph or a portrait photograph is screened, the method has a high possibility that a photograph with a beautiful composition is discarded, and a photograph which is not good in appearance although the condition is satisfied is retained.
Therefore, there is a need to provide an improved solution to the above-mentioned deficiencies in the prior art.
Disclosure of Invention
The invention aims to provide a digital photo album screening method, a digital photo album screening device and electronic equipment based on aesthetic quality, which are used for overcoming the problems in the prior art.
In order to achieve the above purpose, the invention provides the following technical scheme:
a method for screening digital photo albums based on aesthetic quality, the method comprising:
obtaining an AVA data set, and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
acquiring an original photo in a digital photo album, and preprocessing the original photo to obtain a photo to be evaluated;
inputting the photo to be evaluated into the neural network model, and outputting a grading result of the grading classification channel; the grading result comprises composition grading, color grading and illumination grading;
and screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
Further, the process of training the AVA data set to obtain the neural network model includes:
building a neural network structure;
selecting an AVA training set according to the AVA data set;
and processing the AVA training set to obtain training data, and inputting the training data into a built neural network structure for training to obtain the neural network model.
Furthermore, the neural network structure adopts an arrangement mode of 5 layers of convolution and 3 layers of full connection, and the last layer of full connection layer only comprises the full connection layer and does not comprise an activation function.
Further, in the training process, the convergence result of the neural network is judged through the loss function.
Further, the screening out photos in the digital photo album according to the scoring result, wherein the photos meet the aesthetic quality, comprises:
processing the scoring result to obtain a final score;
and screening out photos which accord with aesthetic quality in the digital photo album according to the final scores.
In order to achieve the above object, the present invention provides the following technical solutions
A digital photo album screening apparatus based on aesthetic quality, the digital photo album screening apparatus comprising:
the building and scoring module is used for obtaining an AVA data set and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
the image acquisition module is used for acquiring an original photo in the digital photo album and preprocessing the original photo to obtain a photo to be evaluated;
the photo album scoring module is used for inputting the photo to be evaluated into the neural network model and outputting a scoring result of the scoring classification channel; the grading result comprises composition grading, color grading and illumination grading;
and the photo screening module is used for screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
Further, the build scoring module includes:
the building unit is used for building a neural network structure;
the selection unit is used for selecting an AVA training set according to the AVA data set;
and the training unit is used for processing the AVA training set to obtain training data, and inputting the training data into the built neural network structure for training to obtain the neural network model.
Further, the photo screening module comprises:
the processing unit is used for processing the scoring result to obtain a final score;
and the screening unit is used for screening out the photos meeting the aesthetic quality in the digital photo album according to the final scores.
In order to achieve the above purpose, the invention provides the following technical scheme:
an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the aesthetic quality based digital photo album screening method when executing the computer program.
Has the advantages that:
the digital photo album screening method can be used for automatically constructing a neural network model for aesthetic quality grading, preprocessing original photos in the digital photo album after the neural network model capable of grading according to the aesthetic quality is constructed, inputting the preprocessed photos to be evaluated into the constructed neural network model, grading each photo to be evaluated by the neural network model, outputting a grading result, and finally screening out the photos meeting the aesthetic quality in the digital photo album according to the grading result. The method and the device can make quality judgment similar to subjective thinking of people on the photos in the digital photo album, retain high-quality pictures in the aspect of aesthetic quality, and discard ugly pictures; meanwhile, a large number of images can be rapidly processed, and the problems of time and labor waste are solved, so that the efficiency of screening the photos in the digital photo album is integrally improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the invention and, together with the description, serve to explain the invention and not to limit the invention. Wherein:
FIG. 1 is a flow chart of the aesthetic quality based digital photo album screening method of the present invention;
FIG. 2 is a schematic diagram of a neural network according to the present invention;
FIG. 3 is a schematic diagram of a process for training AVA data sets to obtain neural network models according to the present invention;
FIG. 4 is a schematic structural diagram of the digital photo album screening apparatus based on aesthetic quality according to the present invention;
FIG. 5 is a schematic structural diagram of a scoring module according to the present invention;
FIG. 6 is a schematic structural diagram of a photo screening module according to the present invention.
Detailed Description
The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings. The various examples are provided by way of explanation of the invention, and not limitation of the invention. In fact, it will be apparent to those skilled in the art that modifications and variations can be made in the present invention without departing from the scope or spirit thereof. For instance, features illustrated or described as part of one embodiment, can be used with another embodiment to yield a still further embodiment. It is therefore intended that the present invention encompass such modifications and variations as fall within the scope of the appended claims and equivalents thereof.
Step S1, obtaining an AVA data set, and training to obtain a neural network model based on the AVA data set; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
the AVA (aesthetic Visual analysis) dataset is the largest aesthetic quality dataset, containing 25 million maps, each map containing semantic labels, style labels, and aesthetic scores at different angles; wherein, the semantic labels are in 66 classes, the style labels are in 14 classes, and the score is 1-10 and is in 10 scores.
As shown in fig. 3, the process of obtaining the neural network model by training the AVA dataset of the present application includes:
s101, building a neural network structure;
in this application embodiment, the neural network structure adopts the Keras operation storehouse to build, and concrete neural network structure adopts 5 layers of convolution +3 layers of all-connected arrangement modes, as shown in FIG. 2, specifically includes eight layers, and preceding five layers are the convolution layer, and 3 layers behind are the all-connected layer, and the last all-connected layer outputs the distribution of categorised label. The arrangement mode of the neural network structure can reduce the complexity of the network as much as possible under the condition of ensuring the performance of the neural network, and the neural network structure is easier to deploy at a mobile terminal with weaker computing power than a PC (personal computer) and the like.
The neural network structure is given layer by layer as follows:
the size of the original image input into the neural network is 224 × 3, and then the original image is input into each layer of network structure in the neural network for training. In another embodiment, the original image size may be 227 × 3, and may be set as necessary.
The first layer structure comprises a convolution layer, an activation layer, a normalization layer and a pooling layer in sequence. Wherein the convolution layer size is 11 × 96, i.e. the width and height dimensions are 11 × 11, 96 convolution kernels are used, and the step size is set to 4; after convolution of the convolution layer, the activation process is performed, and in the embodiment of the present application, the activation function of the activation layer specifically uses ReLU, so that the size of the output is 224/4 × 56, and the removed edge is 55, so that each pixel layer image feature map output by the activation function is 55 × 96. After the layer is returned, namely the LRN layer is arranged behind the activation layer, the size of the pixel layer image is unchanged after normalization; and finally, sending the pixel layer image after the normalization processing into a maximum pooling layer, wherein the kernel size is 3 × 3, and the set step size is 2, so that the size edge of the pixel layer image feature map is 27 × 96.
The basis (sensor) of the second layer input is 27 × 96 pixel layer images output by the first layer, the second layer structure is the same as the first layer structure, the size of the convolution layer is 5 × 256, the set step size is 1, the pixel layer image size before and after convolution calculation is unchanged, and the convolution layer is also followed by a ReLU activation layer and an LRN layer; this is followed by the maximum pooling layer, with a kernel size of 3 x 3, and a set step size of 2, so that the pixel layer image feature map is 13 x 256.
The basis (tenor) of the third to fifth layer inputs is 13 × 256, the convolution of the specific third layer is 3 × 384, the step size is set to 1, followed by the ReLU active layer; the fourth layer of convolution is 3 × 384, the step size is set to 1, and the next ReLU active layer follows; the fifth layer convolution is 3 × 256, the step size is set to 1, and a ReLU activation layer is followed; the fifth layer is followed by the largest pooling layer, kernel size 3 x 3, set to step size 2, so the pixel layer image feature map is 6 x 256.
The sixth layer of input data is the output of the fifth layer, with a size of 6 x 256. The sixth layer is a fully connected layer, which has 4096 convolution kernels in total and can output 4096 neurons, that is, 4096 operation results, and 4096 operation results generate 4096 data values through the ReLU activation function.
4096 data values output by the sixth layer are fully connected with 4096 neurons of the seventh layer, and then 4096 data values are generated after the ReLU activation processing. 4096 data values output by the seventh layer are fully connected with neurons of the eighth layer, and prediction results are directly output after training. The three scoring classification channels include composition, color, and illumination.
In the application, the ReLU is specifically used as the activation function, and as the ReLU is linear and the derivative is always 1, the calculated amount is greatly reduced, and the effects of light-weight deployment and rapid convergence can be achieved.
Step S102, an AVA training set is selected according to the AVA data set;
according to the acquired AVA data sets, 50000 images of high-score (7-9), medium-score (5-7) and low-score (3-5) in the AVA data sets are selected as training sets. Tens of thousands of images with different scores are selected from the AVA data set for training, so that the phenomenon of overfitting caused by single score in the training process can be avoided.
In the embodiment of the present application, a chronological relationship does not exist between step S101 and step S102, and step S101 may be executed first, and then step S102 may be executed; step S102 may be executed first, and then step S101 may be executed; or step S101 and step S102 are performed simultaneously. It should be understood that the above description is only exemplary, and the embodiments of the present application do not limit the present invention.
And S103, processing the AVA training set to obtain training data, and inputting the training data into the built neural network structure for training to obtain a neural network model.
In this application, the processing is a scaling operation, and the scaling operation is to perform a scaling operation on each piece of training data, that is, each image, specifically, to perform gaussian blur with a convolution kernel size of 5 × 5 on each piece of training data, and then scale the blurred image to 224 × 3.
The neural network structure training parameters of the present application include batch per training (batch) size, round of training (epoch), and learning rate (learning rate). In this embodiment, the neural network structure training parameters are set as: the size of each training batch is 2000 pictures, the epoch of the training round is 120 rounds, and the learning rate is 0.01.
The training process of the neural network structure adopts 5 times of cross validation, the training set is divided into 5 parts, wherein 4 parts are used as training data, the other 1 part is used as validation data, and then the training and validation data are alternated.
In the process of inputting a training set of an AVA data set as a sample to a neural network structure training, convergence judgment is carried out through a Loss function (Loss function) of the neural network, the Loss function specifically adopts Mean Square Error (MSE), and the calculation formula is as follows:
MSE=(∑(yi-yi’)2)/n
wherein, yiIs the true value of the ith sample, yi' is the predicted value of the ith sample, and n is the total number of samples. From the formula, the closer the MSE is to 0, the better the fitting effect of the network.
In the embodiment of the application, the convergence of the neural network structure training result is judged most intuitively by adopting a mean square error MES mode, and the judgment can be accurate and rapid.
In addition, the neural network optimizes and updates the neuron weight of the neural network in the training process, but the optimized neural network structure is not changed. In the embodiment of the present application, since the picture in the AVA dataset has the score based on composition, color and illumination, the picture becomes the basis for updating the neuron weight in the training process of the neural network. The updating strategy of the neuron weight of the neural network adopts an Adam optimizer to achieve the effect of faster network convergence. It should be noted that the specific values of the neuron parameters are not visible during and after the training.
The algorithmic pseudo-code for the Adam optimizer is as follows:
with the increase of training rounds, the neural weight of the neural network is updated, the Loss function is gradually reduced, after 120 rounds, the Loss function tends to be stable, the neural network converges, and the training process is finished to obtain the neural network model.
In the application, based on the neural network structure and the AVA data set, the trained neural network model is determined according to aesthetic quality and can output scoring results of three scoring classification channels. The scoring result obtained by training is more suitable for the subjective feeling of professional photographers. The three grading classification channels comprise composition, color and illumination, and respectively correspond to three elements of composition, color and illumination of the input image.
Step S2, acquiring an original photo in the digital photo album, and preprocessing the original photo to obtain a photo to be evaluated;
based on the trained neural network model, the pictures required to be input into the neural network model are fixed-size pictures, but the original pictures (i.e. the pictures to be screened) in the digital photo album do not necessarily meet the input size requirement of the neural network model, so that the original pictures need to be preprocessed to meet the input size of the neural network model.
Therefore, the emphasis of the preprocessing is to make the size of the original photos in the digital photo album meet the input requirements of the neural network, i.e. the size is 224 × 3 (length × width × RGB (color elements)). Then, the preprocessing is to perform equal ratio scaling on the original photo, specifically, to perform gaussian blurring with convolution kernel size of 5 × 5 on the original photo in the digital photo album, and then to perform equal ratio scaling on the blurred image to 224 × 3; the original photo is subjected to size stretching after equal ratio scaling, the length and the width of the original photo are only influenced, the composition factor of the picture is not influenced, elements such as golden section and the like are not damaged, and the RGB color elements are not changed. As other embodiments, non-critical operations such as noise reduction, erosion, blurring, etc. may be omitted.
Step S3, inputting the picture to be evaluated into the neural network model, and outputting the scoring result of the scoring classification channel; the grading result comprises composition grading, color grading and illumination grading;
the photos to be evaluated obtained through preprocessing are input into the neural network model obtained through training, and the output of the neural network model obtained through training comprises three grading classification channels of composition, color and illumination, so that after the photos to be evaluated are input into the neural network model, the grading results of the three grading classification channels of each photo to be evaluated can be output. The grading results of the three grading classification channels respectively correspond to the composition grading F of each original photo in the digital photo album1iColor score F2iAnd illumination score F3i。
And step S4, screening out photos which accord with aesthetic quality in the digital photo album according to the scoring result.
This step is intended to screen out photographs that are aesthetically pleasing. Specifically, the step of screening out photos meeting the aesthetic quality in the digital photo album according to the scoring result comprises the following steps:
step S401, processing the scoring result to obtain a final score;
when the scoring result is processed to obtain the final score, the scoring may be performed with emphasis on a certain element of composition, color or illumination, or the final score may be obtained by performing a comprehensive scoring through the scoring of the three elements.
When processed with the scoring results for any of the three elements:
if the scoring is focused only on the composition factor, the final score F ═ F1i;
If the scoring is focused on only the color elements, the final score F ═ F2i;
If the scoring is focused on only the illumination element, the final score F ═ F3i。
For example: if the picture A is input into the trained neural network model, the output scoring result is as follows: the composition score was 7.5, the color score was 9, and the light score was 6. If the scoring is performed with emphasis on the composition elements only, the final score F is 7.5 points; if only the color elements are emphasized to be scored, the final score F is 9; if the score is made with emphasis on only the lighting elements, the final score F is 6.
When the scoring results of the three elements are processed, the final scoring result is:
F=(F1i*i+F2i*j+F3i*k)/3
wherein i + j + k is 3, and i, j and k respectively correspond to the weighting proportion of composition color illumination.
If the user wishes to focus on color, let j be 2.5, i be k 0.25; the final score for photograph a is 8.62 points F.
If the user requires balance for the three elements, i ═ j ═ k ═ 1; the final score for photograph a is 7.5 points F.
And S402, screening out photos meeting aesthetic quality in the digital photo album according to the final scores.
When the photos in the digital photo album are screened according to the final result, screening is carried out according to a set threshold value. And determining a set threshold S for screening judgment according to aesthetic quality requirements and requirements of a user on the photos to be screened.
Screening out corresponding photos when the final score F is larger than or equal to a set threshold S;
when the final score F < the set threshold S, the corresponding photograph is discarded.
Based on the neural network model, after the photos in the digital photo album are input into the neural network model, an aesthetic quality evaluation result close to the subjective of a professional photographer can be given, and the evaluation result comprises a composition score, a color score and a light score which range from 0 to 9 points; and then, according to the processing and judgment of the scoring result, the user can be helped to screen out the photos meeting the aesthetic quality, and the time and energy consumed by the user in the screening and sorting of the digital photo album can be effectively reduced.
The embodiment of the device is as follows:
to achieve the technical purpose of the present application, the present application also proposes an aesthetic quality-based digital album screening apparatus, as shown in fig. 4, comprising:
the building and scoring module is used for obtaining an AVA data set and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
the image acquisition module is used for acquiring an original photo in the digital photo album and preprocessing the original photo to obtain a photo to be evaluated;
the photo album scoring module is used for inputting the photo to be evaluated into the neural network model and outputting a scoring result of the scoring classification channel; the grading result comprises composition grading, color grading and illumination grading;
and the photo screening module is used for screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
The digital photo album screening device can automatically build a neural network model for aesthetic quality scoring, after the neural network model which can be scored according to the aesthetic quality is built, the original photos in the digital photo album are preprocessed through the image acquisition module, the preprocessed photos to be evaluated are input into the built neural network model, each photo to be evaluated is scored through the photo album scoring model, a scoring result is output, and finally the photos which are in line with the aesthetic quality in the digital photo album are screened out through the photo screening module according to the scoring result.
Preferably, as shown in fig. 5, the construction scoring module includes:
the building unit is used for building a neural network structure;
the selection unit is used for selecting an AVA training set according to the AVA data set;
and the training unit is used for processing the AVA training set to obtain training data, and inputting the training data into the built neural network structure for training to obtain the neural network model.
Preferably, as shown in fig. 6, the photo screening module includes:
the processing unit is used for processing the scoring result to obtain a final score;
and the screening unit is used for screening out the photos meeting the aesthetic quality in the digital photo album according to the final scores.
The specific steps of the method for executing different modules and units have been described in detail in the above embodiment of the method for screening digital photo albums based on aesthetic quality, and are not described in detail herein.
Electronic equipment embodiment:
to achieve the technical object of the present application, the present application also proposes an electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned aesthetic quality-based digital album filtering method when executing the computer program, and the method includes:
obtaining an AVA data set, and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
acquiring an original photo in a digital photo album, and preprocessing the original photo to obtain a photo to be evaluated;
inputting the photo to be evaluated into the neural network model, and outputting a grading result of the grading classification channel; the grading result comprises composition grading, color grading and illumination grading;
and screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
The specific process and related details of the method are introduced in the method embodiment, and are not described in detail herein.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc., and may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The electronic device of the embodiments of the present application exists in various forms, including but not limited to:
(1) a mobile communication device: such devices are characterized by mobile communications capabilities and are primarily targeted at providing voice, data communications. Such terminals include: smart phones (e.g., IPhone), multimedia phones, functional phones, and low-end phones, etc.
(2) Ultra mobile personal computer device: the equipment belongs to the category of personal computers, has calculation and processing functions and generally has the characteristic of mobile internet access. Such terminals include: PDA, MID, and UMPC devices, etc., such as Ipad.
(3) A portable entertainment device: such devices can display and play multimedia content. This type of device comprises: audio and video players (e.g., iPod), handheld game players, electronic books, and smart toys and portable car navigation devices.
(4) A server: the device for providing the computing service comprises a processor, a hard disk, a memory, a system bus and the like, and the server is similar to a general computer architecture, but has higher requirements on processing capacity, stability, reliability, safety, expandability, manageability and the like because of the need of providing high-reliability service.
(5) And other electronic devices with data interaction functions.
It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present application may be divided into more components/steps, or two or more components/steps or partial operations of the components/steps may be combined into a new component/step to achieve the purpose of the embodiment of the present application.
The above-described methods according to embodiments of the present application may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine storage medium and to be stored in a local recording medium downloaded through a network, so that the methods described herein may be stored in such software processes on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that the computer, processor, microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the aesthetic quality based digital album screening method described herein. Further, when a general-purpose computer accesses code for implementing the methods illustrated herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the methods illustrated herein.
Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the particular application of the solution and the constraints involved. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus and system embodiments, since they are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described embodiments of the apparatus and system are merely illustrative, and elements not shown as separate may or may not be physically separate, and elements not shown as unit hints may or may not be physical elements, may be located in one place, or may be distributed across multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.
Claims (9)
1. The method for screening the digital photo album based on the aesthetic quality is characterized by comprising the following steps:
obtaining an AVA data set, and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
acquiring an original photo in a digital photo album, and preprocessing the original photo to obtain a photo to be evaluated;
inputting the photo to be evaluated into the neural network model, and outputting a grading result of the grading classification channel; the grading result comprises composition grading, color grading and illumination grading;
and screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
2. The method for screening digital photo albums based on aesthetic quality according to claim 1, wherein the process of training the AVA data set to obtain a neural network model comprises:
building a neural network structure;
selecting an AVA training set according to the AVA data set;
and processing the AVA training set to obtain training data, and inputting the training data into a built neural network structure for training to obtain the neural network model.
3. The method of claim 2, wherein the neural network structure adopts a 5-layer convolution and 3-layer fully-connected arrangement, and the last fully-connected layer only includes the fully-connected layer and does not include the activation function.
4. The method for screening digital photo albums based on aesthetic quality according to claim 2, wherein the convergence result of the neural network is judged by a loss function during the training process.
5. The method for screening the digital photo albums based on the aesthetic quality according to the claim 1, wherein the screening the photos in the digital photo albums according to the scoring result comprises:
processing the scoring result to obtain a final score;
and screening out photos which accord with aesthetic quality in the digital photo album according to the final scores.
6. A digital photo album screening apparatus based on aesthetic quality, characterized in that the digital photo album screening apparatus comprises:
the building and scoring module is used for obtaining an AVA data set and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
the image acquisition module is used for acquiring an original photo in the digital photo album and preprocessing the original photo to obtain a photo to be evaluated;
the photo album scoring module is used for inputting the photo to be evaluated into the neural network model and outputting a scoring result of the scoring classification channel; the grading result comprises composition grading, color grading and illumination grading;
and the photo screening module is used for screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
7. The aesthetic quality-based digital photo album screening apparatus according to claim 6, wherein said construction scoring module comprises:
the building unit is used for building a neural network structure;
the selection unit is used for selecting an AVA training set according to the AVA data set;
and the training unit is used for processing the AVA training set to obtain training data, and inputting the training data into the built neural network structure for training to obtain the neural network model.
8. The aesthetic quality based digital photo album screening apparatus according to claim 6, wherein the photo screening module comprises:
the processing unit is used for processing the scoring result to obtain a final score;
and the screening unit is used for screening out the photos meeting the aesthetic quality in the digital photo album according to the final scores.
9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the aesthetic quality based digital photo album filtering method of any one of claims 1-5.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110250747.2A CN112967358A (en) | 2021-03-08 | 2021-03-08 | Aesthetic quality-based digital photo album screening method and device and electronic equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110250747.2A CN112967358A (en) | 2021-03-08 | 2021-03-08 | Aesthetic quality-based digital photo album screening method and device and electronic equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN112967358A true CN112967358A (en) | 2021-06-15 |
Family
ID=76276908
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110250747.2A Pending CN112967358A (en) | 2021-03-08 | 2021-03-08 | Aesthetic quality-based digital photo album screening method and device and electronic equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112967358A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113743284A (en) * | 2021-08-30 | 2021-12-03 | 杭州海康威视数字技术股份有限公司 | Image recognition method, device, equipment, camera and access control equipment |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107610123A (en) * | 2017-10-11 | 2018-01-19 | 中共中央办公厅电子科技学院 | A kind of image aesthetic quality evaluation method based on depth convolutional neural networks |
CN108492294A (en) * | 2018-03-23 | 2018-09-04 | 北京邮电大学 | A kind of appraisal procedure and device of image color harmony degree |
CN109255374A (en) * | 2018-08-27 | 2019-01-22 | 中共中央办公厅电子科技学院 | A kind of aesthetic properties evaluation method based on intensive convolutional network and multitask network |
CN109544524A (en) * | 2018-11-15 | 2019-03-29 | 中共中央办公厅电子科技学院 | A kind of more attribute image aesthetic evaluation systems based on attention mechanism |
CN110473164A (en) * | 2019-05-31 | 2019-11-19 | 北京理工大学 | A kind of image aesthetic quality evaluation method based on attention mechanism |
CN111095293A (en) * | 2017-12-15 | 2020-05-01 | 华为技术有限公司 | Image aesthetic processing method and electronic equipment |
-
2021
- 2021-03-08 CN CN202110250747.2A patent/CN112967358A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107610123A (en) * | 2017-10-11 | 2018-01-19 | 中共中央办公厅电子科技学院 | A kind of image aesthetic quality evaluation method based on depth convolutional neural networks |
CN111095293A (en) * | 2017-12-15 | 2020-05-01 | 华为技术有限公司 | Image aesthetic processing method and electronic equipment |
CN108492294A (en) * | 2018-03-23 | 2018-09-04 | 北京邮电大学 | A kind of appraisal procedure and device of image color harmony degree |
CN109255374A (en) * | 2018-08-27 | 2019-01-22 | 中共中央办公厅电子科技学院 | A kind of aesthetic properties evaluation method based on intensive convolutional network and multitask network |
CN109544524A (en) * | 2018-11-15 | 2019-03-29 | 中共中央办公厅电子科技学院 | A kind of more attribute image aesthetic evaluation systems based on attention mechanism |
CN110473164A (en) * | 2019-05-31 | 2019-11-19 | 北京理工大学 | A kind of image aesthetic quality evaluation method based on attention mechanism |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113743284A (en) * | 2021-08-30 | 2021-12-03 | 杭州海康威视数字技术股份有限公司 | Image recognition method, device, equipment, camera and access control equipment |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109493350B (en) | Portrait segmentation method and device | |
US11410038B2 (en) | Frame selection based on a trained neural network | |
CN108876745B (en) | Image processing method and device | |
CN111209970B (en) | Video classification method, device, storage medium and server | |
Luo et al. | Photo and video quality evaluation: Focusing on the subject | |
Lee et al. | Automatic content-aware color and tone stylization | |
CN109522945B (en) | Group emotion recognition method and device, intelligent device and storage medium | |
CN106921829A (en) | A kind of photographic method and device and photographing device | |
CN106101541A (en) | A kind of terminal, photographing device and image pickup method based on personage's emotion thereof | |
CN110717058B (en) | Information recommendation method and device and storage medium | |
CN111935479B (en) | Target image determination method and device, computer equipment and storage medium | |
CN112348747A (en) | Image enhancement method, device and storage medium | |
Yan et al. | Enhanced network optimized generative adversarial network for image enhancement | |
CN111614897B (en) | Intelligent photographing method based on multi-dimensional driving of user preference | |
CN112488923A (en) | Image super-resolution reconstruction method and device, storage medium and electronic equipment | |
Yang et al. | Personalized exposure control using adaptive metering and reinforcement learning | |
US20090055336A1 (en) | System and method for classifying multimedia data | |
CN111316628A (en) | Image shooting method and image shooting system based on intelligent terminal | |
WO2022073516A1 (en) | Method and apparatus for generating image, electronic device, and medium | |
CN112967358A (en) | Aesthetic quality-based digital photo album screening method and device and electronic equipment | |
CN114375466A (en) | Video scoring method and device, storage medium and electronic equipment | |
CN113079420A (en) | Video generation method and device, electronic equipment and computer readable storage medium | |
US9888161B2 (en) | Generation apparatus and method for evaluation information, electronic device and server | |
CN113658091A (en) | Image evaluation method, storage medium and terminal equipment | |
US20230066331A1 (en) | Method and system for automatically capturing and processing an image of a user |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |