CN112967358A - Aesthetic quality-based digital photo album screening method and device and electronic equipment - Google Patents

Aesthetic quality-based digital photo album screening method and device and electronic equipment Download PDF

Info

Publication number
CN112967358A
CN112967358A CN202110250747.2A CN202110250747A CN112967358A CN 112967358 A CN112967358 A CN 112967358A CN 202110250747 A CN202110250747 A CN 202110250747A CN 112967358 A CN112967358 A CN 112967358A
Authority
CN
China
Prior art keywords
neural network
grading
digital photo
training
screening
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110250747.2A
Other languages
Chinese (zh)
Inventor
詹明昊
杨亚
范春辉
王春雷
侯晓楠
刘齐轩
王欣然
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Micro Motor Research Institute 21st Research Institute Of China Electronics Technology Corp
Original Assignee
Shanghai Micro Motor Research Institute 21st Research Institute Of China Electronics Technology Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Micro Motor Research Institute 21st Research Institute Of China Electronics Technology Corp filed Critical Shanghai Micro Motor Research Institute 21st Research Institute Of China Electronics Technology Corp
Priority to CN202110250747.2A priority Critical patent/CN112967358A/en
Publication of CN112967358A publication Critical patent/CN112967358A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/60Editing figures and text; Combining figures or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Library & Information Science (AREA)
  • Databases & Information Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a digital photo album screening method, a digital photo album screening device and electronic equipment based on aesthetic quality. The method and the device can make quality judgment similar to subjective thinking of people on the photos in the digital photo album, retain high-quality pictures in the aspect of aesthetic quality, and discard ugly pictures; meanwhile, a large number of images can be rapidly processed, and the problems of time and labor waste are solved, so that the efficiency of screening the photos in the digital photo album is integrally improved.

Description

Aesthetic quality-based digital photo album screening method and device and electronic equipment
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a digital photo album screening method and device based on aesthetic quality and electronic equipment.
Background
With the popularization of intelligent equipment, the smart phone can take pictures anytime and anywhere, the cost of picture taking is greatly reduced, and a large number of pictures can be accumulated in a digital photo album of a user unlike the conventional case that the storage amount of films is small. The user needs to classify the photos differently, and at present, the user mainly screens the photo album by vision in life, so that a great deal of time and energy are consumed.
Because users mostly take pictures randomly, the quality of pictures in the digital photo album is not uniform due to the shaking degree, the camera type, the camera mode and the like in the shooting process. When the difference degree of the picture quality is not large, the user can not distinguish the pictures even if the user has long manual visual recognition.
Recently, intelligent screening methods through some algorithms are also available in the market, but most screening algorithms are based on some parameters or parameters, such as size, aspect ratio, saturation, histogram information and the like of a limited image, and images which meet expectations and are out of the threshold are retained by using the parameters as the threshold of image set filtering; or based on the screening algorithm of the image content, training the image recognition network, using the output of the network as the label of the image, and then performing screening, such as "keeping the picture containing the portrait", "deleting the picture containing the vehicle", and the like. The image screening algorithm intelligently screens and filters according to specific parameters or image elements, but cannot evaluate the image from an aesthetic perspective and cannot judge whether the image is beautiful or not. Particularly, when a landscape photograph or a portrait photograph is screened, the method has a high possibility that a photograph with a beautiful composition is discarded, and a photograph which is not good in appearance although the condition is satisfied is retained.
Therefore, there is a need to provide an improved solution to the above-mentioned deficiencies in the prior art.
Disclosure of Invention
The invention aims to provide a digital photo album screening method, a digital photo album screening device and electronic equipment based on aesthetic quality, which are used for overcoming the problems in the prior art.
In order to achieve the above purpose, the invention provides the following technical scheme:
a method for screening digital photo albums based on aesthetic quality, the method comprising:
obtaining an AVA data set, and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
acquiring an original photo in a digital photo album, and preprocessing the original photo to obtain a photo to be evaluated;
inputting the photo to be evaluated into the neural network model, and outputting a grading result of the grading classification channel; the grading result comprises composition grading, color grading and illumination grading;
and screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
Further, the process of training the AVA data set to obtain the neural network model includes:
building a neural network structure;
selecting an AVA training set according to the AVA data set;
and processing the AVA training set to obtain training data, and inputting the training data into a built neural network structure for training to obtain the neural network model.
Furthermore, the neural network structure adopts an arrangement mode of 5 layers of convolution and 3 layers of full connection, and the last layer of full connection layer only comprises the full connection layer and does not comprise an activation function.
Further, in the training process, the convergence result of the neural network is judged through the loss function.
Further, the screening out photos in the digital photo album according to the scoring result, wherein the photos meet the aesthetic quality, comprises:
processing the scoring result to obtain a final score;
and screening out photos which accord with aesthetic quality in the digital photo album according to the final scores.
In order to achieve the above object, the present invention provides the following technical solutions
A digital photo album screening apparatus based on aesthetic quality, the digital photo album screening apparatus comprising:
the building and scoring module is used for obtaining an AVA data set and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
the image acquisition module is used for acquiring an original photo in the digital photo album and preprocessing the original photo to obtain a photo to be evaluated;
the photo album scoring module is used for inputting the photo to be evaluated into the neural network model and outputting a scoring result of the scoring classification channel; the grading result comprises composition grading, color grading and illumination grading;
and the photo screening module is used for screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
Further, the build scoring module includes:
the building unit is used for building a neural network structure;
the selection unit is used for selecting an AVA training set according to the AVA data set;
and the training unit is used for processing the AVA training set to obtain training data, and inputting the training data into the built neural network structure for training to obtain the neural network model.
Further, the photo screening module comprises:
the processing unit is used for processing the scoring result to obtain a final score;
and the screening unit is used for screening out the photos meeting the aesthetic quality in the digital photo album according to the final scores.
In order to achieve the above purpose, the invention provides the following technical scheme:
an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the aesthetic quality based digital photo album screening method when executing the computer program.
Has the advantages that:
the digital photo album screening method can be used for automatically constructing a neural network model for aesthetic quality grading, preprocessing original photos in the digital photo album after the neural network model capable of grading according to the aesthetic quality is constructed, inputting the preprocessed photos to be evaluated into the constructed neural network model, grading each photo to be evaluated by the neural network model, outputting a grading result, and finally screening out the photos meeting the aesthetic quality in the digital photo album according to the grading result. The method and the device can make quality judgment similar to subjective thinking of people on the photos in the digital photo album, retain high-quality pictures in the aspect of aesthetic quality, and discard ugly pictures; meanwhile, a large number of images can be rapidly processed, and the problems of time and labor waste are solved, so that the efficiency of screening the photos in the digital photo album is integrally improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the invention and, together with the description, serve to explain the invention and not to limit the invention. Wherein:
FIG. 1 is a flow chart of the aesthetic quality based digital photo album screening method of the present invention;
FIG. 2 is a schematic diagram of a neural network according to the present invention;
FIG. 3 is a schematic diagram of a process for training AVA data sets to obtain neural network models according to the present invention;
FIG. 4 is a schematic structural diagram of the digital photo album screening apparatus based on aesthetic quality according to the present invention;
FIG. 5 is a schematic structural diagram of a scoring module according to the present invention;
FIG. 6 is a schematic structural diagram of a photo screening module according to the present invention.
Detailed Description
The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings. The various examples are provided by way of explanation of the invention, and not limitation of the invention. In fact, it will be apparent to those skilled in the art that modifications and variations can be made in the present invention without departing from the scope or spirit thereof. For instance, features illustrated or described as part of one embodiment, can be used with another embodiment to yield a still further embodiment. It is therefore intended that the present invention encompass such modifications and variations as fall within the scope of the appended claims and equivalents thereof.
Step S1, obtaining an AVA data set, and training to obtain a neural network model based on the AVA data set; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
the AVA (aesthetic Visual analysis) dataset is the largest aesthetic quality dataset, containing 25 million maps, each map containing semantic labels, style labels, and aesthetic scores at different angles; wherein, the semantic labels are in 66 classes, the style labels are in 14 classes, and the score is 1-10 and is in 10 scores.
As shown in fig. 3, the process of obtaining the neural network model by training the AVA dataset of the present application includes:
s101, building a neural network structure;
in this application embodiment, the neural network structure adopts the Keras operation storehouse to build, and concrete neural network structure adopts 5 layers of convolution +3 layers of all-connected arrangement modes, as shown in FIG. 2, specifically includes eight layers, and preceding five layers are the convolution layer, and 3 layers behind are the all-connected layer, and the last all-connected layer outputs the distribution of categorised label. The arrangement mode of the neural network structure can reduce the complexity of the network as much as possible under the condition of ensuring the performance of the neural network, and the neural network structure is easier to deploy at a mobile terminal with weaker computing power than a PC (personal computer) and the like.
The neural network structure is given layer by layer as follows:
the size of the original image input into the neural network is 224 × 3, and then the original image is input into each layer of network structure in the neural network for training. In another embodiment, the original image size may be 227 × 3, and may be set as necessary.
The first layer structure comprises a convolution layer, an activation layer, a normalization layer and a pooling layer in sequence. Wherein the convolution layer size is 11 × 96, i.e. the width and height dimensions are 11 × 11, 96 convolution kernels are used, and the step size is set to 4; after convolution of the convolution layer, the activation process is performed, and in the embodiment of the present application, the activation function of the activation layer specifically uses ReLU, so that the size of the output is 224/4 × 56, and the removed edge is 55, so that each pixel layer image feature map output by the activation function is 55 × 96. After the layer is returned, namely the LRN layer is arranged behind the activation layer, the size of the pixel layer image is unchanged after normalization; and finally, sending the pixel layer image after the normalization processing into a maximum pooling layer, wherein the kernel size is 3 × 3, and the set step size is 2, so that the size edge of the pixel layer image feature map is 27 × 96.
The basis (sensor) of the second layer input is 27 × 96 pixel layer images output by the first layer, the second layer structure is the same as the first layer structure, the size of the convolution layer is 5 × 256, the set step size is 1, the pixel layer image size before and after convolution calculation is unchanged, and the convolution layer is also followed by a ReLU activation layer and an LRN layer; this is followed by the maximum pooling layer, with a kernel size of 3 x 3, and a set step size of 2, so that the pixel layer image feature map is 13 x 256.
The basis (tenor) of the third to fifth layer inputs is 13 × 256, the convolution of the specific third layer is 3 × 384, the step size is set to 1, followed by the ReLU active layer; the fourth layer of convolution is 3 × 384, the step size is set to 1, and the next ReLU active layer follows; the fifth layer convolution is 3 × 256, the step size is set to 1, and a ReLU activation layer is followed; the fifth layer is followed by the largest pooling layer, kernel size 3 x 3, set to step size 2, so the pixel layer image feature map is 6 x 256.
The sixth layer of input data is the output of the fifth layer, with a size of 6 x 256. The sixth layer is a fully connected layer, which has 4096 convolution kernels in total and can output 4096 neurons, that is, 4096 operation results, and 4096 operation results generate 4096 data values through the ReLU activation function.
4096 data values output by the sixth layer are fully connected with 4096 neurons of the seventh layer, and then 4096 data values are generated after the ReLU activation processing. 4096 data values output by the seventh layer are fully connected with neurons of the eighth layer, and prediction results are directly output after training. The three scoring classification channels include composition, color, and illumination.
In the application, the ReLU is specifically used as the activation function, and as the ReLU is linear and the derivative is always 1, the calculated amount is greatly reduced, and the effects of light-weight deployment and rapid convergence can be achieved.
Step S102, an AVA training set is selected according to the AVA data set;
according to the acquired AVA data sets, 50000 images of high-score (7-9), medium-score (5-7) and low-score (3-5) in the AVA data sets are selected as training sets. Tens of thousands of images with different scores are selected from the AVA data set for training, so that the phenomenon of overfitting caused by single score in the training process can be avoided.
In the embodiment of the present application, a chronological relationship does not exist between step S101 and step S102, and step S101 may be executed first, and then step S102 may be executed; step S102 may be executed first, and then step S101 may be executed; or step S101 and step S102 are performed simultaneously. It should be understood that the above description is only exemplary, and the embodiments of the present application do not limit the present invention.
And S103, processing the AVA training set to obtain training data, and inputting the training data into the built neural network structure for training to obtain a neural network model.
In this application, the processing is a scaling operation, and the scaling operation is to perform a scaling operation on each piece of training data, that is, each image, specifically, to perform gaussian blur with a convolution kernel size of 5 × 5 on each piece of training data, and then scale the blurred image to 224 × 3.
The neural network structure training parameters of the present application include batch per training (batch) size, round of training (epoch), and learning rate (learning rate). In this embodiment, the neural network structure training parameters are set as: the size of each training batch is 2000 pictures, the epoch of the training round is 120 rounds, and the learning rate is 0.01.
The training process of the neural network structure adopts 5 times of cross validation, the training set is divided into 5 parts, wherein 4 parts are used as training data, the other 1 part is used as validation data, and then the training and validation data are alternated.
In the process of inputting a training set of an AVA data set as a sample to a neural network structure training, convergence judgment is carried out through a Loss function (Loss function) of the neural network, the Loss function specifically adopts Mean Square Error (MSE), and the calculation formula is as follows:
MSE=(∑(yi-yi’)2)/n
wherein, yiIs the true value of the ith sample, yi' is the predicted value of the ith sample, and n is the total number of samples. From the formula, the closer the MSE is to 0, the better the fitting effect of the network.
In the embodiment of the application, the convergence of the neural network structure training result is judged most intuitively by adopting a mean square error MES mode, and the judgment can be accurate and rapid.
In addition, the neural network optimizes and updates the neuron weight of the neural network in the training process, but the optimized neural network structure is not changed. In the embodiment of the present application, since the picture in the AVA dataset has the score based on composition, color and illumination, the picture becomes the basis for updating the neuron weight in the training process of the neural network. The updating strategy of the neuron weight of the neural network adopts an Adam optimizer to achieve the effect of faster network convergence. It should be noted that the specific values of the neuron parameters are not visible during and after the training.
The algorithmic pseudo-code for the Adam optimizer is as follows:
Figure BDA0002965952180000071
with the increase of training rounds, the neural weight of the neural network is updated, the Loss function is gradually reduced, after 120 rounds, the Loss function tends to be stable, the neural network converges, and the training process is finished to obtain the neural network model.
In the application, based on the neural network structure and the AVA data set, the trained neural network model is determined according to aesthetic quality and can output scoring results of three scoring classification channels. The scoring result obtained by training is more suitable for the subjective feeling of professional photographers. The three grading classification channels comprise composition, color and illumination, and respectively correspond to three elements of composition, color and illumination of the input image.
Step S2, acquiring an original photo in the digital photo album, and preprocessing the original photo to obtain a photo to be evaluated;
based on the trained neural network model, the pictures required to be input into the neural network model are fixed-size pictures, but the original pictures (i.e. the pictures to be screened) in the digital photo album do not necessarily meet the input size requirement of the neural network model, so that the original pictures need to be preprocessed to meet the input size of the neural network model.
Therefore, the emphasis of the preprocessing is to make the size of the original photos in the digital photo album meet the input requirements of the neural network, i.e. the size is 224 × 3 (length × width × RGB (color elements)). Then, the preprocessing is to perform equal ratio scaling on the original photo, specifically, to perform gaussian blurring with convolution kernel size of 5 × 5 on the original photo in the digital photo album, and then to perform equal ratio scaling on the blurred image to 224 × 3; the original photo is subjected to size stretching after equal ratio scaling, the length and the width of the original photo are only influenced, the composition factor of the picture is not influenced, elements such as golden section and the like are not damaged, and the RGB color elements are not changed. As other embodiments, non-critical operations such as noise reduction, erosion, blurring, etc. may be omitted.
Step S3, inputting the picture to be evaluated into the neural network model, and outputting the scoring result of the scoring classification channel; the grading result comprises composition grading, color grading and illumination grading;
the photos to be evaluated obtained through preprocessing are input into the neural network model obtained through training, and the output of the neural network model obtained through training comprises three grading classification channels of composition, color and illumination, so that after the photos to be evaluated are input into the neural network model, the grading results of the three grading classification channels of each photo to be evaluated can be output. The grading results of the three grading classification channels respectively correspond to the composition grading F of each original photo in the digital photo album1iColor score F2iAnd illumination score F3i
And step S4, screening out photos which accord with aesthetic quality in the digital photo album according to the scoring result.
This step is intended to screen out photographs that are aesthetically pleasing. Specifically, the step of screening out photos meeting the aesthetic quality in the digital photo album according to the scoring result comprises the following steps:
step S401, processing the scoring result to obtain a final score;
when the scoring result is processed to obtain the final score, the scoring may be performed with emphasis on a certain element of composition, color or illumination, or the final score may be obtained by performing a comprehensive scoring through the scoring of the three elements.
When processed with the scoring results for any of the three elements:
if the scoring is focused only on the composition factor, the final score F ═ F1i
If the scoring is focused on only the color elements, the final score F ═ F2i
If the scoring is focused on only the illumination element, the final score F ═ F3i
For example: if the picture A is input into the trained neural network model, the output scoring result is as follows: the composition score was 7.5, the color score was 9, and the light score was 6. If the scoring is performed with emphasis on the composition elements only, the final score F is 7.5 points; if only the color elements are emphasized to be scored, the final score F is 9; if the score is made with emphasis on only the lighting elements, the final score F is 6.
When the scoring results of the three elements are processed, the final scoring result is:
F=(F1i*i+F2i*j+F3i*k)/3
wherein i + j + k is 3, and i, j and k respectively correspond to the weighting proportion of composition color illumination.
If the user wishes to focus on color, let j be 2.5, i be k 0.25; the final score for photograph a is 8.62 points F.
If the user requires balance for the three elements, i ═ j ═ k ═ 1; the final score for photograph a is 7.5 points F.
And S402, screening out photos meeting aesthetic quality in the digital photo album according to the final scores.
When the photos in the digital photo album are screened according to the final result, screening is carried out according to a set threshold value. And determining a set threshold S for screening judgment according to aesthetic quality requirements and requirements of a user on the photos to be screened.
Screening out corresponding photos when the final score F is larger than or equal to a set threshold S;
when the final score F < the set threshold S, the corresponding photograph is discarded.
Based on the neural network model, after the photos in the digital photo album are input into the neural network model, an aesthetic quality evaluation result close to the subjective of a professional photographer can be given, and the evaluation result comprises a composition score, a color score and a light score which range from 0 to 9 points; and then, according to the processing and judgment of the scoring result, the user can be helped to screen out the photos meeting the aesthetic quality, and the time and energy consumed by the user in the screening and sorting of the digital photo album can be effectively reduced.
The embodiment of the device is as follows:
to achieve the technical purpose of the present application, the present application also proposes an aesthetic quality-based digital album screening apparatus, as shown in fig. 4, comprising:
the building and scoring module is used for obtaining an AVA data set and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
the image acquisition module is used for acquiring an original photo in the digital photo album and preprocessing the original photo to obtain a photo to be evaluated;
the photo album scoring module is used for inputting the photo to be evaluated into the neural network model and outputting a scoring result of the scoring classification channel; the grading result comprises composition grading, color grading and illumination grading;
and the photo screening module is used for screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
The digital photo album screening device can automatically build a neural network model for aesthetic quality scoring, after the neural network model which can be scored according to the aesthetic quality is built, the original photos in the digital photo album are preprocessed through the image acquisition module, the preprocessed photos to be evaluated are input into the built neural network model, each photo to be evaluated is scored through the photo album scoring model, a scoring result is output, and finally the photos which are in line with the aesthetic quality in the digital photo album are screened out through the photo screening module according to the scoring result.
Preferably, as shown in fig. 5, the construction scoring module includes:
the building unit is used for building a neural network structure;
the selection unit is used for selecting an AVA training set according to the AVA data set;
and the training unit is used for processing the AVA training set to obtain training data, and inputting the training data into the built neural network structure for training to obtain the neural network model.
Preferably, as shown in fig. 6, the photo screening module includes:
the processing unit is used for processing the scoring result to obtain a final score;
and the screening unit is used for screening out the photos meeting the aesthetic quality in the digital photo album according to the final scores.
The specific steps of the method for executing different modules and units have been described in detail in the above embodiment of the method for screening digital photo albums based on aesthetic quality, and are not described in detail herein.
Electronic equipment embodiment:
to achieve the technical object of the present application, the present application also proposes an electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned aesthetic quality-based digital album filtering method when executing the computer program, and the method includes:
obtaining an AVA data set, and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
acquiring an original photo in a digital photo album, and preprocessing the original photo to obtain a photo to be evaluated;
inputting the photo to be evaluated into the neural network model, and outputting a grading result of the grading classification channel; the grading result comprises composition grading, color grading and illumination grading;
and screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
The specific process and related details of the method are introduced in the method embodiment, and are not described in detail herein.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc., and may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The electronic device of the embodiments of the present application exists in various forms, including but not limited to:
(1) a mobile communication device: such devices are characterized by mobile communications capabilities and are primarily targeted at providing voice, data communications. Such terminals include: smart phones (e.g., IPhone), multimedia phones, functional phones, and low-end phones, etc.
(2) Ultra mobile personal computer device: the equipment belongs to the category of personal computers, has calculation and processing functions and generally has the characteristic of mobile internet access. Such terminals include: PDA, MID, and UMPC devices, etc., such as Ipad.
(3) A portable entertainment device: such devices can display and play multimedia content. This type of device comprises: audio and video players (e.g., iPod), handheld game players, electronic books, and smart toys and portable car navigation devices.
(4) A server: the device for providing the computing service comprises a processor, a hard disk, a memory, a system bus and the like, and the server is similar to a general computer architecture, but has higher requirements on processing capacity, stability, reliability, safety, expandability, manageability and the like because of the need of providing high-reliability service.
(5) And other electronic devices with data interaction functions.
It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present application may be divided into more components/steps, or two or more components/steps or partial operations of the components/steps may be combined into a new component/step to achieve the purpose of the embodiment of the present application.
The above-described methods according to embodiments of the present application may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine storage medium and to be stored in a local recording medium downloaded through a network, so that the methods described herein may be stored in such software processes on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that the computer, processor, microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the aesthetic quality based digital album screening method described herein. Further, when a general-purpose computer accesses code for implementing the methods illustrated herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the methods illustrated herein.
Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the particular application of the solution and the constraints involved. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus and system embodiments, since they are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described embodiments of the apparatus and system are merely illustrative, and elements not shown as separate may or may not be physically separate, and elements not shown as unit hints may or may not be physical elements, may be located in one place, or may be distributed across multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.

Claims (9)

1. The method for screening the digital photo album based on the aesthetic quality is characterized by comprising the following steps:
obtaining an AVA data set, and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
acquiring an original photo in a digital photo album, and preprocessing the original photo to obtain a photo to be evaluated;
inputting the photo to be evaluated into the neural network model, and outputting a grading result of the grading classification channel; the grading result comprises composition grading, color grading and illumination grading;
and screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
2. The method for screening digital photo albums based on aesthetic quality according to claim 1, wherein the process of training the AVA data set to obtain a neural network model comprises:
building a neural network structure;
selecting an AVA training set according to the AVA data set;
and processing the AVA training set to obtain training data, and inputting the training data into a built neural network structure for training to obtain the neural network model.
3. The method of claim 2, wherein the neural network structure adopts a 5-layer convolution and 3-layer fully-connected arrangement, and the last fully-connected layer only includes the fully-connected layer and does not include the activation function.
4. The method for screening digital photo albums based on aesthetic quality according to claim 2, wherein the convergence result of the neural network is judged by a loss function during the training process.
5. The method for screening the digital photo albums based on the aesthetic quality according to the claim 1, wherein the screening the photos in the digital photo albums according to the scoring result comprises:
processing the scoring result to obtain a final score;
and screening out photos which accord with aesthetic quality in the digital photo album according to the final scores.
6. A digital photo album screening apparatus based on aesthetic quality, characterized in that the digital photo album screening apparatus comprises:
the building and scoring module is used for obtaining an AVA data set and training based on the AVA data set to obtain a neural network model; the neural network model determines a grading classification channel according to aesthetic quality; the grading classification channel comprises composition, color and illumination;
the image acquisition module is used for acquiring an original photo in the digital photo album and preprocessing the original photo to obtain a photo to be evaluated;
the photo album scoring module is used for inputting the photo to be evaluated into the neural network model and outputting a scoring result of the scoring classification channel; the grading result comprises composition grading, color grading and illumination grading;
and the photo screening module is used for screening out photos which accord with aesthetic quality in the digital photo album according to the grading result.
7. The aesthetic quality-based digital photo album screening apparatus according to claim 6, wherein said construction scoring module comprises:
the building unit is used for building a neural network structure;
the selection unit is used for selecting an AVA training set according to the AVA data set;
and the training unit is used for processing the AVA training set to obtain training data, and inputting the training data into the built neural network structure for training to obtain the neural network model.
8. The aesthetic quality based digital photo album screening apparatus according to claim 6, wherein the photo screening module comprises:
the processing unit is used for processing the scoring result to obtain a final score;
and the screening unit is used for screening out the photos meeting the aesthetic quality in the digital photo album according to the final scores.
9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the aesthetic quality based digital photo album filtering method of any one of claims 1-5.
CN202110250747.2A 2021-03-08 2021-03-08 Aesthetic quality-based digital photo album screening method and device and electronic equipment Pending CN112967358A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110250747.2A CN112967358A (en) 2021-03-08 2021-03-08 Aesthetic quality-based digital photo album screening method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110250747.2A CN112967358A (en) 2021-03-08 2021-03-08 Aesthetic quality-based digital photo album screening method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN112967358A true CN112967358A (en) 2021-06-15

Family

ID=76276908

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110250747.2A Pending CN112967358A (en) 2021-03-08 2021-03-08 Aesthetic quality-based digital photo album screening method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN112967358A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113743284A (en) * 2021-08-30 2021-12-03 杭州海康威视数字技术股份有限公司 Image recognition method, device, equipment, camera and access control equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107610123A (en) * 2017-10-11 2018-01-19 中共中央办公厅电子科技学院 A kind of image aesthetic quality evaluation method based on depth convolutional neural networks
CN108492294A (en) * 2018-03-23 2018-09-04 北京邮电大学 A kind of appraisal procedure and device of image color harmony degree
CN109255374A (en) * 2018-08-27 2019-01-22 中共中央办公厅电子科技学院 A kind of aesthetic properties evaluation method based on intensive convolutional network and multitask network
CN109544524A (en) * 2018-11-15 2019-03-29 中共中央办公厅电子科技学院 A kind of more attribute image aesthetic evaluation systems based on attention mechanism
CN110473164A (en) * 2019-05-31 2019-11-19 北京理工大学 A kind of image aesthetic quality evaluation method based on attention mechanism
CN111095293A (en) * 2017-12-15 2020-05-01 华为技术有限公司 Image aesthetic processing method and electronic equipment

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107610123A (en) * 2017-10-11 2018-01-19 中共中央办公厅电子科技学院 A kind of image aesthetic quality evaluation method based on depth convolutional neural networks
CN111095293A (en) * 2017-12-15 2020-05-01 华为技术有限公司 Image aesthetic processing method and electronic equipment
CN108492294A (en) * 2018-03-23 2018-09-04 北京邮电大学 A kind of appraisal procedure and device of image color harmony degree
CN109255374A (en) * 2018-08-27 2019-01-22 中共中央办公厅电子科技学院 A kind of aesthetic properties evaluation method based on intensive convolutional network and multitask network
CN109544524A (en) * 2018-11-15 2019-03-29 中共中央办公厅电子科技学院 A kind of more attribute image aesthetic evaluation systems based on attention mechanism
CN110473164A (en) * 2019-05-31 2019-11-19 北京理工大学 A kind of image aesthetic quality evaluation method based on attention mechanism

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113743284A (en) * 2021-08-30 2021-12-03 杭州海康威视数字技术股份有限公司 Image recognition method, device, equipment, camera and access control equipment

Similar Documents

Publication Publication Date Title
CN109493350B (en) Portrait segmentation method and device
US11410038B2 (en) Frame selection based on a trained neural network
CN108876745B (en) Image processing method and device
CN111209970B (en) Video classification method, device, storage medium and server
Luo et al. Photo and video quality evaluation: Focusing on the subject
Lee et al. Automatic content-aware color and tone stylization
CN109522945B (en) Group emotion recognition method and device, intelligent device and storage medium
CN106921829A (en) A kind of photographic method and device and photographing device
CN106101541A (en) A kind of terminal, photographing device and image pickup method based on personage&#39;s emotion thereof
CN110717058B (en) Information recommendation method and device and storage medium
CN111935479B (en) Target image determination method and device, computer equipment and storage medium
CN112348747A (en) Image enhancement method, device and storage medium
Yan et al. Enhanced network optimized generative adversarial network for image enhancement
CN111614897B (en) Intelligent photographing method based on multi-dimensional driving of user preference
CN112488923A (en) Image super-resolution reconstruction method and device, storage medium and electronic equipment
Yang et al. Personalized exposure control using adaptive metering and reinforcement learning
US20090055336A1 (en) System and method for classifying multimedia data
CN111316628A (en) Image shooting method and image shooting system based on intelligent terminal
WO2022073516A1 (en) Method and apparatus for generating image, electronic device, and medium
CN112967358A (en) Aesthetic quality-based digital photo album screening method and device and electronic equipment
CN114375466A (en) Video scoring method and device, storage medium and electronic equipment
CN113079420A (en) Video generation method and device, electronic equipment and computer readable storage medium
US9888161B2 (en) Generation apparatus and method for evaluation information, electronic device and server
CN113658091A (en) Image evaluation method, storage medium and terminal equipment
US20230066331A1 (en) Method and system for automatically capturing and processing an image of a user

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination