CN113255743A - Image processing method, image processing device, electronic equipment and storage medium - Google Patents
Image processing method, image processing device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113255743A CN113255743A CN202110519176.8A CN202110519176A CN113255743A CN 113255743 A CN113255743 A CN 113255743A CN 202110519176 A CN202110519176 A CN 202110519176A CN 113255743 A CN113255743 A CN 113255743A
- Authority
- CN
- China
- Prior art keywords
- image
- network model
- evaluation network
- aesthetic
- score
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 158
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 238000011156 evaluation Methods 0.000 claims abstract description 247
- 238000000034 method Methods 0.000 claims abstract description 37
- 238000012360 testing method Methods 0.000 claims description 73
- 230000006870 function Effects 0.000 claims description 56
- 238000012795 verification Methods 0.000 claims description 51
- 238000012549 training Methods 0.000 claims description 18
- 238000007781 pre-processing Methods 0.000 claims description 16
- 238000013527 convolutional neural network Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 7
- 238000005457 optimization Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 5
- 241000282414 Homo sapiens Species 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 2
- 238000013077 scoring method Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Game Theory and Decision Science (AREA)
- Evolutionary Biology (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application provides an image processing method, an image processing device, electronic equipment and a storage medium, wherein the method comprises the steps of extracting a target processing area of an image to be processed from the image to be processed; inputting a target processing area of the image to be processed into an image aesthetic evaluation network model, and outputting score probability information corresponding to the target processing area through the image aesthetic evaluation network model; the score probability information is used for representing the probability of each aesthetic score of the target processing area of the image to be processed; and determining the evaluation value of the target processing area according to the score probability information corresponding to the target processing area. By adopting the scheme provided by the embodiment of the application, the problem of poor consistency when the same picture is subjected to aesthetic scoring in the prior art is solved.
Description
Technical Field
The present application relates to the field of image processing, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium.
Background
With the rapid development of mobile internet and the rapid popularization of intelligent mobile devices, visual content data such as images and videos are increasing day by day, and the perceptual understanding of the visual content has become the research direction of multiple interdisciplines such as computer vision, computational camera science and human psychology. Among them, image aesthetic scoring is a research hotspot in recent computer vision perception understanding direction. The image aesthetics reflect the pursuit and the direction of human beings to 'good' things visually, so that the method has important significance in performing visual aesthetics scoring in the fields of photography, advertisement design, artistic work making and the like.
In the prior art, the main process of aesthetically scoring pictures is: an aesthetic picture dataset is obtained, and the same picture in the dataset is scored by a plurality of assessors. And averaging the scoring results of each panel by each panel member to obtain the aesthetic score of the panel. And outputting the aesthetic score of the picture.
However, the above-mentioned aesthetic picture scoring method requires that the assessors score the picture by eye observation, the subjective of the assessment result is strong, and under the condition of no objective reference, different assessors have different criteria for scoring the same picture, and even the same assessor may score the same picture at different time periods. This may cause a problem of poor consistency when the same picture is aesthetically rated.
Disclosure of Invention
In view of this, the present application provides an image processing method, an image processing apparatus, an electronic device, and a storage medium, so as to solve the problem in the prior art that the consistency is poor when performing aesthetic scoring on the same picture.
In a first aspect, an embodiment of the present application provides an image processing method, including:
extracting a target processing area of the image to be processed from the image to be processed;
inputting a target processing area of the image to be processed into an image aesthetic evaluation network model, and outputting score probability information corresponding to the target processing area through the image aesthetic evaluation network model; the score probability information is used for representing the probability of each aesthetic score of the target processing area of the image to be processed;
and determining the evaluation value of the target processing area according to the score probability information corresponding to the target processing area.
Preferably, before the inputting the target processing area into an image aesthetics evaluation network model and outputting the score probability information corresponding to the target processing area through the image aesthetics evaluation network model, the method further includes:
preprocessing the target processing area according to the network characteristics of the image aesthetic evaluation network model;
the inputting the target processing region into an image aesthetic evaluation network model, and the outputting the score probability information corresponding to the target processing region through the image aesthetic evaluation network model includes:
inputting the preprocessed target processing area into an image aesthetic evaluation network model, and outputting score probability information corresponding to the target processing area through the image aesthetic evaluation network model.
Preferably, the method further comprises the following steps:
collecting image data;
preprocessing the image data;
training an image aesthetic evaluation network model based on the preprocessed image data.
Preferably, training an image aesthetic evaluation network model based on the preprocessed image data comprises:
constructing an image aesthetic evaluation network model based on the recognition convolutional neural network;
inputting the preprocessed image data into the image aesthetic evaluation network model, and obtaining test score probability information of the image data through the output of the image aesthetic evaluation network model;
and acquiring information of the image aesthetic score of the image data which is manually evaluated, constructing a loss function according to the test score probability information and the information of the image aesthetic score which is manually evaluated, and training the image aesthetic evaluation network model according to the loss function.
Preferably, the method further comprises the following steps:
acquiring verification image data and preprocessing the verification image data;
inputting the preprocessed verification image data into the image aesthetic evaluation network model, and obtaining verification score probability information of the verification image data through the output of the image aesthetic evaluation network model;
and obtaining the information of the image aesthetic score of the manual evaluation of the verification image data, constructing a loss function according to the information of the image aesthetic score of the manual evaluation of the verification image data and the verification score probability information, and optimizing the image aesthetic evaluation network model according to the loss function.
Preferably, said optimizing said image aesthetic evaluation network model according to said loss function comprises:
determining whether the image aesthetic evaluation network model needs to be optimized according to the output value of the loss function;
and if optimization is needed, optimizing the image aesthetic evaluation network model.
Preferably, the method further comprises the following steps:
acquiring test image data, and preprocessing the test image data;
inputting the preprocessed test image data into the image aesthetic evaluation network model, and obtaining test score probability information of the test image data through the output of the image aesthetic evaluation network model;
acquiring information of an image aesthetic score of the manual evaluation of the test image data, and determining whether the image aesthetic evaluation network model is qualified or not according to the test score probability information of the test image data and the information of the image aesthetic score of the manual evaluation of the test image data;
if the image is qualified, starting the image aesthetic evaluation network model;
and if the image is not qualified, retraining the image aesthetic evaluation network model.
In a second aspect, an embodiment of the present application provides an image processing apparatus, including:
the extraction unit is used for extracting a target processing area of the image to be processed from the image to be processed;
the processing unit is used for inputting a target processing area of the image to be processed into an image aesthetic evaluation network model and outputting score probability information corresponding to the target processing area through the image aesthetic evaluation network model; the score probability information is used for representing the probability of each aesthetic score of the target processing area of the image to be processed;
and the determining unit is used for determining the evaluation value of the target processing area according to the score probability information corresponding to the target processing area.
Preferably, the processing unit is further configured to pre-process the target processing area according to the network characteristics of the image aesthetic evaluation network model;
the processing unit is configured to input the target processing area to an image aesthetics evaluation network model, and outputting, by the image aesthetics evaluation network model, score probability information corresponding to the target processing area specifically includes:
inputting the preprocessed target processing area into an image aesthetic evaluation network model, and outputting score probability information corresponding to the target processing area through the image aesthetic evaluation network model.
Preferably, the method further comprises the following steps:
the acquisition unit is used for acquiring image data;
the processing unit is further used for preprocessing the image data;
the processing unit is further used for training an image aesthetic evaluation network model based on the preprocessed image data.
Preferably, the processing unit is specifically configured to construct an image aesthetic evaluation network model based on the recognition-like convolutional neural network;
inputting the preprocessed image data into the image aesthetic evaluation network model, and obtaining test score probability information of the image data through the output of the image aesthetic evaluation network model;
and acquiring information of the image aesthetic score of the image data which is manually evaluated, constructing a loss function according to the test score probability information and the information of the image aesthetic score which is manually evaluated, and training the image aesthetic evaluation network model according to the loss function.
Preferably, the method further comprises the following steps:
an acquisition unit configured to acquire verification image data and preprocess the verification image data;
the processing unit is further configured to input the preprocessed verification image data to the image aesthetics evaluation network model, and obtain verification score probability information of the verification image data through output of the image aesthetics evaluation network model;
the processing unit is further configured to acquire information of the image aesthetics score of the manual evaluation of the verification image data, construct a loss function according to the information of the image aesthetics score of the manual evaluation of the verification image data and the verification score probability information, and optimize the image aesthetics evaluation network model according to the loss function.
Preferably, the processing unit is specifically configured to determine whether the image aesthetics evaluation network model needs to be optimized according to an output value of the loss function;
and if optimization is needed, optimizing the image aesthetic evaluation network model.
Preferably, the acquiring unit is further configured to acquire test image data and preprocess the test image data;
the processing unit is further configured to input the preprocessed test image data to the image aesthetics evaluation network model, and obtain test score probability information of the test image data through output of the image aesthetics evaluation network model;
the determining unit is further configured to obtain information of an image aesthetics score of the manual evaluation of the test image data, and determine whether the image aesthetics evaluation network model is qualified according to the test score probability information of the test image data and the information of the image aesthetics score of the manual evaluation of the test image data;
the processing unit is further used for enabling the image aesthetic evaluation network model when the image aesthetic evaluation network model is determined to be qualified; retraining the image aesthetic evaluation network model when the image aesthetic evaluation network model is determined to be unqualified.
In a third aspect, an embodiment of the present application provides an electronic device, where the memory stores a computer program, and when the computer program is executed, the electronic device is caused to execute the method of any one of the above first aspects.
In a fourth aspect, an embodiment of the present application provides a storage medium storing a computer program, where the computer program is executed by a processor to implement the method of any one of the first aspect.
By adopting the scheme provided by the embodiment of the application, the target processing area is extracted from the image to be processed, the target processing area is input into the image aesthetic network model, the score probability information of the target processing area is output by the image aesthetic network model, and then the evaluation value of the target processing area can be determined according to the score probability information. According to the method and the device, the target processing area in the image to be processed is evaluated through the image aesthetic network model, information of the image can be mined from multiple angles, and then aesthetic scoring is carried out on the image, so that more accurate evaluation on the image is realized. Moreover, the process does not need manual participation, and the labor cost is reduced. The image can be stably and comprehensively evaluated through the image aesthetic network model, and the problem of poor consistency when the same image is subjected to aesthetic scoring in the prior art is solved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
Fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of another image processing method according to an embodiment of the present application;
fig. 3 is a schematic flowchart of another image processing method according to an embodiment of the present application;
fig. 4 is a schematic flowchart of another image processing method according to an embodiment of the present application;
fig. 5 is a schematic flowchart of another image processing method according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of another image processing apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For better understanding of the technical solutions of the present application, the following detailed descriptions of the embodiments of the present application are provided with reference to the accompanying drawings.
It should be understood that the embodiments described are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be understood that the term "and/or" as used herein is merely one type of associative relationship that describes an associated object, meaning that three types of relationships may exist, e.g., A and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
Before specifically describing the embodiments of the present application, terms applied or likely to be applied to the embodiments of the present application will be explained first.
Aesthetic scoring: the aesthetic of the picture is scored from a photographic aesthetic point of view.
The score probability information is used to characterize the probability of each aesthetic score of the target processing region of the image to be processed.
In the related art, the aesthetic picture scoring method requires that a panel member scores the picture through eye observation, the subjective performance of the assessment result is strong, different panel members score the same picture under the condition of no objective reference, and even the same panel member scores the same picture at different time periods. This may cause a problem of poor consistency when the same picture is aesthetically rated.
In view of the above problems, embodiments of the present application provide an image processing method, an image processing apparatus, an electronic device, and a storage medium, which are used to perform evaluation processing on a target processing region in an image to be processed through an image aesthetic network model, and can mine information of the image from multiple angles, so as to perform aesthetic scoring on the image, thereby implementing more accurate evaluation on the image. Moreover, the process does not need manual participation, and the labor cost is reduced. The image can be stably and comprehensively evaluated through the image aesthetic network model, and the problem of poor consistency when the same image is subjected to aesthetic scoring in the prior art is solved. The details will be described below.
Referring to fig. 1, a schematic diagram of an image processing method provided in an embodiment of the present application includes the following steps:
and step S101, extracting a target processing area of the image to be processed from the image to be processed.
Specifically, the image to be processed is shot or acquired from the user equipment according to the user requirement. The target processing region of the image to be processed means a region capable of reflecting the aesthetic characteristics of the image. The image aesthetic characteristics are features of the image at an aesthetic level, such as composition characteristics, color characteristics, depth of field characteristics, brightness characteristics, content characteristics, and the like. The composition characteristic describes a composition mode of an image, and specifically can be one-third composition, symmetrical composition, frame type composition, center composition, guideline composition, diagonal composition, triangle composition, balanced composition and the like; the color characteristics describe the color composition of the image, and specifically can be color temperature, hue, color composition, color co-scheduling and the like; the luminance characteristic describes the brightness level of the image; the depth of field particularly describes the clear range of a subject object in an image and the blurring degree of the background; the content properties describe the content contained by the image.
Because some image features irrelevant to aesthetics may exist in the image to be processed, the region capable of representing the aesthetic features in the image to be processed can be extracted, that is, the irrelevant image feature region is deleted in order to extract the target processing region.
Step S102, inputting a target processing area of an image to be processed into an image aesthetic evaluation network model, and outputting score probability information corresponding to the target processing area through the image aesthetic evaluation network model.
The score probability information is used to represent the probability of each aesthetic score of the target processing region of the image to be processed, and may be the probability of each aesthetic score of the target processing region, or the probability of each aesthetic score segment of the target processing region.
Specifically, the image aesthetic evaluation network model is a machine learning model with the capability of training image aesthetic scores. The image aesthetic evaluation network model can adopt a convolution neural network model to evaluate a target processing area of an image to be processed, and fractional probability information is output through a segmented probability density function fitter. The convolutional neural network model can adopt a residual model (RestNet), such as RestNet50, the RestNet50 model adopts four layers of blocks, feature extraction is carried out on block output of each layer, deep-layer features are combined with deep-layer and shallow-layer features, the deep-layer features have stronger expression capacity, but are not friendly to small objects, and the shallow-layer features well make up for the defect. The multi-scale feature extraction fusion adapts to the learning of objects with different sizes. And then fusing the extracted features of different blocks and storing the fused features locally. The extracted features for each image are then trained by a piecewise probability density function fitter that uses multiple fully connected layers, e.g., constructed using 3-layer FC and Softmax. And converting the training network into a classification problem through a segmented probability density function fitter, namely outputting the probability that the target processing region of the image to be processed falls into each aesthetic score stage for the image aesthetic evaluation network model.
That is to say, in this embodiment, the target processing region of the image to be processed is input to the image aesthetics evaluation network model, and after the evaluation processing of the image aesthetics evaluation network model, the image aesthetics evaluation network model outputs the score probability information of the target processing region, that is, the probability that the target processing region corresponds to the aesthetics score is output.
Step S103, determining the evaluation value of the target processing area according to the score probability information corresponding to the target processing area.
Specifically, after the score probability information corresponding to the target processing region is obtained through the image aesthetic evaluation network model, the probability of the aesthetic score corresponding to the target processing region can be obtained through the score probability information, so that the score value of the target processing region can be calculated according to the probability of the aesthetic score corresponding to the target processing region.
Further, since the score probability information may be the probability of each aesthetic score value of the target processing region, or the probability of each segment of the aesthetic score value of the target processing region, the score probability information may be different, and the corresponding manner of calculating the score value of the target processing region may also be different. The method comprises the following specific steps:
when the score probability information is the probability of each aesthetic score of the target processing region, it is described that the probability of each aesthetic score of the target processing region is directly output by the image aesthetic evaluation network model, and in this case, the probability can be directly used as the weight of each aesthetic score, and the probability of each aesthetic score is multiplied by each aesthetic score and then accumulated to obtain the evaluation value of the target processing region.
When the score probability information is the probability of each aesthetic score segment of the target processing region, it is described that the probability of each aesthetic score segment of the target processing region is directly output by the image aesthetic evaluation network model, and the probability of each aesthetic score is not directly output, at this time, a reference score is preset for each aesthetic score segment, so that the probabilities of each aesthetic score segment and the corresponding reference scores can be multiplied and accumulated to obtain the evaluation value of the target processing region.
Therefore, in the application, the target processing area in the image to be processed is evaluated through the image aesthetic network model, information of the image can be mined from multiple angles, and then the image is subjected to aesthetic scoring, so that the image is evaluated more accurately. Moreover, the process does not need manual participation, and the labor cost is reduced. The image can be stably and comprehensively evaluated through the image aesthetic network model, and the problem of poor consistency when the same image is subjected to aesthetic scoring in the prior art is solved.
Further, as shown in fig. 2, in the present application, before step S102, the method further includes:
and step S104, preprocessing the target processing area according to the network characteristics of the image aesthetic evaluation network model.
Specifically, based on the network characteristics, the image aesthetics evaluation network model may have certain requirements on the image it inputs, such as size requirements, color requirements, and the like. In order to meet the input requirement of the image aesthetic evaluation network model, before the target processing region extracted from the image to be processed is input into the image aesthetic evaluation network model, the target processing region of the image to be processed needs to be correspondingly preprocessed according to the network characteristics of the image aesthetic evaluation network model, for example, the target processing region needs to be subjected to size scaling according to the size requirement of the image aesthetic evaluation network model, and the target processing region needs to be subjected to color processing according to the color requirement of the image aesthetic evaluation network model.
At this time, step S102 inputs the target processing region to the image aesthetics evaluation network model, and outputting the score probability parameter information corresponding to the target processing region by the image aesthetics evaluation network model includes:
inputting the preprocessed target processing area into an image aesthetic evaluation network model, and outputting score probability parameter information corresponding to the target processing area through the image aesthetic evaluation network model.
Namely, after the target processing region is correspondingly preprocessed according to the network characteristics of the image aesthetic evaluation network model, the preprocessed target processing region is used as an input value of the image aesthetic evaluation network model and is input into the image aesthetic evaluation network model.
Further, as shown in fig. 3, the upper image aesthetic evaluation network model needs to be trained in advance, which is as follows:
step 201, collecting image data.
Specifically, before the image aesthetic evaluation network model is established, a large number of images need to be collected, and at this time, images including various landscapes, objects, human figures and the like can be collected according to requirements to acquire image data.
Step S202, preprocessing the image data.
Specifically, after the image data is collected, in order to make the image data satisfy the image aesthetics evaluation network model, at this time, preprocessing such as size scaling may be performed on the collected image data, so that the image data satisfies the input requirement of the image aesthetics evaluation network model.
And S203, training an image aesthetic evaluation network model based on the preprocessed image data.
Specifically, in the application, the image aesthetic evaluation network model may adopt a convolutional neural network model to evaluate a target processing region of an image to be processed, and output fractional probability information through a piecewise probability density function fitter. At this time, when the identification type convolutional neural network is selected to construct the image aesthetic evaluation network model, the convolutional neural network with a proper depth needs to be selected in consideration of factors such as the abstract meaning of the image, the processing mode of the texture details of the bottom layer of the image, the processing time, the processing efficiency, the storage space and the like. The convolutional neural network model may employ a residual model (RestNet), such as RestNet 50. The piecewise probability density function fitter may be constructed using 3-layer FC and Softmax to learn and fit fractional probability information.
After the construction of the image aesthetic evaluation network model is completed, the image aesthetic evaluation network model is trained through the preprocessed image data.
Further, training the image aesthetic evaluation network model based on the preprocessed image data includes:
and constructing an image aesthetic evaluation network model based on the recognition convolutional neural network. And inputting the preprocessed image data into an image aesthetic evaluation network model, and obtaining test score probability information of the image data through the output of the image aesthetic evaluation network model. Acquiring information of the image aesthetic score of the image data which is manually evaluated, constructing a loss function according to the test score probability information and the information of the image aesthetic score which is manually evaluated, and training the image aesthetic evaluation network model according to the loss function.
It should be noted that, after the image data is acquired, the aesthetic scoring needs to be manually performed on the acquired image data to obtain information of the manually evaluated aesthetic scoring score of the image.
After an image aesthetic evaluation network model is built through the recognition convolutional neural network, preprocessed partial image data are input into the built image aesthetic evaluation network model, the image aesthetic evaluation network model carries out evaluation processing on the input image data, and test score probability information corresponding to the image data is output. The method includes the steps of obtaining information of an image aesthetic evaluation score of manual evaluation corresponding to image data input into an image aesthetic evaluation network model, constructing a loss function through testing score probability information and the information of the image aesthetic evaluation score of manual evaluation, detecting whether an output value of the loss function is smaller than a first preset loss threshold value, adjusting parameters of the image aesthetic evaluation network model according to the output value of the loss function when the output value of the loss function is not smaller than the first preset loss threshold value, inputting preprocessed partial image data into the image aesthetic evaluation network model again, wherein the partial image data input into the image aesthetic evaluation network model can be image data of a previous image aesthetic evaluation network model or other image data, and the application does not limit the parameters. After the preprocessed partial image data is input into the image aesthetic evaluation network model, the image aesthetic evaluation network model evaluates the input image data, and test score probability information corresponding to the image data is output, namely the image aesthetic evaluation network model is trained through the input partial image data. After the image aesthetic evaluation network model outputs the test score probability information, inputting the test score probability information and the corresponding information of the manually-evaluated image aesthetic score to the loss function, detecting whether the output value of the loss function is smaller than a first preset loss threshold value, and realizing iterative training of the image aesthetic evaluation network model according to the loss function until the output value of the loss function is smaller than the first preset loss threshold value, namely convergence of the image aesthetic evaluation network model. When the output value of the loss function is smaller than the first preset loss threshold value, the image aesthetic evaluation network model at this time may be determined as a final image aesthetic evaluation network model.
It should be noted that the loss function is related to the network type of the image aesthetic evaluation network model and the input type thereof. For example, the function may be a cross entropy loss function, or a mean square error loss function, etc., which is not limited in this application.
It should be noted that, the first preset loss threshold is preset according to actual requirements, and the application is not limited to this.
Through the steps, the image aesthetic evaluation network model can be established and trained, so that the image aesthetic evaluation network model can output required fractional probability information to realize the aesthetic scoring of the image.
Further, in order to make the image aesthetic evaluation network model more accurate, in this application, as shown in fig. 4, the method further includes:
step S301, obtaining verification image data and preprocessing the verification image data.
Specifically, in step S203, part of the image data is input to the image aesthetics evaluation network model for training, and in this case, the image data that is not input to the image aesthetics evaluation network model for training in the collected image data may be used as the verification image data. The verification image data is correspondingly preprocessed according to the network characteristics of the image aesthetic evaluation network model, such as size scaling, color processing and the like.
Step S302, inputting the preprocessed verification image data into the image aesthetic evaluation network model, and obtaining verification score probability information of the verification image data through the output of the image aesthetic evaluation network model.
Specifically, after preprocessing, the preprocessed verification image data is used as input and input into the image aesthetic evaluation network model, and verification score probability information corresponding to the verification image data is output through evaluation processing of the image aesthetic evaluation network model.
Step S303, obtaining the information of the image aesthetic score of the artificial evaluation of the verification image data, constructing a loss function according to the information of the image aesthetic score of the artificial evaluation of the verification image data and the verification score probability information, and optimizing an image aesthetic evaluation network model according to the loss function.
Specifically, after the verification score probability information of the verification image data is obtained, the information of the manually evaluated image aesthetic score corresponding to the verification image data can be obtained according to the verification image data, a loss function is further constructed according to the verification score probability information and the information of the manually evaluated image aesthetic score corresponding to the verification image data, and the image aesthetic evaluation network model is optimized according to the output value of the loss function.
Further, optimizing the image aesthetic evaluation network model according to the loss function includes: determining whether the image aesthetic evaluation network model needs to be optimized according to the output value of the loss function; and if the optimization is needed, optimizing the image aesthetic evaluation network model.
That is, whether the output value of the loss function is smaller than a second preset loss threshold is detected, and if the output value of the loss function is smaller than the second preset loss threshold, it is indicated that the output value of the current image aesthetic evaluation network model is suitable for various types of image data, and optimization is not needed. If the loss is not less than the second preset loss threshold, the image aesthetic evaluation network model cannot be adapted to other types of image data, and it is determined that the image aesthetic evaluation network model needs to be optimized, and steps S202-S203 may be re-executed, so as to obtain a more accurate image aesthetic evaluation network model.
Through the steps, verification image data different from training image data of the verification image data can be adopted to realize optimization of the image aesthetic evaluation network model, so that the image aesthetic evaluation network model is more accurate.
Further, as shown in fig. 5, the present application further includes:
step S401, test image data are obtained, and the test image data are preprocessed.
Specifically, after the image aesthetic evaluation network model is output, the model needs to be tested to check whether the model is suitable for different types of images, that is, to verify whether the output result is accurate when different types of images are input to the image aesthetic evaluation network model. Based on this, test image data may be acquired, which may be pre-acquired containing different types of image data. After the test image data is acquired, the network characteristics of the network model are evaluated based on the image aesthetics, and the test image data is preprocessed, for example, by size scaling, color processing, and the like.
And S402, inputting the preprocessed test image data into the image aesthetic evaluation network model, and obtaining test score probability information of the test image data through the output of the image aesthetic evaluation network model.
Specifically, the processed test image data is used as input and input into the image aesthetic evaluation network model, evaluation processing is performed through the image aesthetic evaluation network model, and probability information representing the aesthetic scores of the test image data is output, namely the test score probability information of the output test image data.
Step S403, obtaining information of the image aesthetic score of the manual evaluation of the test image data, and determining whether the image aesthetic evaluation network model is qualified or not according to the test score probability information of the test image data and the information of the image aesthetic score of the manual evaluation of the test image data.
Specifically, when the test image data is collected, the test image data needs to be subjected to aesthetic scoring in advance in a manual evaluation mode, so as to obtain information of the manually evaluated image aesthetic scoring score. After the test score probability information of the test image data output by the image aesthetic evaluation network model is obtained, a loss function can be formed according to the test score probability information and the information of the image aesthetic score of the manual evaluation of the test image data, and whether the output value of the loss function is smaller than a third preset loss threshold value or not is detected. And determining whether the image aesthetic evaluation network model is qualified or not according to the detection result. That is, when the output value of the loss function is smaller than the third preset loss threshold, the image aesthetic evaluation network model is determined to be qualified. And when the output value of the loss function is not less than a third preset loss threshold value, determining that the image aesthetic evaluation network model is unqualified.
It should be noted that the following steps are performed differently depending on whether the image aesthetics evaluation network model is qualified or not. Upon determining that the image aesthetic evaluation network model is qualified, the following step S404 is performed. Upon determining that the image aesthetic evaluation network model is not qualified, the following step S405 is performed.
And S404, if the image is qualified, starting the image aesthetic evaluation network model.
Specifically, after the image aesthetic evaluation network model is determined to be qualified, if the image aesthetic evaluation network model can be normally used, the image aesthetic evaluation network model is started.
And S405, if the image aesthetic evaluation network model is not qualified, retraining the image aesthetic evaluation network model.
Specifically, when it is determined that the image aesthetic evaluation network model is not qualified, it indicates that the currently trained image aesthetic evaluation network model cannot adapt to different types of image data, and at this time, retraining is required. Therefore, the above steps S201 to S203 are re-executed until a qualified image aesthetic evaluation network model is acquired.
Therefore, in the application, the target processing area in the image to be processed can be evaluated through the image aesthetic network model, the information of the image can be mined from multiple angles, the image is further subjected to aesthetic scoring, and the image is evaluated more accurately. Moreover, the process does not need manual participation, and the labor cost is reduced. The image can be stably and comprehensively evaluated through the image aesthetic network model, and the problem of poor consistency when the same image is subjected to aesthetic scoring in the prior art is solved.
As shown in fig. 6, an embodiment of the present application provides an image processing apparatus including:
an extracting unit 501 is configured to extract a target processing region of an image to be processed from the image to be processed.
The processing unit 502 is configured to input a target processing area of an image to be processed into the image aesthetics evaluation network model, and output score probability information corresponding to the target processing area through the image aesthetics evaluation network model.
Wherein the score probability information is used to characterize the probability of each aesthetic score of the target processing region of the image to be processed.
A determining unit 503, configured to determine an evaluation value of the target processing region according to the score probability information corresponding to the target processing region.
Further, the processing unit 502 is further configured to pre-process the target processing area according to the network characteristics of the image aesthetic evaluation network model.
At this time, the processing unit 502 is configured to input the target processing area of the image to be processed into the image aesthetics evaluation network model, and the score probability information corresponding to the target processing area output by the image aesthetics evaluation network model specifically includes:
inputting the preprocessed target processing area into the image aesthetic evaluation network model, and outputting the score probability information corresponding to the target processing area through the image aesthetic evaluation network model.
Further, as shown in fig. 7, the image processing apparatus further includes:
an acquisition unit 504 for acquiring image data.
The processing unit 502 is further configured to perform preprocessing on the image data.
The processing unit 502 is further configured to train an image aesthetic evaluation network model based on the preprocessed image data.
Specifically, the processing unit 502 is specifically configured to construct an image aesthetic evaluation network model based on the recognition-based convolutional neural network. And inputting the preprocessed image data into an image aesthetic evaluation network model, and obtaining test score probability information of the image data through the output of the image aesthetic evaluation network model. Acquiring information of the image aesthetic score of the image data which is manually evaluated, constructing a loss function according to the test score probability information and the information of the image aesthetic score which is manually evaluated, and training an image aesthetic evaluation network model according to the loss function.
Further, as shown in fig. 7, the image processing apparatus further includes:
an obtaining unit 505 is configured to obtain verification image data and pre-process the verification image data.
The processing unit 502 is further configured to input the preprocessed verification image data into the image aesthetics evaluation network model, and obtain verification score probability information of the verification image data through output of the image aesthetics evaluation network model.
The processing unit 502 is further configured to obtain information of the image aesthetics score for verifying the manual evaluation of the image data, construct a loss function according to the information of the image aesthetics score for verifying the manual evaluation of the image data and the verification score probability information, and optimize the image aesthetics evaluation network model according to the function.
Specifically, the processing unit 502 is specifically configured to determine whether the image aesthetics evaluation network model needs to be optimized according to the output value of the loss function; and if the optimization is needed, optimizing the image aesthetic evaluation network model.
Further, the obtaining unit 505 is further configured to obtain test image data and perform preprocessing on the test image data.
The processing unit 502 is further configured to input the preprocessed test image data to the image aesthetics evaluation network model, and obtain test score probability information of the test image data through output of the image aesthetics evaluation network model.
The determining unit 503 is further configured to obtain information of the image aesthetics score of the manual evaluation of the test image data, and determine whether the image aesthetics evaluation network model is qualified according to the test score probability information of the test image data and the information of the image aesthetics score of the manual evaluation of the test image data.
The processing unit 502 is further configured to enable the image aesthetic evaluation network model when the image aesthetic evaluation network model is determined to be qualified; and retraining the image aesthetic evaluation network model after determining that the image aesthetic evaluation network model is unqualified.
Therefore, in the application, the target processing area in the image to be processed can be evaluated through the image aesthetic network model, the information of the image can be mined from multiple angles, the image is further subjected to aesthetic scoring, and the image is evaluated more accurately. Moreover, the process does not need manual participation, and the labor cost is reduced. The image can be stably and comprehensively evaluated through the image aesthetic network model, and the problem of poor consistency when the same image is subjected to aesthetic scoring in the prior art is solved.
Corresponding to the embodiment, the application further provides the electronic equipment. Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where the electronic device 800 may include: a processor 801, a memory 802, and a communication unit 803. The components communicate over one or more buses, and those skilled in the art will appreciate that the configuration of the servers shown in the figures are not meant to limit embodiments of the present invention, and may be in the form of buses, stars, more or fewer components than those shown, some components in combination, or a different arrangement of components.
The communication unit 803 is configured to establish a communication channel so that the storage device can communicate with other devices. Receiving the user data sent by other devices or sending the user data to other devices.
The processor 801, which is a control center of the storage device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and/or processes data by operating or executing software programs and/or modules stored in the memory 802 and calling data stored in the memory. The processor may be composed of Integrated Circuits (ICs), for example, a single packaged IC, or a plurality of packaged ICs connected to the same or different functions. For example, the processor 801 may include only a Central Processing Unit (CPU). In the embodiment of the present invention, the CPU may be a single operation core, or may include multiple operation cores.
The memory 802, for storing instructions executed by the processor 801, may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The execution instructions in the memory 802, when executed by the processor 801, enable the electronic device 800 to perform some or all of the steps in the embodiments described above.
In a specific implementation, the present invention further provides a computer storage medium, where the computer storage medium may store a program, and the program may include some or all of the steps in the embodiments of the image processing method provided by the present invention when executed. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a Random Access Memory (RAM), or the like.
Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented as software plus a required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.
The same and similar parts in the various embodiments in this specification may be referred to each other. Especially, as for the device embodiment and the terminal embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the description in the method embodiment.
Claims (10)
1. An image processing method, comprising:
extracting a target processing area of the image to be processed from the image to be processed;
inputting a target processing area of the image to be processed into an image aesthetic evaluation network model, and outputting score probability information corresponding to the target processing area through the image aesthetic evaluation network model; the score probability information is used for representing the probability of each aesthetic score of the target processing area of the image to be processed;
and determining the evaluation value of the target processing area according to the score probability information corresponding to the target processing area.
2. The method according to claim 1, before the inputting the target processing region into an image aesthetic evaluation network model and outputting the score probability information corresponding to the target processing region through the image aesthetic evaluation network model, further comprising:
preprocessing the target processing area according to the network characteristics of the image aesthetic evaluation network model;
the inputting the target processing region into an image aesthetic evaluation network model, and the outputting the score probability information corresponding to the target processing region through the image aesthetic evaluation network model includes:
inputting the preprocessed target processing area into an image aesthetic evaluation network model, and outputting score probability information corresponding to the target processing area through the image aesthetic evaluation network model.
3. The method of claim 1, further comprising:
collecting image data;
preprocessing the image data;
training an image aesthetic evaluation network model based on the preprocessed image data.
4. The method of claim 3, wherein training an image aesthetic evaluation network model based on the preprocessed image data comprises:
constructing an image aesthetic evaluation network model based on the recognition convolutional neural network;
inputting the preprocessed image data into the image aesthetic evaluation network model, and obtaining test score probability information of the image data through the output of the image aesthetic evaluation network model;
and acquiring information of the image aesthetic score of the image data which is manually evaluated, constructing a loss function according to the test score probability information and the information of the image aesthetic score which is manually evaluated, and training the image aesthetic evaluation network model according to the loss function.
5. The method of claim 4, further comprising:
acquiring verification image data and preprocessing the verification image data;
inputting the preprocessed verification image data into the image aesthetic evaluation network model, and obtaining verification score probability information of the verification image data through the output of the image aesthetic evaluation network model;
and obtaining the information of the image aesthetic score of the manual evaluation of the verification image data, constructing a loss function according to the information of the image aesthetic score of the manual evaluation of the verification image data and the verification score probability information, and optimizing the image aesthetic evaluation network model according to the loss function.
6. The method of claim 5, wherein the optimizing the image aesthetics evaluation network model according to the loss function comprises:
determining whether the image aesthetic evaluation network model needs to be optimized according to the output value of the loss function;
and if optimization is needed, optimizing the image aesthetic evaluation network model.
7. The method of any of claims 3-6, further comprising:
acquiring test image data, and preprocessing the test image data;
inputting the preprocessed test image data into the image aesthetic evaluation network model, and obtaining test score probability information of the test image data through the output of the image aesthetic evaluation network model;
acquiring information of an image aesthetic score of the manual evaluation of the test image data, and determining whether the image aesthetic evaluation network model is qualified or not according to the test score probability information of the test image data and the information of the image aesthetic score of the manual evaluation of the test image data;
if the image is qualified, starting the image aesthetic evaluation network model;
and if the image is not qualified, retraining the image aesthetic evaluation network model.
8. An image processing apparatus characterized by comprising:
the extraction unit is used for extracting a target processing area of the image to be processed from the image to be processed;
the processing unit is used for inputting a target processing area of the image to be processed into an image aesthetic evaluation network model and outputting score probability information corresponding to the target processing area through the image aesthetic evaluation network model; the score probability information is used for representing the probability of each aesthetic score of the target processing area of the image to be processed;
and the determining unit is used for determining the evaluation value of the target processing area according to the score probability information corresponding to the target processing area.
9. An electronic device, comprising a processor and a memory, the memory storing a computer program that, when executed, causes the electronic device to perform the method of any of claims 1-7.
10. A storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110519176.8A CN113255743A (en) | 2021-05-12 | 2021-05-12 | Image processing method, image processing device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110519176.8A CN113255743A (en) | 2021-05-12 | 2021-05-12 | Image processing method, image processing device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113255743A true CN113255743A (en) | 2021-08-13 |
Family
ID=77223248
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110519176.8A Pending CN113255743A (en) | 2021-05-12 | 2021-05-12 | Image processing method, image processing device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113255743A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109063778A (en) * | 2018-08-09 | 2018-12-21 | 中共中央办公厅电子科技学院 | A kind of image aesthetic quality determines method and system |
| CN110689523A (en) * | 2019-09-02 | 2020-01-14 | 西安电子科技大学 | Personalized image information evaluation method based on meta-learning and information data processing terminal |
| CN110782448A (en) * | 2019-10-25 | 2020-02-11 | 广东三维家信息科技有限公司 | Rendered image evaluation method and device |
| CN112560903A (en) * | 2020-12-01 | 2021-03-26 | 汉海信息技术(上海)有限公司 | Method, device and equipment for determining image aesthetic information and storage medium |
-
2021
- 2021-05-12 CN CN202110519176.8A patent/CN113255743A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109063778A (en) * | 2018-08-09 | 2018-12-21 | 中共中央办公厅电子科技学院 | A kind of image aesthetic quality determines method and system |
| CN110689523A (en) * | 2019-09-02 | 2020-01-14 | 西安电子科技大学 | Personalized image information evaluation method based on meta-learning and information data processing terminal |
| CN110782448A (en) * | 2019-10-25 | 2020-02-11 | 广东三维家信息科技有限公司 | Rendered image evaluation method and device |
| CN112560903A (en) * | 2020-12-01 | 2021-03-26 | 汉海信息技术(上海)有限公司 | Method, device and equipment for determining image aesthetic information and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110533097B (en) | Image definition recognition method and device, electronic equipment and storage medium | |
| US20220092882A1 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| CN107330437B (en) | Feature extraction method based on convolutional neural network target real-time detection model | |
| EP3989104A1 (en) | Facial feature extraction model training method and apparatus, facial feature extraction method and apparatus, device, and storage medium | |
| CN109389135B (en) | Image screening method and device | |
| CN110163213B (en) | Remote sensing image segmentation method based on disparity map and multi-scale depth network model | |
| CN107273870A (en) | The pedestrian position detection method of integrating context information under a kind of monitoring scene | |
| CN110059666B (en) | Attention detection method and device | |
| CN110674759A (en) | Monocular face in-vivo detection method, device and equipment based on depth map | |
| CN109919252A (en) | The method for generating classifier using a small number of mark images | |
| CN108446688B (en) | Face image gender judgment method and device, computer equipment and storage medium | |
| CN115131283A (en) | Defect detection and model training method, device, equipment and medium for target object | |
| CN114066812B (en) | No-reference image quality evaluation method based on spatial attention mechanism | |
| CN112446322A (en) | Eyeball feature detection method, device, equipment and computer-readable storage medium | |
| CN112801911A (en) | Method and device for removing Chinese character noise in natural image and storage medium | |
| CN114663662B (en) | Hyper-parameter searching method, device, computer equipment and storage medium | |
| CN116385430A (en) | Machine vision flaw detection method, device, medium and equipment | |
| CN115965934A (en) | Parking space detection method and device | |
| CN111435445A (en) | Training method and device of character recognition model and character recognition method and device | |
| CN106934373A (en) | A kind of library book damages assessment method and system | |
| CN115410240A (en) | Intelligent face pockmark and color spot analysis method and device and storage medium | |
| CN113034446A (en) | Automatic transformer substation equipment defect identification method and system | |
| CN115115552B (en) | Image correction model training method, image correction device and computer equipment | |
| CN110705398A (en) | Mobile-end-oriented test paper layout image-text real-time detection method | |
| CN108665455B (en) | Method and device for evaluating image significance prediction result |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210813 |