CN113592812B - Sketch picture evaluation method and device - Google Patents
Sketch picture evaluation method and device Download PDFInfo
- Publication number
- CN113592812B CN113592812B CN202110866638.3A CN202110866638A CN113592812B CN 113592812 B CN113592812 B CN 113592812B CN 202110866638 A CN202110866638 A CN 202110866638A CN 113592812 B CN113592812 B CN 113592812B
- Authority
- CN
- China
- Prior art keywords
- sketch picture
- sketch
- feature
- training
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 73
- 238000012549 training Methods 0.000 claims abstract description 67
- 238000013210 evaluation model Methods 0.000 claims abstract description 51
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims abstract description 48
- 230000004927 fusion Effects 0.000 claims abstract description 37
- 238000000034 method Methods 0.000 claims abstract description 23
- 238000013135 deep learning Methods 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 19
- 238000012360 testing method Methods 0.000 claims description 15
- 230000008859 change Effects 0.000 claims description 12
- 210000002569 neuron Anatomy 0.000 claims description 11
- 230000004913 activation Effects 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 9
- 235000004257 Cordia myxa Nutrition 0.000 claims description 8
- 244000157795 Cordia myxa Species 0.000 claims description 8
- 238000002790 cross-validation Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000005259 measurement Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 241000282414 Homo sapiens Species 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000008034 disappearance Effects 0.000 description 2
- 238000011478 gradient descent method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000017074 necrotic cell death Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000016273 neuron death Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Abstract
The invention relates to a sketch picture evaluation method and device. The sketch picture evaluation method provided by the invention comprises the following steps: constructing a sketch picture evaluation model, wherein the sketch picture evaluation model comprises a plurality of residual blocks, a feature fusion layer, a first full-connection layer, a second full-connection layer and a classification layer which are sequentially connected; collecting sketch picture works and corresponding evaluations, and constructing a training data set; performing deep learning training on the sketch picture evaluation model by using the training data set to obtain a trained sketch picture evaluation model; inputting the sketch picture to be evaluated into the trained sketch picture evaluation model to obtain an evaluation result of the sketch picture. The sketch picture evaluation method has the advantages that manual participation is not needed in the judgment process, and the classification evaluation result of the sketch picture is more accurate. Corresponding to the method, the invention also provides a sketch picture evaluation device.
Description
Technical Field
The invention relates to the technical field of image classification, in particular to a sketch picture evaluation method and device.
Background
In art teaching work, it is a very important work to evaluate works created by students. In reality teaching, teachers often divide works into excellent, good, medium and bad categories with respect to the merits of visual features such as composition, layout, lines, contours and the like of sketched works. Because of the complexity of the images and the subjectivity of human aesthetic, the realization of automatic evaluation of the quality of sketched images becomes a very challenging task by simulating the evaluation standard of teachers through a computer.
The existing art work classification method goes through 2 stages of the traditional manual feature extraction method and the deep learning method. The traditional manual feature extraction method needs that after a researcher performs a great deal of research on the researched art works or famous universities, a special algorithm can be designed to extract the features of the art works, and then the art works are classified by adopting methods such as machine learning and the like. However, the high-level features of the artwork cannot be perfectly represented by simple manual feature extraction methods, and such methods are not end-to-end and are not as convenient as deep learning methods in practical use.
The deep learning method is superior to the traditional feature extraction algorithm in accuracy and generalization capability because the features learned in the training process are higher and abstract and are closer to the visual feature principle of human beings.
Disclosure of Invention
Based on the above, the invention aims to provide a sketch picture evaluation method and device, wherein a feature fusion layer and a new full-connection layer are added into an original ResNet neural network, the judgment process does not need to be manually participated, and the classification evaluation result of the sketch picture is more accurate.
In a first aspect, the present invention provides a sketch picture evaluation method, including the steps of:
Constructing a sketch picture evaluation model, wherein the sketch picture evaluation model comprises a plurality of residual blocks, a feature fusion layer, a first full-connection layer, a second full-connection layer and a classification layer which are sequentially connected;
collecting sketch picture works and corresponding evaluations, and constructing a training data set;
performing deep learning training on the sketch picture evaluation model by using the training data set to obtain a trained sketch picture evaluation model;
inputting the sketch picture to be evaluated into the trained sketch picture evaluation model to obtain an evaluation result of the sketch picture.
Further, performing deep learning training on the sketch picture evaluation model by using the training data set comprises the following steps:
Randomly dividing the data set into 5 folds, wherein 1 fold is a test set, and 4 fold is a training set;
Inputting a training set into the sketch picture evaluation model for training, calculating a loss value and reversely transmitting and updating model parameters, and testing the accuracy of the model by using a testing set;
performing 5-fold cross validation, repeating training for 4 times, sequentially selecting an untrained 1-fold data set in the training set as a test set, and taking the other data sets as training sets;
And obtaining the measurement standard of the accuracy of the sketch picture evaluation model according to the average accuracy of 5-fold cross validation.
Further, collecting sketch picture works and corresponding evaluations, and constructing a training data set, including:
obtaining classification evaluation of sketch picture works;
preprocessing the images of the sketch picture works to obtain a data set with consistent image sizes;
and performing up-sampling operation and data enhancement operation on the data set to double the scale of the data set, thereby obtaining the training data set.
Further, performing a data enhancement operation on the data set, including:
Carrying out graying, brightness change, random rotation, horizontal overturning and vertical overturning change treatment on the picture;
Wherein, the graying probability is 0.1, the brightness variation amplitude is 0.2, the random rotation parameter is set to 30 degrees, the probability of horizontal overturn is 0.5, and the probability of vertical overturn is 0.5.
Further, the number of residual blocks is 4.
Further, inputting the sketch picture to be evaluated into the trained sketch picture evaluation model to obtain an evaluation result of the sketch picture, including:
sequentially inputting the sketch pictures to be evaluated into 4 residual blocks, so that the 4 residual blocks sequentially carry out convolution extraction on the sketch pictures;
Extracting a second feature extracted from a second residual block, using a convolution kernel to change the dimension and the size of the second feature to be the same as those of a fourth feature extracted from a fourth residual block, and inputting the fourth feature and the changed second feature into the feature fusion layer for splicing to obtain a fusion feature;
and sequentially inputting the fusion characteristics into the first full-connection layer, the second full-connection layer and the classification layer to obtain an evaluation result of the sketch picture.
Further, the number of neurons of the second fully-connected layer is one fourth of the number of neurons of the first fully-connected layer.
Further, the dropout probability of the second full connection layer is 0.5.
Further, the activation function of the second full connection layer is SELU activation functions.
In a second aspect, the present invention also provides a sketch picture evaluation device, including:
The system comprises a model construction module, a sketch picture evaluation module and a classification module, wherein the model construction module is used for constructing a sketch picture evaluation model, and the sketch picture evaluation model comprises a plurality of residual blocks, a characteristic fusion layer, a first full-connection layer, a second full-connection layer and a classification layer which are sequentially connected;
The data set construction module is used for collecting sketch picture works and corresponding evaluations and constructing a training data set;
the model training module is used for performing deep learning training on the sketch picture evaluation model by using the training data set to obtain a trained sketch picture evaluation model;
and the sketch picture evaluation module is used for inputting the sketch picture to be evaluated into the trained sketch picture evaluation model to obtain an evaluation result of the sketch picture.
According to the sketch picture evaluation method and device provided by the invention, improvement is made based on ResNet network, the feature fusion step is added, and the effectiveness of classification information is improved; 1 new full-connection layer is added between the original full-connection layer and the classification layer, and a dropout technology is added into neurons of the new connection layer, so that the training speed of the model is increased; when the data set is acquired, the sketch works are rated into 4 categories of excellent, good, medium and bad according to the examination standard by an art teacher, so that the final classification evaluation result meets the human aesthetic standard. After model training is completed, the whole judging process of the sketch picture to be evaluated does not need to be manually participated, and the full-automatic sketch picture quality classification of the machine is realized.
For a better understanding and implementation, the present invention is described in detail below with reference to the drawings.
Drawings
FIG. 1 is a schematic flow chart of a sketch picture evaluation method provided by the invention;
FIG. 2 is a schematic diagram of a sketch picture evaluation model used in the present invention;
Fig. 3 is a schematic structural diagram of a sketch picture evaluation device provided by the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the following detailed description of the embodiments of the present application will be given with reference to the accompanying drawings.
It should be understood that the described embodiments are merely some, but not all embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the application, are intended to be within the scope of the embodiments of the present application.
The terminology used in the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of embodiments of the application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the application as detailed in the accompanying claims. In the description of the present application, it should be understood that the terms "first," "second," "third," and the like are used merely to distinguish between similar objects and are not necessarily used to describe a particular order or sequence, nor should they be construed to indicate or imply relative importance. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art according to the specific circumstances.
Furthermore, in the description of the present application, unless otherwise indicated, "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
In view of the problems in the background art, in a first aspect, the present invention provides a sketch picture evaluation method, as shown in fig. 1, including the following steps:
S1: and constructing a sketch picture evaluation model, wherein the sketch picture evaluation model comprises a plurality of residual blocks, a feature fusion layer, a first full-connection layer, a second full-connection layer and a classification layer which are sequentially connected.
According to the sketch picture evaluation method provided by the invention, the used sketch picture evaluation model is based on the improvement of ResNet networks. Resnet is an abbreviation for Residual Network (Residual Network) which is widely used in the field of object classification and the like and as part of the classical neural Network of the backbone of computer vision tasks, typical networks are Resnet, resnet101 and the like. The proof network of Resnet networks can evolve towards deeper (containing more hidden layers).
The ResNet network structure contains 4 residual blocks, 1 full connection layer and 1 classification layer. The residual block is used for extracting convolution characteristics of the image, the full connection layer is used for integrating the characteristics extracted before, and the classification layer is used for outputting a final classification result. Images can only be classified using features that pass through the last convolution block into the classification layer when transmitted in ResNet's network.
Aiming at the problem, the invention uses a feature fusion method to extract the convolution group features extracted from the second residual block, obtains the same dimension and size as the convolution group features extracted from the fourth residual block through dimension and size change, splices the two, and inputs the two to the full connection layer and the classification layer for classification. The feature fusion can increase the effectiveness of classification information, and some originally useful information can not be discarded, so that the classification accuracy can be improved.
In addition, the sketch picture evaluation model used in the invention also adds a new full-connection layer (namely a second full-connection layer) between the full-connection layer (namely a first full-connection layer) and the classification layer of the ResNet network. The number of neurons of the second fully-connected layer is one fourth of the number of neurons of the original fully-connected layer. Meanwhile, dropout with rate=0.5 was used in information propagation, and half of neurons were disabled randomly.
After dropout, the second fully connected layer also selects SELU functions as the activation functions.
The common activation function is RELU function, which has the advantages of solving the gradient disappearance and high calculation efficiency, but the negative number part is always 0, which is called unilateral inhibition, and when the learning rate is large, the neural unit necrosis can occur, and the problem of no response to data exists. The SELU function does not have dead zone, compared with the RELU function, the problem of neuron death is avoided, and the problems of gradient disappearance and gradient explosion are avoided.
The SELU function is expressed as follows:
Where α and λ are fixed parameters of the SELU function, the values of α and λ are both demonstrated.
α＝1.6732632423543772848170429916717
λ＝1.0507009873554804934193349852946
After all the above modifications, in a preferred embodiment, the sketch picture evaluation model used in the present invention is as shown in fig. 2, and includes:
4 residual blocks for sequentially extracting the characteristics of the pictures in the input model;
the feature fusion layer is used for splicing and fusing the features extracted by the second residual block and the features extracted by the fourth residual block;
A first full-connection layer for integrating previously extracted features;
A second fully connected layer, wherein the dropout probability is 0.5 and the activation function is SELU activation functions;
and the classification layer is used for outputting the final classification evaluation result.
S2: and collecting sketch picture works and corresponding evaluations, and constructing a training data set.
In order to make the classification evaluation result more in line with the aesthetic human standard, in a preferred embodiment, after the sketch pictures are collected, the art teacher judges the quality of the works according to the examination standard, and the works are classified into 4 categories of excellent, good, medium and bad.
And then, scanning the picture to obtain an electronic image, and carrying out data preprocessing. All images first need to be resized to 400 x 400 for subsequent further processing and computation.
Because the data set is a realistic sketch work, normal distribution is met, few excellent and ineligible works are obtained, and more medium works are obtained. In order to better cope with the unbalanced problem, the invention performs simple data up-sampling and copies the samples of few classes at the data level.
In order to solve the problem of insufficient number of training samples, the invention uses a data enhancement technology to carry out graying, brightness change, random rotation, horizontal overturn and vertical overturn change on the picture, so that the scale of the data set is doubled. Wherein, the graying probability of data enhancement is 0.1, the brightness variation amplitude is 0.2, the random rotation parameter is set to 30 degrees, the probability of horizontal overturn is 0.5, and the probability of vertical overturn is 0.5.
S3: and performing deep learning training on the sketch picture evaluation model by using the training data set to obtain a trained sketch picture evaluation model.
In the model training process, a 5-fold cross validation method is used, and a training set and a testing set are respectively used according to 4:1, performing 5 times of training, and taking the average value of the results of the 5 times of training as a standard for measuring the accuracy of the model.
And inputting a training set into the sketch picture evaluation model for deep learning training, calculating a loss value and carrying out back propagation to update model parameters every time of training.
In the training process, after the classification result is obtained after output from the classification layer, the gradient descent method is used for carrying out back propagation, and the network parameters are updated.
Wherein the gradient descent method comprises the following steps:
where θj is a parameter of the neural network, J (θ) is a loss function, and α is a learning rate. θ refers to all neural network parameters, and θj is a parameter with subscript j.
In the algorithm level, the invention uses the Focal loss, reduces the loss of a simple sample, improves the contribution degree of a difficult sample to the gradient, and the Focal loss function is expressed as follows:
FL(pt)＝-αt(1-pt)γlog(pt)
Where α t and γ are fixed parameters of the Focal loss function, α t＝0.25,γ＝2,pt represents the classification probability.
And inputting the test data set into a deep learning network to test the precision of the model.
S4: inputting the sketch picture to be evaluated into the trained sketch picture evaluation model to obtain an evaluation result of the sketch picture.
In a preferred embodiment, the number of residual blocks is 4. The process for evaluating the sketch picture to be evaluated by using the sketch picture evaluation model specifically comprises the following steps:
s401: and sequentially inputting the sketch pictures to be evaluated into 4 residual blocks, so that the 4 residual blocks sequentially carry out convolution extraction on the sketch pictures to obtain features.
S402: and extracting a second feature extracted from the second residual block, using a convolution kernel to change the dimension and the size of the second feature to be the same as those of a fourth feature extracted from a fourth residual block, and inputting the fourth feature and the changed second feature into the feature fusion layer for splicing to obtain a fusion feature.
Feature fusion in the present invention can be expressed as:
Wherein Z represents a fusion feature; x 1,X2,...Xc1 and Y 1,Y2,...Yc2 represent convolution graphs extracted from the second residual block and the fourth residual block of the network model, respectively, and K 1、K2; i represents a subscript, c1 represents a second residual block convolution feature map number, and c2 represents a fourth residual block convolution feature map number; the number of convolution feature maps = the number of convolution kernels, how many feature maps are per residual block, and how many are c.
S403: and sequentially inputting the fusion characteristics into the first full-connection layer, the second full-connection layer and the classification layer to obtain an evaluation result of the sketch picture.
In a second aspect, corresponding to the foregoing method, the present invention further provides a sketch picture evaluation device, where the structure of the device is shown in fig. 3, and the device includes:
The system comprises a model construction module, a sketch picture evaluation module and a classification module, wherein the model construction module is used for constructing a sketch picture evaluation model, and the sketch picture evaluation model comprises a plurality of residual blocks, a characteristic fusion layer, a first full-connection layer, a second full-connection layer and a classification layer which are sequentially connected;
The data set construction module is used for collecting sketch picture works and corresponding evaluations and constructing a training data set;
the model training module is used for performing deep learning training on the sketch picture evaluation model by using the training data set to obtain a trained sketch picture evaluation model;
and the sketch picture evaluation module is used for inputting the sketch picture to be evaluated into the trained sketch picture evaluation model to obtain an evaluation result of the sketch picture.
Preferably, the model training module comprises:
the diversity unit is used for randomly dividing the data set into 5 folds, wherein 1 fold is a test set, and 4 fold is a training set;
the training unit is used for inputting a training set into the sketch picture evaluation model for training, calculating a loss value and carrying out back propagation to update model parameters, and testing the precision of the model by using a testing set; performing 5-fold cross validation, repeating training for 4 times, sequentially selecting an untrained 1-fold data set in the training set as a test set, and other data sets as training sets;
And the measurement standard acquisition unit is used for obtaining the measurement standard of the accuracy of the sketch picture evaluation model according to the average accuracy of the 5-fold cross validation.
Preferably, the data set construction module includes:
the evaluation acquisition unit is used for acquiring the classification evaluation of the sketch picture works;
the preprocessing unit is used for preprocessing the images of the sketch picture works to obtain a data set with consistent image sizes;
and the data set enhancement unit is used for carrying out up-sampling operation and data enhancement operation on the data set, doubling the scale of the data set and obtaining the training data set.
Preferably, the data enhancement operation is performed on the data set, including:
Carrying out graying, brightness change, random rotation, horizontal overturning and vertical overturning change treatment on the picture;
Wherein, the graying probability is 0.1, the brightness variation amplitude is 0.2, the random rotation parameter is set to 30 degrees, the probability of horizontal overturn is 0.5, and the probability of vertical overturn is 0.5.
Preferably, the number of residual blocks is 4.
Preferably, the sketch picture evaluation module includes:
The feature extraction unit is used for sequentially inputting the sketch pictures to be evaluated into 4 residual blocks, so that the 4 residual blocks sequentially convolve and extract features of the sketch pictures;
The feature fusion unit is used for extracting a second feature extracted by a second residual block, changing the dimension and the size of the second feature to be the same as those of a fourth feature extracted by a fourth residual block by using a convolution kernel, and inputting the fourth feature and the changed second feature into the feature fusion layer for splicing to obtain a fusion feature;
and the evaluation result acquisition unit is used for inputting the fusion characteristics into the first full-connection layer, the second full-connection layer and the classification layer in sequence to obtain the evaluation result of the sketch picture.
Preferably, the number of neurons of the second fully-connected layer is one fourth of the number of neurons of the first fully-connected layer.
Preferably, the dropout probability of the second full connection layer is 0.5.
Preferably, the activation function of the second full connection layer is SELU activation functions.
According to the sketch picture evaluation method and device provided by the invention, improvement is made based on ResNet network, the feature fusion step is added, and the effectiveness of classification information is improved; 1 new full-connection layer is added between the original full-connection layer and the classification layer, and a dropout technology is added into neurons of the new connection layer, so that the training speed of the model is increased; when the data set is acquired, the sketch works are rated into 4 categories of excellent, good, medium and bad according to the examination standard by an art teacher, so that the final classification evaluation result meets the human aesthetic standard. After model training is completed, the whole judging process of the sketch picture to be evaluated does not need to be manually participated, and the full-automatic sketch picture quality classification of the machine is realized.
The above examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention.
Claims (8)
1. The sketch picture evaluation method is characterized by comprising the following steps of:
Constructing a sketch picture evaluation model, wherein the sketch picture evaluation model comprises 4 residual blocks, a feature fusion layer, a first full-connection layer, a second full-connection layer and a classification layer which are sequentially connected;
collecting sketch picture works and corresponding evaluations, and constructing a training data set;
performing deep learning training on the sketch picture evaluation model by using the training data set to obtain a trained sketch picture evaluation model;
Inputting a sketch picture to be evaluated into the trained sketch picture evaluation model to obtain an evaluation result of the sketch picture;
Inputting a sketch picture to be evaluated into the trained sketch picture evaluation model to obtain an evaluation result of the sketch picture, wherein the method comprises the following steps of:
sequentially inputting the sketch pictures to be evaluated into 4 residual blocks, so that the 4 residual blocks sequentially carry out convolution extraction on the sketch pictures;
Extracting a second feature extracted from a second residual block, using a convolution kernel to change the dimension and the size of the second feature to be the same as those of a fourth feature extracted from a fourth residual block, and inputting the fourth feature and the changed second feature into the feature fusion layer for splicing to obtain a fusion feature;
inputting the fusion characteristics into a first full-connection layer, a second full-connection layer and a classification layer in sequence to obtain an evaluation result of the sketch picture;
The formula for acquiring the fusion characteristics by the characteristic fusion layer is as follows:
In the method, in the process of the invention, Representing a fusion feature;，，…， And ，，…，Respectively representing convolution graphs extracted by a second residual block and a fourth residual block of the network model,、… Denotes the number of convolution kernels,Representing a convolution operation; The subscript, c1, and c2 represent the second and fourth residual block convolution feature figures, respectively.
2. The sketch picture evaluation method according to claim 1, wherein the deep learning training of the sketch picture evaluation model using the training data set includes:
Randomly dividing the data set into 5 folds, wherein 1 fold is a test set, and 4 fold is a training set;
Inputting a training set into the sketch picture evaluation model for training, calculating a loss value and reversely transmitting and updating model parameters, and testing the accuracy of the model by using a testing set;
performing 5-fold cross validation, repeating training for 4 times, sequentially selecting an untrained 1-fold data set in the training set as a test set, and taking the other data sets as training sets;
And obtaining the measurement standard of the accuracy of the sketch picture evaluation model according to the average accuracy of 5-fold cross validation.
3. The sketch picture evaluation method of claim 1, wherein collecting sketch picture works and corresponding evaluations and constructing a training dataset comprises:
obtaining classification evaluation of sketch picture works;
preprocessing the images of the sketch picture works to obtain a data set with consistent image sizes;
and performing up-sampling operation and data enhancement operation on the data set to double the scale of the data set, thereby obtaining the training data set.
4. A sketch picture evaluation method as claimed in claim 3 characterized in that the data enhancement operation on the data set comprises:
Carrying out graying, brightness change, random rotation, horizontal overturning and vertical overturning change treatment on the picture;
Wherein, the graying probability is 0.1, the brightness variation amplitude is 0.2, the random rotation parameter is set to 30 degrees, the probability of horizontal overturn is 0.5, and the probability of vertical overturn is 0.5.
5. The sketch picture evaluation method of claim 1, wherein the sketch picture evaluation method is characterized by:
The number of the neurons of the second full-connection layer is one fourth of the number of the neurons of the first full-connection layer.
6. The sketch picture evaluation method of claim 1, wherein the sketch picture evaluation method is characterized by:
the dropout probability of the second full connection layer is 0.5.
7. The sketch picture evaluation method of claim 1, wherein the sketch picture evaluation method is characterized by:
The activation function of the second fully-connected layer is SELU activation functions.
8. A sketch picture evaluation device, characterized by comprising:
The system comprises a model construction module, a sketch picture evaluation module and a classification module, wherein the model construction module is used for constructing a sketch picture evaluation model, and the sketch picture evaluation model comprises 4 residual blocks, a feature fusion layer, a first full-connection layer, a second full-connection layer and a classification layer which are connected in sequence;
The data set construction module is used for collecting sketch picture works and corresponding evaluations and constructing a training data set;
the model training module is used for performing deep learning training on the sketch picture evaluation model by using the training data set to obtain a trained sketch picture evaluation model;
The sketch picture evaluation module is used for inputting a sketch picture to be evaluated into the trained sketch picture evaluation model to obtain an evaluation result of the sketch picture;
wherein, sketch picture evaluation module includes:
The feature extraction unit is used for sequentially inputting the sketch pictures to be evaluated into 4 residual blocks, so that the 4 residual blocks sequentially convolve and extract features of the sketch pictures;
The feature fusion unit is used for extracting a second feature extracted by a second residual block, changing the dimension and the size of the second feature to be the same as those of a fourth feature extracted by a fourth residual block by using a convolution kernel, and inputting the fourth feature and the changed second feature into the feature fusion layer for splicing to obtain a fusion feature;
The evaluation result acquisition unit is used for inputting the fusion characteristics into the first full-connection layer, the second full-connection layer and the classification layer in sequence to obtain an evaluation result of the sketch picture;
The formula for acquiring the fusion characteristics by the characteristic fusion layer is as follows:
In the method, in the process of the invention, Representing a fusion feature;，，…， And ，，…，Respectively representing convolution graphs extracted by a second residual block and a fourth residual block of the network model,、… Denotes the number of convolution kernels,Representing a convolution operation; The subscript, c1, and c2 represent the second and fourth residual block convolution feature figures, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110866638.3A CN113592812B (en) | 2021-07-29 | Sketch picture evaluation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110866638.3A CN113592812B (en) | 2021-07-29 | Sketch picture evaluation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113592812A CN113592812A (en) | 2021-11-02 |
| CN113592812B true CN113592812B (en) | 2024-06-28 |
Family
ID=
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112016574A (en) * | 2020-10-22 | 2020-12-01 | 北京科技大学 | Image classification method based on feature fusion |
| CN112668638A (en) * | 2020-12-25 | 2021-04-16 | 山东大学 | Image aesthetic quality evaluation and semantic recognition combined classification method and system |
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112016574A (en) * | 2020-10-22 | 2020-12-01 | 北京科技大学 | Image classification method based on feature fusion |
| CN112668638A (en) * | 2020-12-25 | 2021-04-16 | 山东大学 | Image aesthetic quality evaluation and semantic recognition combined classification method and system |
Non-Patent Citations (3)
| Title |
|---|
| Sketch works ranking based on improved transfer learning model;SongSen Yu et al;Multimedia Tools and Applications;20210831;第80卷;33663–33678 * |
| 基于改进yolo v3的电连接器缺陷检测;吴伟浩 等;传感技术学报;20200229;第33卷(第2期);299-307 * |
| 基于语义信息跨层特征融合的细粒度鸟类识别;李国瑞 等;计算机应用与软件;20200430;第37卷(第4期);132-136、191 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110210486B (en) | Sketch annotation information-based generation countermeasure transfer learning method | |
| CN107341518A (en) | A kind of image classification method based on convolutional neural networks | |
| CN107123111B (en) | Deep residual error network construction method for mobile phone screen defect detection | |
| CN106780466A (en) | A kind of cervical cell image-recognizing method based on convolutional neural networks | |
| CN108154504A (en) | Method for detecting surface defects of steel plate based on convolutional neural network | |
| CN106980858A (en) | The language text detection of a kind of language text detection with alignment system and the application system and localization method | |
| CN106920215A (en) | A kind of detection method of panoramic picture registration effect | |
| CN108289222A (en) | A kind of non-reference picture quality appraisement method mapping dictionary learning based on structural similarity | |
| CN108665005A (en) | A method of it is improved based on CNN image recognition performances using DCGAN | |
| CN106548169A (en) | Fuzzy literal Enhancement Method and device based on deep neural network | |
| CN104866868A (en) | Metal coin identification method based on deep neural network and apparatus thereof | |
| CN112257741B (en) | Method for detecting generative anti-false picture based on complex neural network | |
| CN111339935B (en) | Optical remote sensing picture classification method based on interpretable CNN image classification model | |
| CN110245711A (en) | The SAR target identification method for generating network is rotated based on angle | |
| CN106651887A (en) | Image pixel classifying method based convolutional neural network | |
| CN111161224A (en) | Casting internal defect grading evaluation system and method based on deep learning | |
| CN110751644A (en) | Road surface crack detection method | |
| CN116152209A (en) | Earphone cover defect detection method, device, equipment and storage medium | |
| CN113392930A (en) | Traffic sign target detection method based on multi-level divide-and-conquer network | |
| CN113592812B (en) | Sketch picture evaluation method and device | |
| CN112818777A (en) | Remote sensing image target detection method based on dense connection and feature enhancement | |
| CN115346091B (en) | Method and device for generating Mura defect image data set | |
| CN113496485B (en) | Satellite remote sensing image quality evaluation method and device | |
| CN114529689B (en) | Ceramic cup defect sample amplification method and system based on antagonistic neural network | |
| CN106682604A (en) | Method for detecting blurred image based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |