CN111127364B - Image data enhancement strategy selection method and face recognition image data enhancement method - Google Patents
Image data enhancement strategy selection method and face recognition image data enhancement method Download PDFInfo
- Publication number
- CN111127364B CN111127364B CN201911365687.8A CN201911365687A CN111127364B CN 111127364 B CN111127364 B CN 111127364B CN 201911365687 A CN201911365687 A CN 201911365687A CN 111127364 B CN111127364 B CN 111127364B
- Authority
- CN
- China
- Prior art keywords
- data enhancement
- image
- enhancement strategy
- strategy
- face recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000010187 selection method Methods 0.000 title claims abstract description 19
- 238000005457 optimization Methods 0.000 claims abstract description 50
- 230000002708 enhancing effect Effects 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims description 18
- 230000009466 transformation Effects 0.000 claims description 17
- 239000011159 matrix material Substances 0.000 claims description 14
- 238000003062 neural network model Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 238000005520 cutting process Methods 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 9
- 238000009499 grossing Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 34
- 238000012549 training Methods 0.000 description 12
- 238000013210 evaluation model Methods 0.000 description 9
- 238000013461 design Methods 0.000 description 7
- 238000010845 search algorithm Methods 0.000 description 7
- 238000009826 distribution Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000007781 pre-processing Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 206010050392 Face injury Diseases 0.000 description 1
- 101100311330 Schizosaccharomyces pombe (strain 972 / ATCC 24843) uap56 gene Proteins 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013398 bayesian method Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 101150018444 sub2 gene Proteins 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Abstract
The invention relates to an image data enhancement strategy selection method and a face recognition image data enhancement method. The image data enhancement strategy selection method comprises the following steps: acquiring a data enhancement strategy search space and a target object image set; randomly selecting a plurality of data enhancement strategies in a search space; respectively calculating image classification errors corresponding to a plurality of randomly selected data enhancement strategies to obtain an initial data enhancement strategy error pair set; and searching and selecting an optimal data enhancement strategy from the search space by adopting a Bayesian optimization method. The method for enhancing the face recognition image data comprises the step of applying the selected optimal data enhancement strategy to a face recognition image to be enhanced. The data enhancement strategy selection method provided by the invention only needs to run once according to the existing target object image set to select the optimal data enhancement strategy, so that the optimal data enhancement strategy can be applied to the image data of the same type of target object to enhance the image data, and the use efficiency of the image data enhancement method is improved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image data enhancement strategy selection method and a face recognition image data enhancement method.
Background
With the rapid development of image processing technology, a large amount of marked data such as face recognition, vehicle recognition and the like are required in many aspects, and acquiring sufficient data is a time-consuming and labor-consuming task. Manually tagging training data is one means to obtain larger tagged data sets, however, it is a difficult and time consuming way. Data enhancement is an effective alternative that can take advantage of existing data to generate new data by way of tag-preserving transformations.
For example, in the face recognition technology, a deep neural network model is usually used for recognition. The deep neural network model has high recognition accuracy, however, in order to ensure the performance of the deep neural network model, a large number of labeled samples are usually required for training, and in order to obtain a large number of labeled samples, multi-angle photos of a user need to be collected in a face recognition system in advance. Due to the change of the real environment, such as illumination, face angles, changes of face injuries and the like, a plurality of photos with different angles and under different light rays need to be collected for each user, so that the workload of photo collection is huge, the collection process is very complicated, and higher cost needs to be paid out. Therefore, the data enhancement technology is a better method for reducing the image acquisition cost.
Most of the existing data enhancement strategies utilized by many models are designed manually and need abundant expert experience, and the strategies designed in the mode usually cannot achieve optimal performance. Thus, prior work has proposed some automated data enhancement techniques to alleviate this problem. For example, a Bayesian method is used to generate new labeled training samples, depending on the distribution learned from the training set; establishing a network which is concentrated in learning enhancement, and fully utilizing mutual information in the class so as to generate enhanced data in the training process of the target network; new data is generated by generating a countermeasure network, and so on.
However, the strategies generated by these methods are usually not reusable, and are found based on a specific model or data set, which results in that the optimal strategy must be searched again each time the data enhancement strategy needs to be used, thereby greatly reducing the use efficiency of the image data enhancement method.
Disclosure of Invention
The invention aims to provide an image data enhancement strategy selection method and a face recognition image data enhancement method, and improve the use efficiency of the image data enhancement method.
In order to achieve the purpose, the invention provides the following scheme:
a method of image data enhancement policy selection, the method comprising:
acquiring a data enhancement strategy search space and a target object image set; the search space comprises a plurality of data enhancement strategies;
randomly selecting a plurality of data enhancement strategies in the search space;
respectively calculating image classification errors corresponding to a plurality of randomly selected data enhancement strategies according to the target object image set to obtain an initial data enhancement strategy error pair set;
and searching and selecting an optimal data enhancement strategy from the search space by adopting a Bayesian optimization method according to the target object image set and the initial data enhancement strategy error pair set.
Optionally, the data enhancement policy includes a plurality of sub-policies;
the sub-policy comprises a number of image conversion operations;
the image transformation operation is an image transformation operation in a Python image library.
Optionally, after the acquiring the data enhancement strategy search space and the target object image set, before the randomly selecting a plurality of data enhancement strategies in the search space, the method further includes:
standardizing each picture in the target object image set to obtain a standardized pixel value matrix of each picture;
filling a set number of zero pixel values in the standardized pixel value matrix of each picture to obtain a filled image of each picture;
randomly cutting the filled image of each picture in the same size to obtain a cut image of each picture;
and horizontally turning the cut image of each picture to obtain a preprocessed target object image set.
Optionally, the calculating, according to the target object image set, image classification errors corresponding to the randomly selected multiple data enhancement strategies specifically includes:
the following operations are executed on a plurality of randomly selected data enhancement strategies:
enhancing the target object image set according to a randomly selected data enhancement strategy to obtain an enhanced target object image set;
and carrying out image classification error evaluation on the enhancement target object image set by adopting a width residual error neural network model to obtain an image classification error corresponding to the randomly selected data enhancement strategy.
Optionally, the searching and selecting an optimal data enhancement policy from the search space by using a bayesian optimization method according to the target object image set and the initial data enhancement policy error pair set specifically includes:
according to the target object image set and the initial data enhancement strategy error pair set, maximizing a Bayesian optimization acquisition function, and searching a data enhancement strategy from the search space to obtain a search data enhancement strategy;
calculating an image classification error corresponding to the search data enhancement strategy according to the target object image set and the search data enhancement strategy to obtain a search data enhancement strategy error pair;
adding the search data enhancement strategy error pair to the initial data enhancement strategy error pair set, and updating the initial data enhancement strategy error pair set;
judging whether the maximum iteration times is reached to obtain a judgment result;
if the judgment result is negative, adding 1 to the iteration number, and returning to the step of searching a data enhancement strategy from the search space according to the target object image set and the initial data enhancement strategy error pair set and the acquisition function of the maximum Bayesian optimization to obtain a search data enhancement strategy;
and if so, taking the data enhancement strategy corresponding to the minimum image classification error value as the selected optimal data enhancement strategy.
A method of face recognition image data enhancement, the method comprising:
acquiring a data enhancement strategy search space and a face recognition image set; the search space comprises a plurality of data enhancement strategies;
randomly selecting a plurality of data enhancement strategies in the search space;
respectively calculating image classification errors corresponding to a plurality of randomly selected data enhancement strategies according to the face recognition image set to obtain an initial data enhancement strategy error pair set;
searching and selecting an optimal data enhancement strategy from the search space by adopting a Bayesian optimization method according to the face recognition image set and the initial data enhancement strategy error pair set;
and enhancing the face recognition image to be enhanced according to the optimal data enhancement strategy.
Optionally, the data enhancement policy includes a plurality of sub-policies;
the sub-policy comprises a number of image conversion operations;
the image transformation operation is an image transformation operation in a Python image library.
Optionally, after the obtaining of the data enhancement policy search space and the face recognition image set, before the randomly selecting a plurality of data enhancement policies in the search space, the method further includes:
standardizing each picture in the face recognition image set to obtain a standardized pixel value matrix of each picture;
filling a set number of zero pixel values in the standardized pixel value matrix of each picture to obtain a filled image of each picture;
randomly cutting the filled image of each picture according to the same size to obtain a cut image of each picture;
and horizontally turning the cut image of each picture to obtain a preprocessed face recognition image set.
Optionally, the calculating, according to the face recognition image set, image classification errors corresponding to a plurality of randomly selected data enhancement strategies specifically includes:
the following operations are executed on a plurality of randomly selected data enhancement strategies:
enhancing the face recognition image set according to a randomly selected data enhancement strategy to obtain an enhanced face recognition image set;
and carrying out image classification error evaluation on the enhanced face recognition image set by adopting a width residual neural network model to obtain an image classification error corresponding to a randomly selected data enhancement strategy.
Optionally, the searching and selecting an optimal data enhancement policy from the search space by using a bayesian optimization method according to the face recognition image set and the initial data enhancement policy error pair set specifically includes:
according to the face recognition image set and the initial data enhancement strategy error pair set, maximizing a Bayesian optimization acquisition function, and searching a data enhancement strategy from the search space to obtain a search data enhancement strategy;
calculating an image classification error corresponding to the search data enhancement strategy according to the face recognition image set and the search data enhancement strategy to obtain a search data enhancement strategy error pair;
adding the search data enhancement strategy error pair to the initial data enhancement strategy error pair set, and updating the initial data enhancement strategy error pair set;
judging whether the maximum iteration times is reached to obtain a judgment result;
if the judgment result is negative, adding 1 to the iteration times, and returning to the step of 'obtaining a data enhancement strategy by searching a data enhancement strategy from the search space according to the face recognition image set and the initial data enhancement strategy error pair set and maximizing the acquisition function of Bayesian optimization';
and if so, taking the data enhancement strategy corresponding to the minimum image classification error value as the selected optimal data enhancement strategy.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
according to the image data enhancement strategy selection method, the optimal data enhancement strategy is searched in the data enhancement strategy search space by utilizing the Bayesian optimization algorithm and maximizing the acquisition function according to the target object image set.
The image data enhancement strategy selection method provided by the invention only needs to run once according to the existing target object image set to select the optimal data enhancement strategy, so that the optimal data enhancement strategy can be applied to the image data of the same type of target object to enhance the data, and the use efficiency of the image data enhancement method is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a flow chart of a method for selecting an image data enhancement strategy according to the present invention;
fig. 2 is a flow chart of a face recognition image data enhancement method provided by the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide an image data enhancement strategy selection method and a face recognition image data enhancement method, which improve the use efficiency of the image data enhancement method.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a flowchart of an image data enhancement policy selection method provided by the present invention, and as shown in fig. 1, the image data enhancement policy selection method includes:
s101: acquiring a data enhancement strategy search space and a target object image set; the search space includes several data enhancement strategies. The data enhancement strategy comprises a plurality of sub-strategies; the sub-policy comprises a number of image conversion operations; the image transformation operation is an image transformation operation in a Python image library.
S102: and randomly selecting a plurality of data enhancement strategies in the search space.
S103: and respectively calculating image classification errors corresponding to a plurality of randomly selected data enhancement strategies according to the target object image set to obtain an initial data enhancement strategy error pair set.
The following operations are executed on a plurality of randomly selected data enhancement strategies:
enhancing the target object image set according to a randomly selected data enhancement strategy to obtain an enhanced target object image set;
and carrying out image classification error evaluation on the enhancement target object image set by adopting a width residual error neural network model to obtain an image classification error corresponding to the randomly selected data enhancement strategy.
S104: and searching and selecting an optimal data enhancement strategy from the search space by adopting a Bayesian optimization method according to the target object image set and the initial data enhancement strategy error pair set.
According to the target object image set and the initial data enhancement strategy error pair set, maximizing a Bayesian optimization acquisition function, and searching a data enhancement strategy from the search space to obtain a search data enhancement strategy;
calculating an image classification error corresponding to the search data enhancement strategy according to the target object image set and the search data enhancement strategy to obtain a search data enhancement strategy error pair;
adding the search data enhancement strategy error pair to the initial data enhancement strategy error pair set, and updating the initial data enhancement strategy error pair set;
judging whether the maximum iteration times is reached to obtain a judgment result;
if the judgment result is negative, adding 1 to the iteration number, and returning to the step of searching a data enhancement strategy from the search space according to the target object image set and the initial data enhancement strategy error pair set and the acquisition function of the maximum Bayesian optimization to obtain a search data enhancement strategy;
and if so, taking the data enhancement strategy corresponding to the minimum image classification error value as the selected optimal data enhancement strategy.
In this embodiment, after the obtaining of the data enhancement strategy search space and the target object image set, a preprocessing operation on the target object image set is further included before the randomly selecting a plurality of data enhancement strategies in the search space. The pretreatment operation specifically comprises:
standardizing each picture in the target object image set to obtain a standardized pixel value matrix of each picture;
filling a set number of zero pixel values in the standardized pixel value matrix of each picture to obtain a filled image of each picture;
randomly cutting the filled image of each picture in the same size to obtain a cut image of each picture;
and horizontally turning the cut image of each picture to obtain a preprocessed target object image set.
The embodiment also provides a method for enhancing face recognition image data, as shown in fig. 2, the method includes:
s201: acquiring a data enhancement strategy search space and a face recognition image set; the search space includes several data enhancement strategies. The data enhancement strategy comprises a plurality of sub-strategies; the sub-policy comprises a number of image conversion operations; the image transformation operation is an image transformation operation in a Python image library.
S202: and randomly selecting a plurality of data enhancement strategies in the search space.
S203: and respectively calculating image classification errors corresponding to a plurality of randomly selected data enhancement strategies according to the face recognition image set to obtain an initial data enhancement strategy error pair set.
The following operations are executed on a plurality of randomly selected data enhancement strategies:
enhancing the face recognition image set according to a randomly selected data enhancement strategy to obtain an enhanced face recognition image set;
and carrying out image classification error evaluation on the enhanced face recognition image set by adopting a width residual neural network model to obtain an image classification error corresponding to a randomly selected data enhancement strategy.
S204: and searching and selecting an optimal data enhancement strategy from the search space by adopting a Bayesian optimization method according to the face recognition image set and the initial data enhancement strategy error pair set.
According to the face recognition image set and the initial data enhancement strategy error pair set, maximizing a Bayesian optimization acquisition function, and searching a data enhancement strategy from the search space to obtain a search data enhancement strategy;
calculating an image classification error corresponding to the search data enhancement strategy according to the face recognition image set and the search data enhancement strategy to obtain a search data enhancement strategy error pair;
adding the search data enhancement strategy error pair to the initial data enhancement strategy error pair set, and updating the initial data enhancement strategy error pair set;
judging whether the maximum iteration times is reached to obtain a judgment result;
if the judgment result is negative, adding 1 to the iteration times, and returning to the step of 'obtaining a data enhancement strategy by searching a data enhancement strategy from the search space according to the face recognition image set and the initial data enhancement strategy error pair set and maximizing the acquisition function of Bayesian optimization';
and if so, taking the data enhancement strategy corresponding to the minimum image classification error value as the selected optimal data enhancement strategy.
S205: and performing data enhancement on the face recognition image to be enhanced according to the optimal data enhancement strategy.
In the method for enhancing face recognition image data of this embodiment, after the data enhancement policy search space and the face recognition image set are obtained, a preprocessing operation on the face recognition image set is further included before the plurality of data enhancement policies are randomly selected in the search space. The preprocessing operation performed on the face recognition image set specifically includes:
standardizing each picture in the face recognition image set to obtain a standardized pixel value matrix of each picture;
filling a set number of zero pixel values in the standardized pixel value matrix of each picture to obtain a filled image of each picture;
randomly cutting the filled image of each picture according to the same size to obtain a cut image of each picture;
and horizontally turning the cut image of each picture to obtain a preprocessed face recognition image set.
The image data enhancement strategy selection method and the face recognition image data enhancement method of the embodiment are mainly oriented to image classification tasks. The task of searching for the optimal data enhancement strategy is defined as an optimization problem in a continuous space, the optimal strategy is selected by utilizing a search algorithm, and then the found strategy is applied to the existing image classification data set.
In the data enhancement strategy selection, the design of a strategy search space and the design of a search algorithm are mainly involved. The search space contains several data enhancement policies, each policy comprising several sub-policies, each sub-policy comprising several image transformation operations, each policy being set in this embodiment to consist of 3 sub-policies, each sub-policy consisting of 2 consecutive image transformation operations (e.g. clipping, translation or rotation) and their corresponding parameters (i.e. probabilities and magnitudes of application operations). In the embodiment, a bayesian optimization method is adopted as a search algorithm, and the bayesian optimization method is an effective global optimization algorithm. The aim of the invention is to find an optimal data enhancement strategy at as little cost as possible. The key point is to optimize a black box function which takes the data enhancement strategy as the input and takes the performance of the strategy on the neural network classification model as the output. The objective function is:
wherein P represents a data enhancement strategy, P represents a strategy search space, m represents an image classification model, d represents an image classification dataset, u represents a process of applying a strategy on a dataset to obtain a new converted dataset, and f represents a process of training an image classification feedback model based on a dataset to obtain a classification error, which is a multi-peak function, so that our goal is to find points that can make the function reach an optimal value with as few bayesian optimization iterations (i.e., true strategy evaluation times) as possible, in particular relating to both the design of the strategy search space and the design of the search algorithm.
(1) Design of policy search space
The image conversion operation used in this embodiment is taken from the Python Image Library (PIL), which is a standard library of image processing that contains most of the basic image processing operations. Without loss of generality, the present embodiment employs 14 kinds of all conversion functions of the PIL, which accept an image and its corresponding operating parameters as inputs and output the converted image. Image transformation operation information as shown in table 1, the magnitude range for each operation is shown in the third column, and there are some transformations that do not require magnitude information (e.g., inversion and equalization).
TABLE 1 mapping relationship between image conversion operation and magnitude value range
For each operation, the probability of execution is always between 0 and 1, while the magnitude of the application of the operation depends on the particular type of operation. Conveniently, during the strategy search, the magnitude of all operations is set between 0 and 9, which is translated to a value within the true magnitude range of the operation when the strategy is used. Which comprisesSeveral operations that do not contain magnitude information, such as automatic contrast (autocontrost), inversion (Invert), and color equalization (Equalize). Since the problem of finding the optimal data enhancement strategy to be solved by the present invention is a continuous optimization problem, the parameters of the operation can take any value within the parameter range, including high precision fractions. The invention can help us find more accurate parameter configuration than manual design. Considering that each sub-policy consists of 2 consecutive operations, there are 14 operations in total, and therefore, two operations in each sub-policy can be represented by a number between 0 and 196 (as shown in table 2). For each operation, there are two parameters that are relevant. Thus, each sub-strategy can be represented by a 5-dimensional vector, denoted as p sub ＝[opers,pro 1 ,m 1 ,pro 2 ,m 2 ]. Wherein the first dimension icons of the vector represent two operation types of the sub-policy; second dimension pro 1 And a third dimension m 1 Respectively representing the application probability and magnitude of the first operation; pro in the fourth dimension 2 And the fifth dimension m 2 Respectively, collectively represent the parameters of the second operation. An example of representing a sub-policy (taken from the real sub-policy chosen by the present invention) with a five-dimensional vector is: [106.7859,0.7195,7.4252,0.3982,3.3927]The sub-strategy is represented by two operations of Contrast (Contrast) and Color (Color), wherein the probability parameters of the two operations are 0.7195 and 0.3982 respectively, and the magnitude parameters are 7.4252 and 3.3927 respectively. Since a data enhancement policy contains 3 sub-policies, each policy can be represented by a 15-dimensional vector, denoted as p ═ p (p) sub1 ,p sub2 ,p sub3 ) I.e. the strategy search space is
Enrichment is increased by finding a plurality of such strategies, and 8 such strategies are set to be found in the present embodiment.
TABLE 2 mapping relationship of sub-policy operation coding values to corresponding two operation types
(2) Design of search algorithm
The present embodiment uses bayesian optimization as a search algorithm, and the search process is shown in table 3. It first considers some known evaluation function observations, which represent the policy-based return values of the true evaluation model in the data enhancement scenario. Bayesian optimization utilizes a probabilistic proxy model to fit these observations and then evaluates by selecting a new point by maximizing the acquisition function in parameter space. The selected points are evaluated using an evaluation model and then added to the existing observation set. This process is performed iteratively until a predefined number of iterations is reached or a preset termination condition is reached. Bayesian optimization has several key components: selection of a real evaluation model, a probability agent model and selection of an acquisition function.
TABLE 3 search Algorithm complete iteration Process
In each iteration of the bayesian optimization, a performance evaluation is performed for the selected strategy, which is used as a feedback signal to update the probabilistic proxy model. In the invention, a Wide Residual neural Network (Wide Residual Network) is used as an evaluation model, and a Network type WRN-40-2(40 layers, width coefficient is 2) is specifically used as a feedback model, so that the generalization performance of the evaluation model based on a specific strategy is evaluated. The selected strategy is applied to the data set to generate a new data set, and then the feedback model is trained based on the new data set. In a small batch training process, each picture randomly selects one of all sub-strategies to apply. After training is finished, the feedback model evaluates on a pre-reserved verification set, and therefore the obtained classification error is used as a feedback signal.
In the image classification task performance evaluation model, an image data set is used as input, and classification errors of the model on the data set are used as output; and in response to the face recognition problem, the image classification task performance evaluation model inputs a face data set and outputs a classification error of the model on the face data set.
The probabilistic proxy model is used for simulating an unknown objective function, and starting from an assumed prior, the information amount is increased iteratively, and the prior is corrected, so that a more accurate proxy model is obtained. The invention selects the Gaussian process as a proxy model, which is a commonly used non-parametric model. Currently, the gaussian process has been widely applied to regression, classification, and other fields requiring reasoning black box functions. The Gaussian process includes a mean function
And a semi-definite kernel function (covariance function)
The kernel function is a similarity measure of the position of two points in the parameter space of the objective function. The invention adopts a Matern kernel function, has high flexibility and sets the smoothing parameter to 5/2. The kernel function formula is:
wherein r is | p-p' |, l is { l ═ l i I-0, …,14 is a scale parameter, l i Scale parameters for each dimension in the bayesian optimization input are represented. Consider a set of points p 1:t Wherein p is i ∈P，y 1:t ＝f(p i ) And y' 1:t The black box function value and the noise observation value are respectively expressed. Without loss of generality, the a priori mean function is typically set to m (p) 0, with y 1:t N (0, K) and y' 1:t ～N(y 1:t ,σ 2 I) In which K is i,j ＝k(p i ,p j ) And σ 2 Is the system noise. Let Dt be { p } 1:t ,y’ 1:t Denotes a set of observations, p * Representing any one candidate point. His posterior predicted distribution can be obtained: f. of * |Dt～N(μ(p * ),σ(p * ) Wherein μ (p) * )＝K * (K+σ 2 I) - 1 y' 1:t ，σ(p * )＝k(p * ,p * )-K * (K+σ 2 I) -1 K *T ，
That is, for any candidate point, the posterior prediction mean and variance can be calculated respectively, so as to represent the prediction value and uncertainty of the model. Notably, the present invention uses the Monte Carlo Markov Method (MCMC) to estimate the hyper-parameters (scale parameters) of the Gaussian process. Specifically, in each iteration of Bayesian optimization, MCMC is used for sampling each hyper-parameter for multiple times, then the hyper-parameters are applied to the proxy models respectively, and finally the mean value of the models is used as the final proxy model. MCMC allows for an overall distribution of the hyper-parameters, so generally better results can be obtained. The invention uses EI function as collection function, and maximizes EI by means of Covariance Matrix adaptive evolution strategy (CMA-ES) to find candidate points. The EI function expression is:
wherein phi (-) represents a standard normal distribution cumulative density function, phi (-) represents a standard normal distribution probability density function,
y max is the current optimum function value.
In the setting of the invention, Bayesian optimization can select one data enhancement strategy each time, and independently run for 8 times to find 24 sub-strategies in total for final training of the model on each data set.
This embodiment takes face identification of access control system as an example, specifically explains, and the specific application steps are divided into two stages:
the first stage belongs to preparation work before practical application: namely, the optimal data enhancement strategy is selected by utilizing the algorithm provided by the invention.
The second stage belongs to the practical application link: the marked sample size is expanded by using the optimal data enhancement strategy selected in the first stage, so that more accurate and efficient face recognition of the access control system is realized.
The method for the face recognition of the access control system is specifically executed as follows:
the first step is as follows: considering that the access control system only needs to recognize the face, the existing face recognition technology research is relatively mature, and a large amount of face data sets are reserved, so the invention aims to select the optimal data enhancement strategy by using the existing face data sets. On one hand, the existing data set may have the problem of data noise which needs to be considered in practical application, so that the selected optimal data enhancement strategy is more applicable/robust in practical application, and on the other hand, the tedious work of actually acquiring face data is avoided. The invention aims to select a data enhancement strategy by utilizing a CASIA-faceV5 face recognition data set, wherein the data set comprises 2500 face pictures from 500 persons, the pictures in the data set comprise variants such as different illumination conditions, different face angles and the like, and more fully cover various noise conditions. The invention is to select 2000 pictures as a training set and 500 pictures as a testing set.
The second step is that: for each face picture in the data set, a basic image pre-processing operation is performed. The specific pre-treatment operation sequence is as follows:
A) performing standardization operation on each picture in the data set, namely calculating the mean value and standard deviation of all pictures in the training set, subtracting the mean value from the pixel value of each face picture, and dividing the mean value by the standard deviation to obtain a standardized face picture;
B) zero filling is carried out on each picture in the data set, namely a pixel value matrix of each face picture is filled with a certain number of zero pixel values on each side;
C) randomly cutting each picture in the data set, namely randomly cutting the face picture subjected to zero padding to ensure that each face picture in the data set has the same size;
D) horizontally turning each picture in the data set;
the third step: and searching an optimal data enhancement strategy by using a Bayesian optimization algorithm. The Bayesian optimization algorithm is an effective global optimization algorithm, the algorithm is a continuous iteration process, and the detailed steps of the algorithm are as follows:
A) initializing a group of data enhancement strategy error pair sets for a Bayesian optimization algorithm in a random initialization mode, and recording the sets as D { (p) i ,y i ) 1, …,10}, where p i Denotes the ith policy, y i The classification error returned after the ith strategy is applied to the face data set to generate an enhanced data set and the enhanced data set is sent to the image classification task performance evaluation model WRN-40-2 is represented;
B) maximizing a Bayesian optimization collection function, and searching out a new data enhancement strategy p;
C) enhancing an original face picture data set by using p to obtain an enhanced data set;
D) sending the enhanced data set into an image classification task performance evaluation model WRN-40-2 to obtain a classification error y;
E) extending (p, y) into D;
F) updating the Gaussian process, judging whether the algorithm reaches the preset iteration times, if not, turning to B) to continue the iteration, and if so, turning to G);
G) selecting strategy p corresponding to minimum y value from D optimal The strategy is an optimal data enhancement strategy searched by the Bayesian optimization.
In order to increase the richness of the selected strategy, 8 optimal data enhancement strategies are to be searched, and because the Bayesian optimization can select one data enhancement strategy each time, 8 Bayesian optimization algorithms need to be independently operated to find the 8 optimal data enhancement strategies for finally expanding the image on the face recognition task.
The fourth step: the optimal data enhancement strategy is selected, the access control system only needs to collect a small amount of face picture data (for example, only needs to collect one piece of face picture data) for each person, and then the data enhancement strategy is utilized to expand the labeled sample size of each person. Specifically, for an existing picture of each person, a data enhancement strategy is directly utilized to generate a plurality of variants of the original picture, wherein the variants may include rotation, cutting, illumination change, pixel change and the like of the picture, and the newly generated picture is regarded as a noisy picture thereof and has the same category label information as the original picture. At the moment, a large number of labeled pictures are generated for each person, and at the moment, the face recognition neural network model is used for training, so that a model with high recognition accuracy and strong anti-noise capability can be obtained. Therefore, accurate face recognition of the access control system can be still realized under the condition that only one face picture is collected for each person.
The image data enhancement strategy selection method and the face recognition image data enhancement method also have the following effects:
the Bayesian optimization technology is utilized to solve the data enhancement problem facing the image classification task, so that the time overhead is effectively reduced, a large amount of computing resources are saved, and the automation of the data enhancement task is realized;
the search data enhancement strategy problem is regarded as a search problem in a continuous space, and a more accurate hyper-parameter is selected for data enhancement operation, so that the effect of applying the strategy to an image classification task is improved;
the image data enhancement strategy selection method provided by the invention only needs to run once according to the existing target object image set to select the optimal data enhancement strategy, so that the optimal data enhancement strategy can be applied to the image data of the same type of target object to enhance the image data, and the use efficiency of the image data enhancement method is improved.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (8)
1. An image data enhancement policy selection method, the method comprising:
acquiring a data enhancement strategy search space and a target object image set; the search space comprises a plurality of data enhancement strategies;
randomly selecting a plurality of data enhancement strategies in the search space;
respectively calculating image classification errors corresponding to a plurality of randomly selected data enhancement strategies according to the target object image set to obtain an initial data enhancement strategy error pair set;
searching and selecting an optimal data enhancement strategy from the search space by adopting a Bayesian optimization method according to the target object image set and the initial data enhancement strategy error pair set;
the searching and selecting the optimal data enhancement strategy from the search space by adopting a Bayesian optimization method according to the target object image set and the initial data enhancement strategy error pair set specifically comprises the following steps:
according to the target object image set and the initial data enhancement strategy error pair set, maximizing a Bayesian optimization acquisition function, and searching a data enhancement strategy from the search space to obtain a search data enhancement strategy;
calculating an image classification error corresponding to the search data enhancement strategy according to the target object image set and the search data enhancement strategy to obtain a search data enhancement strategy error pair;
adding the search data enhancement strategy error pair to the initial data enhancement strategy error pair set, and updating the initial data enhancement strategy error pair set;
judging whether the maximum iteration times is reached to obtain a judgment result;
if the judgment result is negative, adding 1 to the iteration number, and returning to the step of searching a data enhancement strategy from the search space according to the target object image set and the initial data enhancement strategy error pair set and the acquisition function of the maximum Bayesian optimization to obtain a search data enhancement strategy;
if the judgment result is yes, taking the data enhancement strategy corresponding to the minimum image classification error value as the selected optimal data enhancement strategy;
the probability agent model in the Bayesian optimization method is a Gaussian process; the kernel function of the Gaussian process is a Matern kernel function; the smoothing parameter of the Matern kernel function is 5/2; the Matern kernel function formula is as follows:
wherein r is | p-p' |, l is { l |, and i i-0, …,14 is a scale parameter, l i Scale parameters for each dimension in the bayesian optimization input are represented.
2. The image data enhancement strategy selection method of claim 1,
the data enhancement strategy comprises a plurality of sub-strategies;
the sub-policy comprises a number of image conversion operations;
the image transformation operation is an image transformation operation in a Python image library.
3. The image data enhancement strategy selection method of claim 1,
after the acquiring the data enhancement strategy search space and the target object image set, before the randomly selecting a plurality of data enhancement strategies in the search space, the method further includes:
standardizing each picture in the target object image set to obtain a standardized pixel value matrix of each picture;
filling a set number of zero pixel values in the standardized pixel value matrix of each picture to obtain a filled image of each picture;
randomly cutting the filled image of each picture in the same size to obtain a cut image of each picture;
and horizontally turning the cut image of each picture to obtain a preprocessed target object image set.
4. The image data enhancement strategy selection method of claim 1,
the step of calculating image classification errors corresponding to the randomly selected data enhancement strategies according to the target object image set specifically includes:
the following operations are executed on a plurality of randomly selected data enhancement strategies:
enhancing the target object image set according to a randomly selected data enhancement strategy to obtain an enhanced target object image set;
and carrying out image classification error evaluation on the enhancement target object image set by adopting a width residual error neural network model to obtain an image classification error corresponding to the randomly selected data enhancement strategy.
5. A method for enhancing face recognition image data, the method comprising:
acquiring a data enhancement strategy search space and a face recognition image set; the search space comprises a plurality of data enhancement strategies;
randomly selecting a plurality of data enhancement strategies in the search space;
respectively calculating image classification errors corresponding to a plurality of randomly selected data enhancement strategies according to the face recognition image set to obtain an initial data enhancement strategy error pair set;
searching and selecting an optimal data enhancement strategy from the search space by adopting a Bayesian optimization method according to the face recognition image set and the initial data enhancement strategy error pair set;
enhancing the face recognition image to be enhanced according to the optimal data enhancement strategy;
the searching and selecting the optimal data enhancement strategy from the search space by adopting a Bayesian optimization method according to the face recognition image set and the initial data enhancement strategy error pair set specifically comprises the following steps:
according to the face recognition image set and the initial data enhancement strategy error pair set, maximizing a Bayesian optimization acquisition function, and searching a data enhancement strategy from the search space to obtain a search data enhancement strategy;
calculating an image classification error corresponding to the search data enhancement strategy according to the face recognition image set and the search data enhancement strategy to obtain a search data enhancement strategy error pair;
adding the search data enhancement strategy error pair to the initial data enhancement strategy error pair set, and updating the initial data enhancement strategy error pair set;
judging whether the maximum iteration times is reached to obtain a judgment result;
if the judgment result is negative, adding 1 to the iteration times, and returning to the step of 'obtaining a data enhancement strategy by searching a data enhancement strategy from the search space according to the face recognition image set and the initial data enhancement strategy error pair set and maximizing the acquisition function of Bayesian optimization';
if the judgment result is yes, taking the data enhancement strategy corresponding to the minimum image classification error value as the selected optimal data enhancement strategy;
the probability agent model in the Bayesian optimization method is a Gaussian process; the kernel function of the Gaussian process is a Matern kernel function; the smoothing parameter of the Matern kernel function is 5/2; the Matern kernel function formula is as follows:
wherein r is | p-p' |, l is { l ═ l i I-0, …,14 is a scale parameter, l i Representing Bayesian optimization inputsThe scale parameter of each dimension in (1).
6. The face recognition image data enhancement method of claim 5,
the data enhancement strategy comprises a plurality of sub-strategies;
the sub-policy comprises a number of image conversion operations;
the image transformation operation is an image transformation operation in a Python image library.
7. The method of enhancing face recognition image data according to claim 5,
after the acquiring the data enhancement strategy search space and the face recognition image set, before the randomly selecting a plurality of data enhancement strategies in the search space, the method further includes:
standardizing each picture in the face recognition image set to obtain a standardized pixel value matrix of each picture;
filling a set number of zero pixel values in the standardized pixel value matrix of each picture to obtain a filled image of each picture;
randomly cutting the filled image of each picture according to the same size to obtain a cut image of each picture;
and horizontally turning the cut image of each picture to obtain a preprocessed face recognition image set.
8. The face recognition image data enhancement method of claim 5,
the step of calculating image classification errors corresponding to the randomly selected data enhancement strategies according to the face recognition image set specifically comprises:
the following operations are executed on a plurality of randomly selected data enhancement strategies:
enhancing the face recognition image set according to a randomly selected data enhancement strategy to obtain an enhanced face recognition image set;
and carrying out image classification error evaluation on the enhanced face recognition image set by adopting a width residual neural network model to obtain an image classification error corresponding to a randomly selected data enhancement strategy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911365687.8A CN111127364B (en) | 2019-12-26 | 2019-12-26 | Image data enhancement strategy selection method and face recognition image data enhancement method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911365687.8A CN111127364B (en) | 2019-12-26 | 2019-12-26 | Image data enhancement strategy selection method and face recognition image data enhancement method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111127364A CN111127364A (en) | 2020-05-08 |
| CN111127364B true CN111127364B (en) | 2022-08-02 |
Family
ID=70503026
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911365687.8A Active CN111127364B (en) | 2019-12-26 | 2019-12-26 | Image data enhancement strategy selection method and face recognition image data enhancement method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111127364B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111667004A (en) * | 2020-06-05 | 2020-09-15 | 北京百度网讯科技有限公司 | Data generation method, device, equipment and storage medium |
| CN111695624B (en) * | 2020-06-09 | 2024-04-16 | 北京市商汤科技开发有限公司 | Updating method, device, equipment and storage medium of data enhancement strategy |
| CN111696066B (en) * | 2020-06-13 | 2022-04-19 | 中北大学 | Multi-band image synchronous fusion and enhancement method based on improved WGAN-GP |
| CN111931492A (en) * | 2020-07-16 | 2020-11-13 | 平安科技（深圳）有限公司 | Data expansion mixing strategy generation method and device and computer equipment |
| CN112651892A (en) * | 2020-12-22 | 2021-04-13 | 中国科学技术大学 | Automatic data enhancement strategy selection method based on image sample |
| CN113033410B (en) * | 2021-03-26 | 2023-06-06 | 中山大学 | Domain generalization pedestrian re-recognition method, system and medium based on automatic data enhancement |
| CN113537406B (en) * | 2021-08-30 | 2023-04-07 | 重庆紫光华山智安科技有限公司 | Method, system, medium and terminal for enhancing image automatic data |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831325A (en) * | 2012-09-04 | 2012-12-19 | 北京航空航天大学 | Method for predicting bearing fault based on Gaussian process regression |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150235073A1 (en) * | 2014-01-28 | 2015-08-20 | The Trustees Of The Stevens Institute Of Technology | Flexible part-based representation for real-world face recognition apparatus and methods |
| CN106228142B (en) * | 2016-07-29 | 2019-02-15 | 西安电子科技大学 | Face verification method based on convolutional neural networks and Bayesian decision |
| US10235601B1 (en) * | 2017-09-07 | 2019-03-19 | 7D Labs, Inc. | Method for image analysis |
| CN110378419B (en) * | 2019-07-19 | 2021-07-16 | 广东浪潮大数据研究有限公司 | Image set expansion method, device, equipment and readable storage medium |
-
2019
- 2019-12-26 CN CN201911365687.8A patent/CN111127364B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831325A (en) * | 2012-09-04 | 2012-12-19 | 北京航空航天大学 | Method for predicting bearing fault based on Gaussian process regression |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111127364A (en) | 2020-05-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111127364B (en) | Image data enhancement strategy selection method and face recognition image data enhancement method | |
| US11341424B2 (en) | Method, apparatus and system for estimating causality among observed variables | |
| US8108324B2 (en) | Forward feature selection for support vector machines | |
| CN110674323B (en) | Unsupervised cross-modal Hash retrieval method and system based on virtual label regression | |
| CN108446741B (en) | Method, system and storage medium for evaluating importance of machine learning hyper-parameter | |
| CN113837205B (en) | Method, apparatus, device and medium for image feature representation generation | |
| CN113139664B (en) | Cross-modal migration learning method | |
| Lee et al. | Localization uncertainty estimation for anchor-free object detection | |
| CN113128478A (en) | Model training method, pedestrian analysis method, device, equipment and storage medium | |
| CN112381079A (en) | Image processing method and information processing apparatus | |
| Ilić et al. | Active learning using a self-correcting neural network (ALSCN) | |
| Pham et al. | Unsupervised training of Bayesian networks for data clustering | |
| CN115496144A (en) | Power distribution network operation scene determining method and device, computer equipment and storage medium | |
| CN111611395B (en) | Entity relationship identification method and device | |
| CN111161238A (en) | Image quality evaluation method and device, electronic device, and storage medium | |
| CN114255381B (en) | Training method of image recognition model, image recognition method, device and medium | |
| CN113723572B (en) | Ship target identification method, computer system, program product and storage medium | |
| CN116208399A (en) | Network malicious behavior detection method and device based on metagraph | |
| US20230267175A1 (en) | Systems and methods for sample efficient training of machine learning models | |
| CN112884065A (en) | Deep learning model robust boundary assessment method and device based on support vector machine and application | |
| Toubeh et al. | Risk-aware planning by confidence estimation using deep learning-based perception | |
| CN113282705B (en) | Case pre-judgment intelligent body training method and system capable of being automatically updated | |
| US20220366188A1 (en) | Parameterized neighborhood memory adaptation | |
| CN116257633A (en) | Text clustering method and device | |
| CN117875322A (en) | Entity extraction method, system, equipment and medium of text data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |