CN114140619A - Image data generation method, model training method, device, equipment and medium - Google Patents
Image data generation method, model training method, device, equipment and medium Download PDFInfo
- Publication number
- CN114140619A CN114140619A CN202111278700.3A CN202111278700A CN114140619A CN 114140619 A CN114140619 A CN 114140619A CN 202111278700 A CN202111278700 A CN 202111278700A CN 114140619 A CN114140619 A CN 114140619A
- Authority
- CN
- China
- Prior art keywords
- image
- feature map
- neural network
- network model
- iteration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000012549 training Methods 0.000 title claims abstract description 63
- 238000003062 neural network model Methods 0.000 claims abstract description 114
- 238000010586 diagram Methods 0.000 claims abstract description 21
- 238000010606 normalization Methods 0.000 claims abstract description 21
- 230000006870 function Effects 0.000 claims description 136
- 238000004364 calculation method Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 16
- 238000000605 extraction Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 6
- 238000010276 construction Methods 0.000 description 5
- 238000013473 artificial intelligence Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The application discloses an image data generation method, a model training method, a device, equipment and a medium, wherein the method comprises the following steps: constructing an objective function containing a first regular term and a second regular term; acquiring an initial neural network model trained in advance, and extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model; creating a noise image, taking the noise image as a primary iteration image, and extracting a feature map of the primary iteration image; and taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the target function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data. By the scheme, the situation that the original data set is difficult to obtain is avoided, and the accuracy of the initial neural network model is improved.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to an image data generation method, a model training method, a device, equipment and a medium.
Background
With the rapid development and application of artificial intelligence and deep neural network models, convolutional neural models have been widely used, but the shortcomings of convolutional neural models are obvious, such as large parameters, large calculation amount, and large memory occupation. The model fine tuning can solve the problems of large parameter quantity, large calculation quantity, large memory occupation and the like of the conventional convolutional neural network, has the potential advantages of compressing parameters, improving the speed, reducing the memory occupation and the like of the neural network, and can optimize the single precision to lower precision when the model is fine tuned, so that the accuracy of the model is greatly reduced. Currently, model fine tuning methods are used to reduce the loss of model accuracy. The basic principle of the method is to perform a small amount of 'retraining' on the model by means of the original data set to improve the accuracy, but in practical application, the original data set is usually huge in scale and difficult to acquire, so that the method cannot be operated practically.
In summary, how to avoid the situation that the original data set is difficult to obtain, and further improve the accuracy of the initial neural network model is a problem to be solved in the art.
Disclosure of Invention
In view of this, the present invention provides an image data generation method, a model training method, an apparatus, a device and a medium, which can avoid the situation that it is difficult to obtain an original data set, and further improve the accuracy of an initial neural network model. The specific scheme is as follows:
in a first aspect, the present application discloses an image data generating method, including:
constructing an objective function containing a first regular term and a second regular term; the first regular term takes an iterative image as a variable parameter, the second regular term takes a feature map of the iterative image as a variable parameter, and the iterative image of non-primary iteration is an image constructed based on a function value of the objective function;
acquiring a pre-trained initial neural network model, and extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model;
creating a noise image, taking the noise image as a primary iteration image, and then extracting a feature map of the primary iteration image;
and taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data.
Optionally, the constructing an objective function including a first regularization term and a second regularization term includes:
constructing a classification loss function corresponding to the initial neural network model based on the initial neural network model trained in advance and a pre-specified image class to be generated;
constructing a first regular term which takes the iterative image as a variable parameter and is used for controlling the natural characteristics of the image, and constructing a second regular term which takes a feature map of the iterative image as a variable parameter and is used for controlling the fidelity of the image;
constructing an objective function comprising the first regularization term, the second regularization term, and the classification loss function.
Optionally, the constructing a first regularization term for controlling natural characteristics of the image with the iterative image as a variable parameter includes:
and respectively constructing a global variance and an L2 regular term by taking the iterative image as variable parameters, and distributing corresponding weight coefficients for the global variance and the L2 regular term to obtain a first regular term for controlling the natural characteristics of the image.
Optionally, the constructing a second regularization term for controlling the image fidelity with the feature map of the iterative image as a variable parameter includes:
and constructing a feature map information calculation function by taking the feature map of the iterative image as a variable parameter, and constructing a second regular term for controlling the image fidelity based on the difference between the feature map information calculation function and the pre-stored feature map information.
Optionally, the extracting pre-stored feature map information from the normalization layer of the initial neural network model includes:
extracting pre-stored feature map mean information and feature map variance information from a normalization layer of the initial neural network model;
correspondingly, the constructing a feature map information calculation function by using the feature map of the iterative image as a variable parameter, and constructing a second regular term for controlling the image fidelity based on the difference between the feature map information calculation function and the pre-stored feature map information includes:
respectively constructing a feature map mean function and a feature map variance function by taking the feature map of the iterative image as a variable parameter;
constructing a first L2 regularization term based on a difference between the feature map mean function and the feature map mean information, and a second L2 regularization term based on a difference between the feature map variance function and the feature map variance information;
constructing a second regularization term for controlling image fidelity that includes the first L2 regularization term and the second L2 regularization term.
Optionally, before the performing a plurality of iterative operations on the objective function based on the preset iteration number to generate corresponding image data, the method further includes:
determining hyper-parameter information for configuring the target function, and configuring the target function by using the hyper-parameter information; wherein, one or more parameter information of learning rate, batch size and iteration frequency is determined as the hyper-parameter information.
In a second aspect, the present application discloses a neural network model training method, including:
acquiring an initial neural network model which is trained in advance;
determining the image demand according to the training precision requirement;
repeatedly executing the image data generation method disclosed in the foregoing multiple times based on the image demand to obtain a training data set including a corresponding number of image data;
and retraining the initial neural network model by using the training data set to obtain an optimized neural network model.
In a third aspect, the present application discloses an image data generating apparatus comprising:
the function building module is used for building an objective function containing a first regular term and a second regular term; the first regular term takes an iterative image as a variable parameter, the second regular term takes a feature map of the iterative image as a variable parameter, and the iterative image of non-primary iteration is an image constructed based on a function value of the objective function;
the characteristic diagram information extraction module is used for acquiring an initial neural network model which is trained in advance, and then extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model;
the iterative image creating module is used for creating a noise image, taking the noise image as a primary iterative image and then extracting a feature map of the primary iterative image;
and the image data generation module is used for taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data.
In a fourth aspect, the present application discloses an image data generation apparatus comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the image data generation method disclosed in the foregoing.
In a fifth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein the computer program realizes the steps of the image data generation method disclosed in the foregoing when executed by a processor.
As can be seen, the present application first constructs an objective function including a first regularization term and a second regularization term; the first regular term takes an iterative image as a variable parameter, the second regular term takes a feature map of the iterative image as a variable parameter, and the iterative image of non-primary iteration is an image constructed based on a function value of the objective function; acquiring a pre-trained initial neural network model, and extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model; creating a noise image, taking the noise image as a primary iteration image, and then extracting a feature map of the primary iteration image; and taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data. It can be seen that the present application provides for a method of generating a normalized image by creating an objective function comprising a first regularization term and a second regularization term, and feature map information, a noise image and a feature map of the noise image prestored in the initial neural network model trained in advance are input to the objective function, to generate image data corresponding to the initial neural network model trained in advance as training data, therefore, the original data of the initial neural network model which is trained in advance is not required to be obtained, the difficulty of obtaining the training data is reduced, and the obtained image data is ensured to be closer to the original data by carrying out a plurality of times of iterative operation on the target function based on the preset iteration times, and then the image data can be used as training data to improve the efficiency of retraining the initial neural network model by utilizing the training data set subsequently.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of an image data generation method disclosed herein;
FIG. 2 is a flow chart of a particular method of generating image data as disclosed herein;
FIG. 3 is a flow chart of a neural network model training method disclosed herein;
FIG. 4 is a flow chart of a particular neural network model training method disclosed herein;
FIG. 5 is a schematic diagram of an image data generating apparatus according to the present disclosure;
fig. 6 is a block diagram of an electronic device disclosed in the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Currently, the method of reducing the loss of model accuracy is to perform a small amount of "retraining" on the initial neural network model that is trained in advance by requiring a raw data set to improve accuracy, but in practical applications, the raw data set is usually large in scale and difficult to acquire, and thus cannot be implemented.
Therefore, the image data generation method is correspondingly provided, the situation that the original data set is difficult to obtain can be avoided, and the accuracy of the initial neural network model is further improved.
Referring to fig. 1, an embodiment of the present application discloses an image data generation method, including:
step S11: constructing an objective function containing a first regular term and a second regular term; the first regular term takes an iterative image as a variable parameter, the second regular term takes a feature map of the iterative image as a variable parameter, and the iterative image of the non-primary iteration is an image constructed based on a function value of the objective function.
In this embodiment, the constructing an objective function including a first regularization term and a second regularization term may include: constructing a classification loss function corresponding to an initial neural network model based on the initial neural network model trained in advance and a pre-specified image class to be generated; constructing a first regular term which takes the iterative image as a variable parameter and is used for controlling the natural characteristics of the image, and constructing a second regular term which takes a feature map of the iterative image as a variable parameter and is used for controlling the fidelity of the image; constructing an objective function comprising the first regularization term, the second regularization term, and the classification loss function. It can be understood that the pre-specified to-be-generated image category may be all categories as to-be-generated image categories based on the initial neural network model, or may be a to-be-generated image category selected in a targeted manner, for example, a category defined in the ImageNet data set may be arbitrarily selected as to-be-generated image category in this embodiment.
Step S12: the method comprises the steps of obtaining an initial neural network model which is trained in advance, and then extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model.
It can be understood that, in this embodiment, after the pre-stored feature map information is extracted from the normalization layer of the initial neural network model, the pre-stored feature map information is extracted to obtain feature map mean information and feature map variance information. For example, feature map mean information and feature map variance information related to an original data set, which are saved, are extracted in a Normalization layer (Batch Normalization) of a Convolutional Neural Network (CNN) model, so that the feature map mean information and the feature map variance information are input into the objective function subsequently.
Step S13: and creating a noise image, taking the noise image as a primary iteration image, and then extracting a feature map of the primary iteration image.
In this embodiment, the created noise image may be a random noise image, and the size is set based on a preset image requirement, for example, the size is set to 224 × 224 in this embodiment, it should be noted that, since the objective function needs to perform a plurality of iterations before corresponding image data is subsequently generated, the created noise image is only used as an initial iteration image, and therefore it can be understood that the feature map extracted from the noise image is also only the feature map of the initial iteration image
Step S14: and taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data.
In this embodiment, before the performing a plurality of iterative operations on the objective function based on the preset iteration number to generate corresponding image data, the method specifically includes: determining hyper-parameter information for configuring the target function, and configuring the target function by using the hyper-parameter information; wherein, one or more parameter information of Learning Rate (LR), batch size (batch size) and iteration number (Epoch) is determined as the hyper-parameter information. In this embodiment, for example, in this embodiment, the learning rate is first set to 0.05, the batch size is set to 512, the number of iterations is set to 10000, then the feature map of the first iteration image, the pre-stored feature map information, and the first iteration image are input into the objective function, the iteration operation with the number of iterations of 10000 is performed, and finally, image data is obtained as training data.
It can be understood that, when it is expected that the image category to be generated includes all defined categories, all the categories are sequentially used as pre-specified image categories to be generated, then a classification loss function corresponding to the initial neural network model is constructed based on the pre-trained initial neural network model and the pre-specified image categories to be generated, then an objective function including the first regular term, the second regular term and the classification loss function is constructed, and the above steps S12, S13 and S14 are repeated to generate a corresponding image data set. For example, all defined classes in the ImageNet data set are sequentially used as pre-specified classes of images to be generated, and after the steps are repeatedly executed, 100 to 1000 pieces of image data are finally acquired. In the embodiment, all the defined classes are used as the pre-specified image classes to be generated, so that the generated image data is more comprehensive, and the image data is used as the training data, so that the training data has higher reliability, and the initial neural network model trained again by the training data can keep better accuracy.
As can be seen, the present application first constructs an objective function including a first regularization term and a second regularization term; the first regular term takes an iterative image as a variable parameter, the second regular term takes a feature map of the iterative image as a variable parameter, and the iterative image of non-primary iteration is an image constructed based on a function value of the objective function; acquiring a pre-trained initial neural network model, and extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model; creating a noise image, taking the noise image as a primary iteration image, and then extracting a feature map of the primary iteration image; and taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data. It can be seen that the present application provides for a method of generating a normalized image by creating an objective function comprising a first regularization term and a second regularization term, inputting pre-stored characteristic diagram information, a noise image and a characteristic diagram of the noise image in the pre-trained initial neural network model into the objective function, to generate image data corresponding to the initial neural network model trained in advance as training data, therefore, the original data of the initial neural network model which is trained in advance is not required to be obtained, the difficulty of obtaining the training data is reduced, and the obtained image data is ensured to be closer to the original data by carrying out a plurality of times of iterative operation on the target function based on the preset iteration times, and then the image data can be used as training data, and the efficiency of retraining the initial neural network model by using the training data set subsequently is improved.
Referring to fig. 2, the embodiment of the present invention discloses a specific image data generation method, and compared with the previous embodiment, the present embodiment further describes and optimizes the technical solution. Specifically, the method comprises the following steps:
step S21: and constructing a first regular term which takes the iteration image as a variable parameter and is used for controlling the natural characteristics of the image.
In this embodiment, the constructing a first regularization term for controlling natural characteristics of the image with the iterative image as a variable parameter may include: and respectively constructing a global variance and an L2 regular term by taking the iterative image as variable parameters, and distributing corresponding weight coefficients for the global variance and the L2 regular term to obtain a first regular term for controlling the natural characteristics of the image. In this embodiment, the global variance may reduce a difference between each point in the image and the surrounding upper, lower, left, and right, and the L2 regular term may prevent the generated image from being over-fitted, so that the image is more harmonious. Wherein the first regular term formula is as follows:
wherein the above
It refers to the first regularization term that,
is the global variance (TV, also known as total variance),
is a regular term of L2, αTVAnd
the global variance and the weighting coefficients of the L2 regularization term, respectively.
Step S22: and constructing a second regular term which takes the characteristic graph of the iterative image as a variable parameter and is used for controlling the image fidelity.
In this embodiment, the constructing a second regularization term for controlling the image fidelity with the feature map of the iterative image as a variable parameter includes: and constructing a feature map information calculation function by taking the feature map of the iterative image as a variable parameter, and constructing a second regular term for controlling the image fidelity based on the difference between the feature map information calculation function and the pre-stored feature map information.
In this embodiment, the constructing a feature map information calculation function by using the feature map of the iterative image as a variable parameter, and constructing a second regular term for controlling the image fidelity based on a difference between the feature map information calculation function and the pre-stored feature map information includes: respectively constructing a feature map mean function and a feature map variance function by taking the feature map of the iterative image as a variable parameter; constructing a first L2 regular term based on a difference between the feature map mean function and pre-stored feature map mean information subsequently extracted from the normalization layer of the initial neural network model, and constructing a second L2 regular term based on a difference between the feature map variance function and pre-stored feature map variance information subsequently extracted from the normalization layer of the initial neural network model; constructing a second regularization term for controlling image fidelity that includes the first L2 regularization term and the second L2 regularization term. Specifically, the formula of the second regularization term is as follows:
wherein the content of the first and second substances,
the above is the second regularization term, xlFeature maps of the l-th layer of the iterative image, Mean (x)l) Is the l-th layer feature map mean function, Var (x)l) Is the variance function of the first layer feature map, mulTo pre-store the ith layer feature map mean information,
to pre-store the first layer of feature map variance information, | ·| non-calculation2The operator is an L2 regular calculation.
Step S23: constructing an objective function comprising the first regularization term and the second regularization term; the iterative image of the non-primary iteration is an image constructed based on the function value of the objective function.
In this embodiment, an objective function for generating image data needs to be constructed first, and the objective function needs to include the following two items: the image processing method comprises a first regular term taking an iterative image as a variable parameter and a second regular term taking a feature map of the iterative image as the variable parameter. It should be noted that the iteration image of the non-initial iteration is an image constructed based on the function value of the objective function. That is, in this embodiment, the constructed objective function is iterated multiple times to generate final image data, and except for the first iteration, each iteration thereafter, the objective function is an image constructed by a function value output by the objective function after the last iteration is completed, and the purpose of continuously iterating and updating is achieved in this way.
It should be noted that, in this embodiment, the objective function includes not only the first regular term and the second regular term, but also a classification loss function that represents a difference degree between the predicted data and the actual data, i.e., robustness of the model, where the classification function is constructed based on an initial neural network model trained in advance and a pre-specified class of an image to be generated, and when the classification loss function is smaller, the robustness of the model is better. The specific objective function formula is as follows:
wherein, x is*Representing the objective function, H representing the length of the size of the iterative image, W representing the width of the iterative image, C being the RGB (R (red), G (green), B (blue)) channels,
a domain representing an iterative image, L (-) is a classification loss function of a pre-trained initial neural network model, φ (x) is the initial neural network model, φ (x) is a domain representing an iterative image0Is a pre-specified category of image to be generated,
is the first regularization term that is,
is the second regularization term.
Step S24: the method comprises the steps of obtaining an initial neural network model which is trained in advance, and then extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model.
Step S25: and creating a noise image, taking the noise image as a primary iteration image, and then extracting a feature map of the primary iteration image.
Step S26: and taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data.
For more specific processing procedures of the steps S24, S25, and S26, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Therefore, the global variance in the first regular term in the embodiment of the application has the effect that the difference between each pixel point in an image and the surrounding pixel points is not too large, the L2 regular term has the effect that the generated image is more harmonious, and through the setting of the global variance and the L2 regular term, the first regular term can control the natural characteristics of the image, that is, the first regular term can realize that the generated image is more similar to a natural image, and because the second regular term enables the generated image to be more vivid, the generated image data is closer to the original data of the initial neural network model, the generated image data can replace the original data of the initial neural network model, so that the difficulty of subsequently retraining the initial neural network model is reduced, and the accuracy of the subsequently retrained initial neural network model is ensured.
Referring to fig. 3, an embodiment of the present application discloses a neural network model training method, including:
step S31: and acquiring a pre-trained initial neural network model.
In this embodiment, before retraining the model, an initial neural network model that has been trained with the original data set is obtained.
Step S32: and determining the image demand according to the training precision requirement.
In this embodiment, the training precision requirement is also the precision that the trained neural network model needs to achieve, and the required quantity of the image is determined according to the precision requirement. It can be understood that when the model is trained, the more comprehensive the training data is, the better the performance of the trained model is, that is, the higher the accuracy is, but at the same time, too much training data also brings more calculation amount and memory occupation. Therefore, the number of training data needs to be determined according to specific requirements, and when the training precision requirement is not too high, a proper image demand is obtained, so that the excessive calculation amount and memory occupation are avoided.
Step S33: the image data generation method disclosed in the foregoing is repeatedly performed a plurality of times based on the image demand to obtain a training data set containing a corresponding number of image data.
In this embodiment, after the image demand is determined according to the training accuracy requirement, a corresponding number of image data are generated by the image data generation method disclosed in the foregoing, so that the image data are sorted and used as a training data set.
Step S34: and retraining the initial neural network model by using the training data set to obtain an optimized neural network model.
In this embodiment, as shown in fig. 4, after the training data sets with the quantity corresponding to the image demand are obtained, the initial neural network model is iteratively retrained, that is, model fine-tuned, by using the training data sets, so as to obtain the optimized neural network model.
In this embodiment, after obtaining the optimized neural network model, the method may further include: the optimized neural network model is verified to determine a rate of accuracy degradation of the optimized neural network model compared to the initial neural network model. Under the condition of medium and high bit retraining, the accuracy rate of the optimized neural network model is equal to that of the initial neural network model, and under the condition of ultra-low bit, namely less than or equal to 4 bits, the accuracy rate reduction rate of the optimized neural network model is kept at 10% compared with that of the initial neural network model. When the accuracy degradation rate of the optimized neural network model is too high compared to the initial neural network model, the optimized neural network model cannot be accepted for use.
Therefore, the method comprises the steps of firstly obtaining an initial neural network model which is trained in advance; determining the image demand according to the training precision requirement; repeatedly executing the image data generation method disclosed in the foregoing multiple times based on the image demand to obtain a training data set including a corresponding number of image data; and retraining the initial neural network model by using the training data set to obtain an optimized neural network model. Therefore, the training data used for training the initial neural network model can be obtained by repeatedly executing the image data generation method disclosed by the application for multiple times based on the image demand, the original data of the pre-trained initial neural network model does not need to be obtained, the problem that the initial neural network model cannot be trained again due to the fact that the original data are difficult to obtain is solved, the image demand is determined according to the training precision demand, efficient training of the initial neural network model is achieved, and good precision can be kept.
Referring to fig. 5, an embodiment of the present application discloses an image data generation apparatus, including:
a function construction module 11, configured to construct an objective function including a first regularization term and a second regularization term; the first regular term takes an iterative image as a variable parameter, the second regular term takes a feature map of the iterative image as a variable parameter, and the iterative image of non-primary iteration is an image constructed based on a function value of the objective function;
the characteristic diagram information extraction module 12 is configured to acquire an initial neural network model trained in advance, and then extract pre-stored characteristic diagram information from a normalization layer of the initial neural network model;
an iterative image creating module 13, configured to create a noise image, use the noise image as a primary iterative image, and then extract a feature map of the primary iterative image;
and the image data generation module 14 is configured to take the feature map of the first iteration image, the pre-stored feature map information, and the first iteration image as first iteration parameters, and perform multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data, so as to retrain the initial neural network model by using the image data as training data.
As can be seen, the present application first constructs an objective function including a first regularization term and a second regularization term; the first regular term takes an iterative image as a variable parameter, the second regular term takes a feature map of the iterative image as a variable parameter, and the iterative image of non-primary iteration is an image constructed based on a function value of the objective function; acquiring a pre-trained initial neural network model, and extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model; creating a noise image, taking the noise image as a primary iteration image, and then extracting a feature map of the primary iteration image; and taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data. It can be seen that the present application provides for a method of generating a normalized image by creating an objective function comprising a first regularization term and a second regularization term, inputting pre-stored characteristic diagram information, a noise image and a characteristic diagram of the noise image in the pre-trained initial neural network model into the objective function, to generate image data corresponding to the initial neural network model trained in advance as training data, therefore, the original data of the initial neural network model which is trained in advance is not required to be obtained, the difficulty of obtaining the training data is reduced, and the obtained image data is ensured to be closer to the original data by carrying out a plurality of times of iterative operation on the target function based on the preset iteration times, and then the image data can be used as training data, and the efficiency of retraining the initial neural network model by using the training data set subsequently is improved.
In some embodiments, the function building module 11 includes:
and the classification loss function building unit is used for building a classification loss function corresponding to the initial neural network model based on the initial neural network model trained in advance and a pre-specified image class to be generated.
The target function construction unit is used for constructing a first regular term which takes the iterative image as a variable parameter and is used for controlling the natural characteristics of the image, and constructing a second regular term which takes a feature map of the iterative image as a variable parameter and is used for controlling the fidelity of the image; constructing an objective function comprising the first regularization term, the second regularization term, and the classification loss function.
In some embodiments, the function building module 11 includes:
the first regularization term creating unit is used for respectively constructing a global variance and an L2 regularization term by taking the iterative image as a variable parameter, and distributing corresponding weight coefficients to the global variance and the L2 regularization term to obtain a first regularization term used for controlling the natural characteristics of the image.
And the second regular item creating unit is used for taking the feature map of the iterative image as a variable parameter, constructing a feature map information calculation function, and constructing a second regular item for controlling the image fidelity degree based on the difference between the feature map information calculation function and the pre-stored feature map information.
In some specific embodiments, the second regularization term creation unit includes:
and the mean variance information extraction unit is used for extracting pre-stored feature map mean information and feature map variance information from the normalization layer of the initial neural network model.
And the mean variance function construction unit is used for respectively constructing a feature map mean function and a feature map variance function by taking the feature map of the iterative image as a variable parameter.
An L2 regular term construction unit, configured to construct a first L2 regular term based on a difference between the feature map mean function and the feature map mean information, and construct a second L2 regular term based on a difference between the feature map variance function and the feature map variance information.
A second regularization term construction unit for constructing a second regularization term for controlling a degree of image fidelity including the first L2 regularization term and the second L2 regularization term.
In some embodiments, the image data generation module 14 includes:
the hyper-parameter configuration unit is used for determining hyper-parameter information for configuring the target function and configuring the target function by utilizing the hyper-parameter information; wherein, one or more parameter information of learning rate, batch size and iteration frequency is determined as the hyper-parameter information.
Fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present invention. The method specifically comprises the following steps: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. Wherein the memory 22 is used for storing a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the image data generating method executed by the computer device disclosed in any of the foregoing embodiments.
In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the computer device 20; the communication interface 24 can create a data transmission channel between the computer device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
The processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 21 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 21 may further include an AI (Artificial Intelligence) processor for processing a calculation operation related to machine learning.
In addition, the storage 22 is used as a carrier for storing resources, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., the resources stored thereon include an operating system 221, a computer program 222, data 223, etc., and the storage may be a transient storage or a permanent storage.
The operating system 221 is used for managing and controlling each hardware device and the computer program 222 on the computer device 20, so as to realize the operation and processing of the mass data 223 in the memory 22 by the processor 21, which may be Windows, Unix, Linux, or the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the image data generation method by the computer device 20 disclosed in any of the foregoing embodiments. The data 223 may include data received by the computer device and transmitted from an external device, data collected by the input/output interface 25, and the like.
Furthermore, an embodiment of the present invention further discloses a storage medium, in which a computer program is stored, and when the computer program is loaded and executed by a processor, the method steps executed in the image data generating process disclosed in any of the foregoing embodiments are implemented.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The image data generation method, the model training method, the device, the equipment and the medium provided by the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. An image data generation method characterized by comprising:
constructing an objective function containing a first regular term and a second regular term; the first regular term takes an iterative image as a variable parameter, the second regular term takes a feature map of the iterative image as a variable parameter, and the iterative image of non-primary iteration is an image constructed based on a function value of the objective function;
acquiring a pre-trained initial neural network model, and extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model;
creating a noise image, taking the noise image as a primary iteration image, and then extracting a feature map of the primary iteration image;
and taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data.
2. The image data generation method according to claim 1, wherein the constructing an objective function including a first regularization term and a second regularization term includes:
constructing a classification loss function corresponding to the initial neural network model based on the initial neural network model trained in advance and a pre-specified image class to be generated;
constructing a first regular term which takes the iterative image as a variable parameter and is used for controlling the natural characteristics of the image, and constructing a second regular term which takes a feature map of the iterative image as a variable parameter and is used for controlling the fidelity of the image;
constructing an objective function comprising the first regularization term, the second regularization term, and the classification loss function.
3. The image data generation method according to claim 2, wherein the constructing a first regularization term for controlling natural characteristics of the image with the iterative image as a variable parameter includes:
and respectively constructing a global variance and an L2 regular term by taking the iterative image as variable parameters, and distributing corresponding weight coefficients for the global variance and the L2 regular term to obtain a first regular term for controlling the natural characteristics of the image.
4. The image data generation method according to claim 2, wherein the constructing of the second regularization term for controlling the degree of image fidelity with the feature map of the iterative image as a variable parameter includes:
and constructing a feature map information calculation function by taking the feature map of the iterative image as a variable parameter, and constructing a second regular term for controlling the image fidelity based on the difference between the feature map information calculation function and the pre-stored feature map information.
5. The method according to claim 4, wherein the extracting pre-saved feature map information from the normalization layer of the initial neural network model comprises:
extracting pre-stored feature map mean information and feature map variance information from a normalization layer of the initial neural network model;
correspondingly, the constructing a feature map information calculation function by using the feature map of the iterative image as a variable parameter, and constructing a second regular term for controlling the image fidelity based on the difference between the feature map information calculation function and the pre-stored feature map information includes:
respectively constructing a feature map mean function and a feature map variance function by taking the feature map of the iterative image as a variable parameter;
constructing a first L2 regularization term based on a difference between the feature map mean function and the feature map mean information, and a second L2 regularization term based on a difference between the feature map variance function and the feature map variance information;
constructing a second regularization term for controlling image fidelity that includes the first L2 regularization term and the second L2 regularization term.
6. The image data generation method according to any one of claims 1 to 5, wherein before the performing a plurality of iterative operations on the objective function based on a preset number of iterations to generate corresponding image data, the method further includes:
determining hyper-parameter information for configuring the target function, and configuring the target function by using the hyper-parameter information; wherein, one or more parameter information of learning rate, batch size and iteration frequency is determined as the hyper-parameter information.
7. A neural network model training method is characterized by comprising the following steps:
acquiring an initial neural network model which is trained in advance;
determining the image demand according to the training precision requirement;
repeatedly performing the image data generation method of any one of claims 1 to 6 a plurality of times based on the image demand to obtain a training data set containing a corresponding number of image data;
and retraining the initial neural network model by using the training data set to obtain an optimized neural network model.
8. An image data generation apparatus characterized by comprising:
the function building module is used for building an objective function containing a first regular term and a second regular term; the first regular term takes an iterative image as a variable parameter, the second regular term takes a feature map of the iterative image as a variable parameter, and the iterative image of non-primary iteration is an image constructed based on a function value of the objective function;
the characteristic diagram information extraction module is used for acquiring an initial neural network model which is trained in advance, and then extracting pre-stored characteristic diagram information from a normalization layer of the initial neural network model;
the iterative image creating module is used for creating a noise image, taking the noise image as a primary iterative image and then extracting a feature map of the primary iterative image;
and the image data generation module is used for taking the feature map of the initial iteration image, the pre-stored feature map information and the initial iteration image as initial iteration parameters, and performing multiple iteration operations on the objective function based on preset iteration times to generate corresponding image data so as to retrain the initial neural network model by taking the image data as training data.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program for carrying out the steps of the image data generation method according to any one of claims 1 to 6.
10. A computer-readable storage medium for storing a computer program; wherein the computer program realizes the steps of the image data generation method according to any one of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111278700.3A CN114140619A (en) | 2021-10-31 | 2021-10-31 | Image data generation method, model training method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111278700.3A CN114140619A (en) | 2021-10-31 | 2021-10-31 | Image data generation method, model training method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114140619A true CN114140619A (en) | 2022-03-04 |
Family
ID=80391870
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111278700.3A Pending CN114140619A (en) | 2021-10-31 | 2021-10-31 | Image data generation method, model training method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114140619A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115563687A (en) * | 2022-10-27 | 2023-01-03 | 湖北绿森林新材料有限公司 | 3D printing decorative prefabricated part and design method and system |
-
2021
- 2021-10-31 CN CN202111278700.3A patent/CN114140619A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115563687A (en) * | 2022-10-27 | 2023-01-03 | 湖北绿森林新材料有限公司 | 3D printing decorative prefabricated part and design method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111758105A (en) | Learning data enhancement strategy | |
| EP4350572A1 (en) | Method, apparatus and system for generating neural network model, devices, medium and program product | |
| CN112771578A (en) | Image generation using subdivision scaling and depth upscaling | |
| CN115965840A (en) | Image style migration and model training method, device, equipment and medium | |
| CN111652376A (en) | Quantum bit signal reading method and device | |
| CN114140619A (en) | Image data generation method, model training method, device, equipment and medium | |
| CN117710534B (en) | Animation collaborative making method based on improved teaching and learning optimization algorithm | |
| CN113837965B (en) | Image definition identification method and device, electronic equipment and storage medium | |
| CN114819163A (en) | Quantum generation countermeasure network training method, device, medium, and electronic device | |
| EP3896608A2 (en) | Meme generation method, apparatus, electronic device, storage medium and program product | |
| CN113052143A (en) | Handwritten digit generation method and device | |
| CN110866866B (en) | Image color imitation processing method and device, electronic equipment and storage medium | |
| CN114758130B (en) | Image processing and model training method, device, equipment and storage medium | |
| WO2024001108A1 (en) | Text answer determination method and apparatus, device, and medium | |
| CN115082800B (en) | Image segmentation method | |
| CN116777732A (en) | Image generation method, device, equipment and storage medium based on random noise | |
| CN113592074B (en) | Training method, generating method and device and electronic equipment | |
| CN115587173A (en) | Dialog text prediction method, device, equipment and storage medium | |
| CN114358198A (en) | Instance segmentation method and device and electronic equipment | |
| CN113327194A (en) | Image style migration method, device, equipment and storage medium | |
| CN112445933A (en) | Model training method, device, equipment and storage medium | |
| KR102684693B1 (en) | Method and Apparatus for Extrapolating of Adaptive Image Through Graph-Based Patch Rearrangement, and Computer Program | |
| US11875446B2 (en) | Procedural media generation | |
| CN117910536B (en) | Text generation method, and model gradient pruning method, device, equipment and medium thereof | |
| CN112507197B (en) | Model searching method, device, electronic equipment, storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |