CN114219095A - Training method and device of machine learning model and readable storage medium - Google Patents
Training method and device of machine learning model and readable storage medium Download PDFInfo
- Publication number
- CN114219095A CN114219095A CN202111350153.5A CN202111350153A CN114219095A CN 114219095 A CN114219095 A CN 114219095A CN 202111350153 A CN202111350153 A CN 202111350153A CN 114219095 A CN114219095 A CN 114219095A
- Authority
- CN
- China
- Prior art keywords
- training
- preset
- enhancement
- training sample
- machine learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012549 training Methods 0.000 title claims abstract description 283
- 238000010801 machine learning Methods 0.000 title claims abstract description 70
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000012545 processing Methods 0.000 claims abstract description 57
- 238000004590 computer program Methods 0.000 claims description 13
- 230000002708 enhancing effect Effects 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 7
- 241001134453 Lista Species 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000001965 increasing effect Effects 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 238000013145 classification model Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000010008 shearing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a training method, a device and a readable storage medium of a machine learning model, wherein the method comprises the following steps: obtaining a plurality of first training samples; performing first random enhancement processing on the first training sample to obtain a second training sample; after the second training sample is judged to meet the preset enhancement condition based on the preset enhancement total number, second random enhancement processing is carried out on the second training sample to obtain a third training sample, wherein the preset enhancement total number is a divisor of the batch processing number; and training the machine learning model by using the training samples to obtain the trained machine learning model, wherein the training samples comprise second training samples or third training samples, and the batch processing quantity is the quantity of the training samples input to the machine learning model each time. By the mode, the convergence rate of the model can be accelerated.
Description
Technical Field
The application relates to the technical field of machine learning, in particular to a training method and device of a machine learning model and a readable storage medium.
Background
In order to obtain a better model training effect, some schemes perform data cleaning operation before model training, namely screening the acquired data to acquire refined data to form training data for model training, so that the efficiency of model training is improved, but the model is easy to over-fit, so that the capability of model detection or classification in a real scene is insufficient; some schemes randomly select part of original training data (such as images, texts or voices) during each iteration, and then perform enhancement modes such as random cropping (crop), random scaling (resize) or random tone adjustment to improve the generalization capability of the model, but in continuous iterative training, the training data may bias a certain feature, so that the weight of the model is updated for a certain feature in a certain time period, and the loss (loss) fluctuation in the whole training process is large, which not only leads to infinite increase of the training time, but also leads to incapability of convergence of the training.
Disclosure of Invention
The application provides a training method and device of a machine learning model and a readable storage medium, which can accelerate the convergence speed of the model.
In order to solve the technical problem, the technical scheme adopted by the application is as follows: a training method of a machine learning model is provided, the method comprises the following steps: obtaining a plurality of first training samples; performing first random enhancement processing on the first training sample to obtain a second training sample; after the second training sample is judged to meet the preset enhancement condition based on the preset enhancement total number, second random enhancement processing is carried out on the second training sample to obtain a third training sample, wherein the preset enhancement total number is a divisor of the batch processing number; and training the machine learning model by using the training samples to obtain the trained machine learning model, wherein the training samples comprise second training samples or third training samples, and the batch processing quantity is the quantity of the training samples input to the machine learning model each time.
In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided an electronic device comprising a memory and a processor connected to each other, wherein the memory is used for storing a computer program, and the computer program is used for implementing the training method of the machine learning model in the above technical solution when being executed by the processor.
In order to solve the above technical problem, another technical solution adopted by the present application is: a computer-readable storage medium is provided, which is used for storing a computer program, and when the computer program is executed by a processor, the computer program is used for implementing the training method of the machine learning model in the above technical solution.
Through the scheme, the beneficial effects of the application are that: firstly, carrying out first random enhancement treatment on an obtained first training sample to obtain a second training sample; then, judging whether the current second training sample meets a preset enhancement condition or not by using a preset enhancement total number, and if the current second training sample meets the preset enhancement condition, performing second random enhancement processing on the second training sample to obtain a third training sample; if the current second training sample does not meet the preset enhancement condition, the second training sample is not processed, so that the second random enhancement processing of the second training sample is selected; then training the machine learning model by using the second training sample and/or the third training sample; by adopting at least one random enhancement treatment, the diversity of training samples is increased, the phenomenon of model overfitting can not occur, all the characteristics to be trained participate in the training as uniformly as possible in the whole training process, the situation that the weight in the model is updated in a certain period is too single can not be caused, the weight updating is more balanced, and then the loss value in the training process can not generate huge fluctuation, and the convergence speed of the machine learning model can be accelerated.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. Wherein:
FIG. 1 is a schematic flow chart diagram illustrating an embodiment of a method for training a machine learning model provided herein;
FIG. 2 is a schematic flow chart diagram illustrating another embodiment of a training method for a machine learning model provided herein;
FIG. 3 is a schematic flow chart of step 206 in the embodiment shown in FIG. 2;
FIG. 4 is a schematic diagram of random occlusion of an image provided herein;
FIG. 5 is a schematic structural diagram of an embodiment of an electronic device provided in the present application;
FIG. 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided in the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be noted that the following examples are only illustrative of the present application, and do not limit the scope of the present application. Likewise, the following examples are only some examples and not all examples of the present application, and all other examples obtained by a person of ordinary skill in the art without any inventive work are within the scope of the present application.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
It should be noted that the terms "first", "second" and "third" in the present application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of the feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless explicitly specifically limited otherwise. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an embodiment of a training method of a machine learning model provided in the present application, the method including:
step 11: a plurality of first training samples is obtained.
The machine learning model may be a common model such as a neural network, a linear regression model, a logistic regression model, or a support vector machine, and data participating in calculation of the loss function is selected as a first training sample according to the loss function of the trained machine learning model, where the first training sample may be an image, a text, or audio, and may be obtained directly from an existing database, or may be obtained by means of shooting or recording.
Further, taking a machine learning model as an example of a neural network model, a loss function is an objective function for training the neural network model, the function is used for calculating the loss between a predicted value and a true value (namely, a label value) of the neural network model obtained based on a current training sample, and the weight of the neural network model is updated through the loss by adopting a back propagation algorithm so as to reduce the loss and further enhance the detection capability of the model.
Step 12: and carrying out first random enhancement processing on the first training sample to obtain a second training sample.
After the first training sample is obtained, in order to expand the diversity of training samples used for training the machine learning model and accelerate the convergence rate or accuracy of the machine learning model, some of the first training samples may be subjected to enhancement processing or all of the first training samples may be subjected to enhancement processing, for example: the first random enhancement process is performed on the first training sample to generate the second training sample, and the first random enhancement process may be a conventional enhancement method, that is, the first random enhancement process includes horizontal flipping, vertical flipping, rotation, histogram equalization, random clipping, random scaling, or random tone adjustment (for example, adjusting contrast, saturation, brightness, or sharpness of an image).
Step 13: and after the second training sample is judged to meet the preset enhancement condition based on the preset enhancement total number, carrying out second random enhancement treatment on the second training sample to obtain a third training sample.
After the first training sample is subjected to the enhancement processing for one time, selectively performing second random enhancement processing on a second generated training sample; specifically, a preset total number of enhancements and a batch number (batch _ size) may be obtained, where the batch number is the number of training samples input to the machine learning model each time (i.e., the number of samples per iteration), and the preset total number of enhancements is a divisor of the batch number, i.e., the batch number is an integer multiple of the preset total number of enhancements; then, whether the current second training sample meets a preset enhancement condition is determined by using the preset enhancement total number, if the current second training sample meets the preset enhancement condition, second random enhancement processing is performed on the second training sample to generate a third training sample, and the second random enhancement processing may be random occlusion processing or other enhancement modes, such as: filtering, image or Gamma (Gamma) correction, the second random enhancement process being different from the first random enhancement process.
In a specific embodiment, the preset enhancement total is the same as the batch processing number, and it can be determined whether the sum of the number of the first training samples that have been currently subjected to enhancement processing and the number of the third training samples (recorded as the sample total number of the current round) is greater than the batch processing number, if the sample total number of the current round is greater than the batch processing number, it indicates that the sample number required by the current training is sufficient, and at this time, the second random enhancement processing is not required to be performed on the second training sample; if the total number of samples in the round is less than the batch processing number, the training is not enough, and the second random enhancement processing is carried out on the second training sample. Or, whether the ratio of the total number of the current third training samples to the batch processing number is greater than a preset ratio or not can be judged, if so, the second random enhancement processing is not performed on the second training samples, and the first training samples, the second training samples and the third training samples which are currently subjected to enhancement processing are screened to form current training samples, wherein the current training samples are data required by training currently input to the machine learning model, and the number of the samples in the current training samples is the same as the batch processing number; and if the ratio is smaller than the preset ratio, continuing to perform second random enhancement processing on the second training sample to generate a third training sample.
In other embodiments, the first training sample may be subjected to the second random enhancement processing, and then subjected to the first random enhancement processing; or the first training sample is further subjected to the enhancement processing more than twice, and the mode and the times of the enhancement processing can be set according to the specific application requirements.
Step 14: and training the machine learning model by using the training samples to obtain the trained machine learning model.
After the enhancement processing of the first training sample is finished, training the machine learning model by using the current training sample obtained in the steps 11-13 so as to obtain a trained machine learning model; specifically, when the second training sample meets the preset enhancement condition, the current training sample is a third training sample; and when the second training sample does not meet the preset enhancement condition, the current training sample is the second training sample, namely the current training sample is data obtained by performing random enhancement processing on the first training sample at least once.
It is understood that the training method of the machine learning model is similar to the model training method in the related art, and the setting of the parameters used for training is also similar to the existing scheme, such as: the number of batches, the preset loss threshold, or the preset number of iterations threshold, will not be described in detail herein.
The embodiment provides a method for enhancing training data used in machine learning model training, which comprises the steps of firstly carrying out first random enhancement processing on a first training sample, and then carrying out selective second random enhancement processing on data after random enhancement; in addition to random shearing, random zooming or random tone adjustment, the method can randomly shield the training data in a small range, thereby increasing the diversity of the training data and avoiding overfitting; in addition, all the characteristics to be trained in the training data participate in the training as uniformly as possible in the whole training process, and due to the fact that a certain attribute in the training data is shielded, the model learning characteristics can be more comprehensive, so that the model is more generalized, the training time is further shortened, and the convergence speed of the model is accelerated.
Referring to fig. 2, fig. 2 is a schematic flowchart illustrating a method for training a machine learning model according to another embodiment of the present application, the method including:
step 201: a plurality of first training samples is obtained.
Step 202: the enhanced statistics table is initialized.
The enhancement statistical table includes first training samples of different categories and enhancement times corresponding to the first training samples, and all the enhancement times in the enhancement statistical table may be initialized to a first preset value, where the first preset value may be 0. It is understood that the first training samples in the enhanced statistical table may be all the first training samples, or may be some of all the first training samples; for example, taking the machine learning model as an example to perform the classification task, assuming that the total number of the first training samples is 1 ten thousand and the number of the classifications is 1000, the length of the enhanced statistical table may be 1000, that is, it includes 1000 first training samples, and the categories of the first training samples are different.
Step 203: and selecting a first training sample with the enhancement times of a first preset value from the enhancement statistical table to form the enhancement sample set of the current round.
After the initialization of the enhanced statistical table is completed, selecting a first training sample with the enhanced times being a first preset value in the enhanced statistical table to form a current round enhanced sample set, wherein the current round enhanced sample set comprises at least one first training sample with the enhanced times being the first preset value; the enhancing times in the enhanced statistical table are all the second preset values, and the step of initializing the enhanced statistical table, that is, the step 202 is executed, where the second preset values are greater than the first preset values, for example: the second preset value is 1.
Further, the number of the first training samples in the local round of enhancement sample set is related to the number of batch processing, and the ratio of the number of batch processing to the preset enhancement total number is recorded as a preset ratio, wherein the preset ratio can be the number of the first training samples in the local round of enhancement sample set; specifically, when the number of first training samples with the enhancement times being a first preset value in the enhancement statistical table is greater than C, C first training samples can be randomly selected from all the first training samples meeting the conditions; when the number of the first training samples with the enhancement times of the first preset value in the enhancement statistical table is less than C, the enhancement statistical table can be initialized. For example, assuming that the number of the first training samples in the enhancement sample set of the current round is 3 and the batch processing number is 30, the preset enhancement total number is 10, that is, the processing of steps 205 to 206 is performed on all the 3 first training samples in the enhancement sample set of the current round.
Step 204: and carrying out first random enhancement processing on the first training sample to obtain a second training sample.
Step 205: and initializing the updated number of the samples in the current round to a third preset value.
The updated number of samples in this round is the number of times of the first training samples that have been subjected to the enhancement processing in the process of performing the enhancement processing on the first training samples at present, and the updated number of samples in this round may be initialized to 0 before each round of the enhancement processing is performed, that is, the third preset value is 0.
Step 206: and after judging that the second training sample meets the preset enhancement condition based on the comparison result of the updated number of the samples in the current round and the preset enhancement total number, carrying out random shielding treatment on the second training sample to obtain a third training sample.
Comparing the updated number of the samples in the current round with the preset enhanced total number to obtain a comparison result; then judging whether the current second training sample meets a preset enhancement condition or not based on the comparison result; if the current second training sample meets the preset enhancement condition, performing second random enhancement treatment on the second training sample to obtain a third training sample; specifically, the scheme shown in fig. 3 may be adopted to determine whether the currently acquired second training sample meets the preset enhancement condition, including the following steps:
step 31: and judging whether the updating number of the samples in the current round is smaller than the preset enhancement total number.
Step 32: and if the updated number of the samples in the current round is less than the preset enhancement total number, judging whether the current second training sample meets the preset occlusion condition.
When the updating number of the samples in the current round is smaller than the preset enhancement total number, randomly generating a shielding factor within a preset numerical range; then judging whether the shielding factor is larger than a preset shielding threshold value or not; if the shielding factor is larger than a preset shielding threshold value, determining that the current second training sample meets a preset shielding condition; and if the shielding factor is smaller than or equal to the preset shielding threshold value, determining that the current second training sample does not meet the preset shielding condition.
Step 33: and if the current second training sample meets the preset shielding condition, determining that the second training sample meets the preset enhancement condition, and performing second random enhancement processing on the second training sample to obtain a third training sample.
The second training sample comprises a plurality of first data to be processed, if the currently generated occlusion factor is larger than a preset occlusion threshold value, the current second training sample is indicated to meet a preset enhancement condition, and at least one first data to be processed can be randomly selected from all the first data to be processed to obtain second data to be processed; then randomizing the second data to be processed to obtain third data to be processed; and replacing the corresponding first data to be processed in the second training sample by the third data to be processed to obtain a third training sample. For example, if the second training sample is an image, the first data to be processed is a pixel value, a region may be randomly selected from the image, and the pixel value of the region may be replaced with a preset pixel value, such as: all replaced with 0 or 255, or the pixel values of this area are randomized.
Step 34: and adding the updated number of the samples in the current round with the preset step length.
When the current second training sample does not meet the preset shielding condition, adding the updated number of the samples in the current round with a preset step length, wherein the preset step length can be 1; after the second training sample is subjected to shielding treatment, adding the updated number of the samples in the current round with the preset step length; after the updated number of samples in the current round is added to the preset step length, the step of judging whether the updated number of samples in the current round is smaller than the preset total number of enhancements, that is, the step 31, is executed until the updated number of samples in the current round is judged to be larger than or equal to the preset total number of enhancements. And if the update quantity of the samples in the current round is larger than or equal to the batch processing quantity, executing the operation of training the machine learning model by using the training samples.
For example, assuming that the first training sample is an image (denoted as a current image), the resolution of the current image is w × h, the preset value range may be 0-1, the preset occlusion threshold is 0.2, the preset enhancement total number is denoted as T, and the updated number of samples in this round is denoted as i, the following operations are performed:
i is initialized to 0.
Judging whether i is smaller than T, if i is smaller than T, executing step (c), otherwise ending the enhancement process.
Randomly generating a decimal (namely a shielding factor) of 0-1, if the shielding factor is less than 0.2, not performing random shielding treatment, and executing the fifth step; if the shielding factor is greater than or equal to 0.2, random shielding processing is carried out, and the step (four) is executed.
(r) occlusion of the random position of the current image using the solid black squares of 1/36 (i.e., (w/6) × (h/6)) of the resolution of the current image.
As shown in fig. 4, after blocking part of the mouth of the big-mouth bird, one of the characteristics of the big-mouth bird is no longer significant, so that the model can be forced to pay attention to learning other characteristics of the big-mouth bird, such as: the shape or color of the giant-mouth bird, etc., thereby increasing the generalization ability of the model.
And (6) returning to execute the step (II) until i is larger than or equal to T.
Step 207: and training the machine learning model by using the training samples.
Step 207 is the same as step 14 in the above embodiment, and will not be described again here.
Step 208: and judging whether the current machine learning model meets the preset training end condition or not based on the label value corresponding to the training sample and the predicted value output by the machine learning model.
Firstly, calculating a loss value between a label value corresponding to a current training sample and a predicted value output by a machine learning model; and then judging whether the loss value is smaller than a preset loss threshold or whether the current training times is larger than a preset iteration time threshold, wherein the current training times are the total times of training till now.
In other embodiments, the preset training end condition may also be: the loss value is converged or the accuracy obtained when the test set is used for testing reaches a set condition (for example, the accuracy exceeds a preset accuracy), and the like.
Step 209: and if the current machine learning model does not meet the preset training end condition, updating the enhancement times corresponding to the first training sample in the enhancement sample set of the current round.
And if the current machine learning model does not meet the preset training end condition, updating the enhancement times corresponding to the first training sample in the enhancement sample set of the current round, such as adding one to the enhancement times, and returning to the step of selecting the first training sample with the enhancement times being the first preset value from the enhancement statistical table until the machine learning model meets the preset training end condition to obtain the trained machine learning model.
Step 210: and if the current machine learning model meets the preset training end condition, obtaining the trained machine learning model.
And if the current loss value is smaller than the preset loss threshold value or the current training times is larger than the preset iteration time threshold value, determining that the current machine learning model meets the preset training ending condition, and ending the training at the moment.
In a specific embodiment, assuming that the machine learning model is a classification model, the first training samples are images, and taking the classification model as a Residual Network (rest) and the training set as a database ImageNet as an example, 1000 classes of features participating in calculating the objective function are provided, that is, the length of an enhanced statistical table (denoted as listA) is 1000, and the number of the first training samples in the enhanced sample set of the current round is denoted as C, the following steps are performed:
1) the number of enhancements in listA is initialized to 0.
2) Selecting images with the enhancement times of 0 from listA (if the images with the enhancement times of 0 are too many, C images with the enhancement times of 0 can be randomly selected from the images), carrying out image enhancement on the images, and then carrying out iterative training.
3) And (4) judging whether the current whole training process reaches a preset training end condition, and if not, executing the step 4).
4) Counting the images used in the iterative training, updating the enhancement times corresponding to the C images in the listA to 1 (if the enhancement times in the listA are all 1, resetting the enhancement times in the listA to 0), and then entering the step 2) to continue the training.
On the basis of 2 1080 Ti-based Graphics Processing Units (GPUs) and a Pythroch training framework, the following results can be obtained through experiments: this scheme reaches more fast than traditional scheme and predetermines the loss threshold value, and training duration shortens to 3 days from 4 days, can save 25% training time, and the precision (Accuracy, acc) of model has also promoted 1.2%.
In the whole training process of the model, the training samples used in the next training are balanced according to the training samples already participating in the training, so that the aim of enabling all the training samples to participate in the training uniformly is fulfilled, the situation that the weight in the model is updated only singly in a certain period of time is avoided, the weight updating is more balanced, the loss value in the training process is prevented from generating huge fluctuation, and the convergence speed can be accelerated; moreover, the embodiment also provides an enhancement method for randomly shielding the training samples in the training process, so that the model does not bias a certain significant feature in the learning process, and more other features can be learned, thereby increasing the generalization capability of the model and improving the reasoning accuracy of the model.
Referring to fig. 5, fig. 5 is a schematic structural diagram of an embodiment of an electronic device provided in the present application, the electronic device 50 includes a memory 51 and a processor 52 connected to each other, the memory 51 is used for storing a computer program, and the computer program is used for implementing the training method of the machine learning model in the foregoing embodiment when being executed by the processor 52.
All training data can participate in training uniformly in the whole training process, so that the weight updating in a model in a certain period is not too single, the weight updating is more balanced, the loss value is reduced uniformly, the loss value cannot generate overlarge fluctuation, the training is accelerated to be converged, and the training time is shortened; meanwhile, as the training data is specially enhanced (namely, the training data is subjected to second random enhancement processing), the model does not bias a certain significant feature in the learning process, and more other features can be learned, so that the generalization capability of the model is improved, and the reasoning accuracy of the model is improved.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium 60 provided by the present application, where the computer-readable storage medium 61 is used for storing a computer program 61, and when the computer program 61 is executed by a processor, the computer program is used for implementing a training method of a machine learning model in the foregoing embodiment.
The computer-readable storage medium 60 may be a server, a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of modules or units is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are intended to be included within the scope of the present application.
Claims (10)
1. A method for training a machine learning model, comprising:
obtaining a plurality of first training samples;
performing first random enhancement processing on the first training sample to obtain a second training sample;
after the second training sample is judged to meet the preset enhancement condition based on the preset enhancement total number, second random enhancement processing is carried out on the second training sample to obtain a third training sample, wherein the preset enhancement total number is a divisor of the batch processing number;
training the machine learning model by using training samples to obtain the trained machine learning model, wherein the training samples comprise the second training sample or the third training sample, and the batch processing quantity is the quantity of the training samples input to the machine learning model each time.
2. The method for training a machine learning model according to claim 1, wherein the step of performing the first stochastic enhancement on the first training sample to obtain the second training sample is preceded by the step of:
initializing an enhancement statistical table, wherein the enhancement statistical table comprises the first training samples of different categories and enhancement times corresponding to the first training samples;
selecting a first training sample with the enhancement times of a first preset value from the enhancement statistical table to form a current round of enhancement sample set;
and when the enhancing times in the enhancing statistical table are all the second preset values, the step of initializing the enhancing statistical table is executed.
3. The method for training a machine learning model according to claim 2, wherein the step of performing the second stochastic enhancement on the second training sample to obtain a third training sample is preceded by the step of:
initializing the updated number of the samples in the current round to a third preset value;
comparing the updated number of the samples in the current round with the preset total number of the enhancements to obtain a comparison result;
and judging whether the second training sample meets the preset enhancement condition or not based on the comparison result.
4. The method for training a machine learning model according to claim 3, wherein the second stochastic enhancement process is a stochastic occlusion process, and the step of comparing the updated number of samples of the current round with the preset total number of enhancements includes:
judging whether the updating number of the samples in the current round is smaller than the preset enhancement total number or not;
if the updated number of the samples in the current round is smaller than the preset enhancement total number, judging whether the current second training sample meets a preset occlusion condition;
if the current second training sample meets the preset shielding condition, determining that the second training sample meets the preset enhancement condition, and performing random shielding treatment on the second training sample to obtain a third training sample;
adding the updated number of the samples in the current round with a preset step length, and returning to the step of judging whether the updated number of the samples in the current round is smaller than the preset enhancement total number or not until the updated number of the samples in the current round is larger than or equal to the preset enhancement total number;
and after the updated number of the samples in the current round is larger than or equal to the preset enhancement total number, executing the step of training the machine learning model by using the training samples.
5. The method for training a machine learning model according to claim 4, wherein the step of determining whether the current second training sample satisfies a preset occlusion condition comprises:
randomly generating a shielding factor within a preset numerical range;
judging whether the shielding factor is larger than a preset shielding threshold value or not;
if yes, determining that the current second training sample meets the preset shielding condition.
6. The method according to claim 4, wherein the second training sample includes a plurality of first data to be processed, and the step of performing random occlusion processing on the second training sample to obtain the third training sample includes:
randomly selecting at least one first data to be processed from all the first data to be processed to obtain second data to be processed;
randomizing the second data to be processed to obtain third data to be processed;
and replacing the corresponding first data to be processed in the second training sample with the third data to be processed to obtain the third training sample.
7. A method of training a machine learning model according to claim 2, the method comprising:
judging whether the current machine learning model meets a preset training end condition or not based on the label value corresponding to the training sample and the predicted value output by the machine learning model;
and if not, updating the enhancement times corresponding to the first training sample in the enhancement sample set of the current round, and returning to the step of selecting the first training sample with the enhancement times being the first preset value from the enhancement statistical table until the machine learning model meets the preset training end condition.
8. The method for training a machine learning model according to claim 7, wherein the step of determining whether the current machine learning model satisfies a preset training end condition based on the label value corresponding to the training sample and the predicted value output by the machine learning model comprises:
calculating a loss value between a label value corresponding to the training sample and a predicted value output by the machine learning model;
judging whether the loss value is smaller than a preset loss threshold or whether the current training times are larger than a preset iteration time threshold;
if yes, determining that the current machine learning model meets the preset training end condition.
9. An electronic apparatus, comprising a memory and a processor connected to each other, wherein the memory is configured to store a computer program, which when executed by the processor is configured to implement the method of training a machine learning model of any one of claims 1-8.
10. A computer-readable storage medium storing a computer program, the computer program, when being executed by a processor, is adapted to carry out the method of training a machine learning model according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111350153.5A CN114219095B (en) | 2021-11-15 | 2021-11-15 | Training method and device for machine learning model and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111350153.5A CN114219095B (en) | 2021-11-15 | 2021-11-15 | Training method and device for machine learning model and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114219095A true CN114219095A (en) | 2022-03-22 |
| CN114219095B CN114219095B (en) | 2024-05-10 |
Family
ID=80697213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111350153.5A Active CN114219095B (en) | 2021-11-15 | 2021-11-15 | Training method and device for machine learning model and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114219095B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807448A (en) * | 2020-01-07 | 2020-02-18 | 南京甄视智能科技有限公司 | Human face key point data enhancement method, device and system and model training method |
| CN111353441A (en) * | 2020-03-03 | 2020-06-30 | 成都大成均图科技有限公司 | Road extraction method and system based on position data fusion |
| CN112101530A (en) * | 2020-11-10 | 2020-12-18 | 南京集成电路产业服务中心有限公司 | Neural network training method, device, equipment and storage medium |
| US20210303922A1 (en) * | 2019-11-16 | 2021-09-30 | Uatc, Llc | Systems and Methods for Training Object Detection Models Using Adversarial Examples |
| US20210303886A1 (en) * | 2020-03-31 | 2021-09-30 | Robert Bosch Gmbh | Semantically-consistent augmented training data for traffic light detection |
| CN113642707A (en) * | 2021-08-12 | 2021-11-12 | 深圳平安智汇企业信息管理有限公司 | Model training method, device, equipment and storage medium based on federal learning |
-
2021
- 2021-11-15 CN CN202111350153.5A patent/CN114219095B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210303922A1 (en) * | 2019-11-16 | 2021-09-30 | Uatc, Llc | Systems and Methods for Training Object Detection Models Using Adversarial Examples |
| CN110807448A (en) * | 2020-01-07 | 2020-02-18 | 南京甄视智能科技有限公司 | Human face key point data enhancement method, device and system and model training method |
| CN111178337A (en) * | 2020-01-07 | 2020-05-19 | 南京甄视智能科技有限公司 | Human face key point data enhancement method, device and system and model training method |
| CN111353441A (en) * | 2020-03-03 | 2020-06-30 | 成都大成均图科技有限公司 | Road extraction method and system based on position data fusion |
| US20210303886A1 (en) * | 2020-03-31 | 2021-09-30 | Robert Bosch Gmbh | Semantically-consistent augmented training data for traffic light detection |
| CN113468946A (en) * | 2020-03-31 | 2021-10-01 | 罗伯特·博世有限公司 | Semantically consistent enhanced training data for traffic light detection |
| CN112101530A (en) * | 2020-11-10 | 2020-12-18 | 南京集成电路产业服务中心有限公司 | Neural network training method, device, equipment and storage medium |
| CN113642707A (en) * | 2021-08-12 | 2021-11-12 | 深圳平安智汇企业信息管理有限公司 | Model training method, device, equipment and storage medium based on federal learning |
Non-Patent Citations (3)
| Title |
|---|
| GUOFA LI等: "a deep learning based image enhancemet approach for autonomous driving at night", KNOWLEDGE-BASED SYSTEMS, vol. 213, 15 February 2021 (2021-02-15), pages 1 - 10 * |
| 符祖峰;刘松;王德红;牟珍;郑维鑫;: "基于深度学习的异常事件检测研究综述", 安顺学院学报, vol. 22, no. 02, 15 April 2020 (2020-04-15), pages 130 - 135 * |
| 贾宇霞;樊帅昌;易晓梅;: "基于显著性增强和迁移学习的鱼类识别研究", 渔业现代化, vol. 47, no. 01, 15 February 2020 (2020-02-15), pages 38 - 46 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114219095B (en) | 2024-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230030267A1 (en) | Method and apparatus for selecting face image, device, and storage medium | |
| CN108765340B (en) | Blurred image processing method and device and terminal equipment | |
| US11068746B2 (en) | Image realism predictor | |
| CN108960425B (en) | Rendering model training method, system, equipment, medium and rendering method | |
| CN111160531B (en) | Distributed training method and device for neural network model and electronic equipment | |
| US9613405B2 (en) | Scalable massive parallelization of overlapping patch aggregation | |
| CN111695624B (en) | Updating method, device, equipment and storage medium of data enhancement strategy | |
| US8983188B1 (en) | Edge-aware smoothing in images | |
| CN112927144A (en) | Image enhancement method, image enhancement device, medium, and electronic apparatus | |
| US11314970B1 (en) | Reinforcement learning techniques for automated video summarization | |
| CN116363261A (en) | Training method of image editing model, image editing method and device | |
| CN108229286A (en) | Language model generates and application process, device, electronic equipment and storage medium | |
| CN111145202B (en) | Model generation method, image processing method, device, equipment and storage medium | |
| US20220155398A1 (en) | Deep learning-based eddy current correction | |
| CN113177579A (en) | Feature fusion method based on attention mechanism | |
| CN114302226B (en) | Intelligent cutting method for video picture | |
| US20220004849A1 (en) | Image processing neural networks with dynamic filter activation | |
| CN116894985A (en) | Semi-supervised image classification method and semi-supervised image classification system | |
| CN111325671B (en) | Network training method and device, image processing method and electronic equipment | |
| CN109871942B (en) | Neural network training method, device, system and storage medium | |
| CN114219095A (en) | Training method and device of machine learning model and readable storage medium | |
| CN115063641B (en) | CT artifact identification method and device based on deep learning | |
| CN114049539B (en) | Collaborative target identification method, system and device based on decorrelation binary network | |
| CN113807234B (en) | Method, device, computer equipment and storage medium for checking mouth-shaped synthesized video | |
| CN113177954B (en) | Image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |