WO2022095476A1 - 数据增强方法、装置、计算设备以及计算机可读存储介质 - Google Patents

数据增强方法、装置、计算设备以及计算机可读存储介质 Download PDF

Info

Publication number
WO2022095476A1
WO2022095476A1 PCT/CN2021/103015 CN2021103015W WO2022095476A1 WO 2022095476 A1 WO2022095476 A1 WO 2022095476A1 CN 2021103015 W CN2021103015 W CN 2021103015W WO 2022095476 A1 WO2022095476 A1 WO 2022095476A1
Authority
WO
WIPO (PCT)
Prior art keywords
data enhancement
target
strategy
operations
node
Prior art date
Application number
PCT/CN2021/103015
Other languages
English (en)
French (fr)
Inventor
罗罡
李庆瑞
李俊明
陶韬
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP21888178.7A priority Critical patent/EP4235525A4/en
Publication of WO2022095476A1 publication Critical patent/WO2022095476A1/zh
Priority to US18/313,869 priority patent/US20230273914A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2358Change logging, detection, and notification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • G06N20/20Ensemble learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/185Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting

Definitions

  • the present application relates to the technical field of artificial intelligence, and in particular, to a data enhancement method, apparatus, computing device, and computer-readable storage medium.
  • AI artificial intelligence
  • deep learning models are widely used in computer vision, natural language processing, language recognition and other fields, and have achieved excellent performance.
  • deep learning models can extract hierarchical representations from a large amount of labeled data (ie, samples labeled by labels). These learned, hierarchical representations are the key to the superior performance of deep learning models (whether for classification or regression).
  • a deep learning model with a recognition function (referred to as a recognition model) requires a large number of samples in the learning process.
  • the quantity and quality of the samples have a crucial or even decisive influence on the performance of the trained recognition model.
  • existing samples (including public or private samples) cannot fully meet the quantitative and qualitative needs of some business scenarios.
  • the data volume of the samples is insufficient or the sample categories are single, resulting in problems such as low accuracy and low generalization ability of the trained recognition model.
  • samples are generally augmented by data augmentation techniques.
  • each data augmentation operation is performed on the sample according to the operation intensity of each data augmentation operation in the data augmentation strategy, and the obtained augmented sample is used as the actual sample in the application scenario to expand The sample size for this application scenario.
  • each data enhancement operation is performed on the sample, and the obtained enhanced sample has a single category and lacks diversity.
  • the embodiments of the present application provide a data enhancement method, apparatus, computing device, chip, and computer storage medium, which can improve the diversity of samples.
  • the technical solution is as follows:
  • a data enhancement method executed by a first node, and the method includes:
  • Obtain a first target data enhancement strategy based on the first target data enhancement strategy, perform the various data enhancement operations on the first sample within the operation intensity variation interval indicated by the target operation levels of the various data enhancement operations , to obtain multiple target enhancement samples of the first sample, wherein the first target data enhancement strategy is used to indicate the target operation level of the data enhancement operation, and a target operation level is used to indicate an operation intensity variation interval.
  • This method processes the samples through a variety of data enhancement operations indicated by the data enhancement strategy, and can obtain multiple target enhanced samples, thereby achieving the purpose of expanding the samples.
  • Multiple data enhancement operations are performed on the samples within the indicated operation intensity change interval, so that the obtained multiple target enhancement samples can have changes within the operation intensity change interval indicated by the target operation level of the various data enhancement operations, which improves the performance. Diversity of samples.
  • a change corresponding to any data enhancement operation occurs in the any data in the plurality of target enhancement samples.
  • a data enhancement operation is within the operation intensity variation interval indicated by the target intensity level of the data enhancement operation.
  • the any data is any pixel in the image
  • the target enhancement sample of the first sample is an enhanced image
  • any data enhancement operation is a rotation operation.
  • Part of the enhanced image is that the first node performs different rotations on the image within the operation intensity variation interval. The rotation operation of the angle is obtained.
  • the position of any pixel point in this part of the enhanced image has undergone a rotation change, and the rotation change is reflected in the rotation angle.
  • the change in the rotation angle is within the operation intensity variation interval indicated by the target intensity level of the rotation operation. It can be understood that, for a certain fixed position in the image, since this part of the enhanced image is obtained by rotating the image with different rotation angles, the pixels at the fixed position in this part of the enhanced image have changed.
  • the first target data enhancement strategy is obtained by searching based on multiple operation levels of the configured multiple data enhancement operations.
  • Any data enhancement operation is configured with multiple operation levels, and one operation Levels are used to indicate a range of intensity variations.
  • the first target data enhancement strategy is determined based on multiple data enhancement strategies, and the multiple data enhancement strategies are obtained by searching through multiple operation levels of the configured multiple data enhancement operations, Any data enhancement operation is configured with a plurality of operation levels, and one operation level is used to indicate an intensity change interval.
  • the multiple data enhancement strategies are evaluated by multiple evaluation devices.
  • the objects of the second samples used in the evaluation of the multiple data augmentation strategies are the same object or objects of the same type.
  • the multiple operation levels configured by any one of the data enhancement operations are obtained based on the variation range of the target operation intensity of the any one of the data enhancement operations, and the target of the any one of the data enhancement operations
  • the operating intensity variation range is configured by the configuration device.
  • the configuration process of any one of the data enhancement operations includes:
  • the target operation intensity change range corresponding to the any one of the data enhancement operations is divided into multiple operation intensity change intervals; Each operation intensity change interval in the interval configures the operation level.
  • an operation level is configured for each operation intensity change interval, so that the data enhancement strategy
  • the intensity level can be used to indicate the operation intensity change interval, which simplifies the representation of the operation intensity change area interval.
  • the first target data enhancement strategy is further used to indicate the target operation probability of each data enhancement operation among the multiple data enhancement operations, and the target operation of any one of the data enhancement operations
  • the probability is the probability of performing any one of the data enhancement operations on the first sample;
  • the obtaining the first target data enhancement strategy includes:
  • the first target data enhancement strategy is determined based on a plurality of data enhancement strategies, and any data enhancement strategy is used to indicate the operation probability and operation level of each data enhancement operation among the multiple data enhancement operations, and any data enhancement strategy
  • the operation probability of the operation is the probability of performing any one of the data augmentation operations on the second sample.
  • the first node generates an optimal data enhancement strategy based on multiple data enhancement strategies, that is, the first target data enhancement strategy, so as to expand the samples based on the optimal target data enhancement strategy subsequently and increase the diversity of samples.
  • the determining of the first target data enhancement strategy based on multiple data enhancement strategies includes:
  • a plurality of second target data enhancement strategies are selected from the plurality of data enhancement strategies, and the first target data enhancement strategy is generated based on the plurality of second target data enhancement strategies data augmentation strategies;
  • the evaluation values of the plurality of second target data enhancement strategies are all higher than the evaluation values of the data enhancement strategies other than the plurality of second target data enhancement strategies among the plurality of data enhancement strategies.
  • the evaluation value of the data augmentation strategy is used to indicate the pros and cons of the recognition model trained based on the augmented samples obtained by any of the data augmentation strategies.
  • the first node may first select some optimal data enhancement strategies from the plurality of data enhancement strategies according to the evaluation values of the plurality of data enhancement strategies, such as the second target data enhancement strategy , and then generate an optimal first target data enhancement strategy based on the selected multiple second target data enhancement strategies, thereby ensuring the quality of the first target data enhancement strategy.
  • the generating the first target data enhancement strategy based on the plurality of second target data enhancement strategies includes:
  • the target operation probabilities and target operation levels of the multiple data enhancement operations are determined; based on the determined multiple data enhancement operations The target operation probability and target operation level of each data enhancement operation are used to generate the first target data enhancement strategy.
  • Target operating levels include:
  • any one of the data enhancement operations clustering multiple operation levels of the any one of the data enhancement operations in the plurality of second target data enhancement strategies to obtain at least one operation level category; for the at least one data enhancement strategy Any operation level category in an operation level category, based on the operation level in the any operation level category, any data in the third target data enhancement strategy in the plurality of second target data enhancement strategies
  • the operation probability of the enhancement operation, determining the target operation intensity and target operation probability of any one of the data enhancement operations, and the third target data enhancement strategy is the second target data enhancement to which the operation level of the any operation level category belongs Strategy.
  • the data enhancement based on the operation level in any of the operation level categories and the third target data enhancement strategy in the plurality of second target data enhancement strategies
  • the operation probability of the operation, and determining the target operation intensity and target operation probability of any one of the data enhancement operations include:
  • the determining of the target operation level of any one of the data enhancement operations based on the operation level in the any operation level category includes:
  • the first node determines each operation level that is greater than or equal to the minimum operation level and less than or equal to the maximum operation level as the target operation level of any data enhancement operation, so that all If the multiple target operation levels of any one of the data enhancement operations are continuous, then the intensity change intervals corresponding to the continuous target operation levels are also continuous, so that the first node is subsequently at the target operation level of any data enhancement operation. Perform any one of the data enhancement operations on the first sample within the indicated operation intensity change interval, and the obtained enhanced sample changes in the continuous operation intensity change interval, thereby further increasing the diversity of the samples and more in line with reality Changes in samples in the application scenario.
  • the determining of the target operation level of any one of the data enhancement operations based on the operation level in the any operation level category includes:
  • Each operation level in the any operation level category is determined as the target operation level of the any one data enhancement operation.
  • the target operation probability of any one of the data enhancement operations is the operation of any one of the data enhancement operations in the third target data enhancement strategy in the plurality of second target data enhancement strategies.
  • the average probability of probabilities is the average probability of probabilities.
  • the method before determining the first target data enhancement strategy based on multiple data enhancement strategies, the method further includes:
  • Iterative calculation is performed based on the initial data enhancement strategy to obtain the multiple data enhancement strategies.
  • the iterative calculation based on the initial data enhancement strategy includes:
  • the plurality of data enhancement strategies are determined based on the data enhancement strategy determined in the previous i-1 iteration calculation process and each data enhancement strategy in the initial data enhancement strategy.
  • the i-th data enhancement strategy in wherein the i is an integer greater than or equal to 1 or less than or equal to N, and the i is the total number of iterative calculations.
  • the plurality of data enhancement strategies are determined based on the data enhancement strategy determined in the previous i-1 iterative calculation process and each data enhancement strategy in the initial data enhancement strategy
  • the i-th data augmentation strategy in the strategy includes:
  • any one of the data enhancement operations predict the i-th data based on multiple operation levels of the any one of the data enhancement strategies and multiple evaluation values of the respective data enhancement strategies
  • the operation level of any data enhancement operation described in the enhancement strategy, and the evaluation value of any data enhancement strategy is used to indicate the pros and cons of the recognition model trained based on the enhanced samples obtained from the any data enhancement strategy;
  • any one of the data enhancement operations predict the i-th data based on multiple operation levels of the any one of the data enhancement strategies and multiple evaluation values of the respective data enhancement strategies
  • the operation level of any data enhancement operation described in the enhancement strategy, and the evaluation value of any data enhancement strategy is used to indicate the pros and cons of the recognition model trained based on the enhanced samples obtained from the any data enhancement strategy;
  • any one of the data enhancement operations predict the i-th data based on multiple operation probabilities of the any one of the data enhancement strategies and multiple evaluation values of the respective data enhancement strategies the operation probability of any of the data augmentation operations described in the augmentation strategy;
  • the i-th data enhancement strategy is generated based on the predicted operation levels and operation probabilities of the various data enhancement operations.
  • the first node predicts any one of the new data enhancement strategies based on the operation level, the operation probability, and the evaluation value of any one of the data enhancement operations in the respective data strategies The operation level and operation probability of the data enhancement operation, so that the subsequently predicted data enhancement strategy can have a higher evaluation value.
  • predicting the i-th data enhancement strategy based on multiple operation levels of any one of the data enhancement strategies and multiple evaluation values of the respective data enhancement strategies includes:
  • the operation level that makes the acquisition function take the maximum value is determined as the operation level of any one of the data enhancement operations in the i-th data enhancement strategy .
  • predicting the i-th data enhancement strategy based on multiple operation probabilities of any one of the data enhancement strategies and multiple evaluation values of the respective data enhancement strategies includes:
  • the operation probability that makes the acquisition function take the maximum value is determined as the operation probability of any one of the data enhancement operations in the i-th data enhancement strategy .
  • the method further includes:
  • the multiple data enhancement operations are performed on the plurality of second samples to obtain the multiple data enhancement operations of the multiple second samples.
  • the training based on the plurality of enhanced samples to obtain the ith recognition model includes:
  • the pre-trained model is trained to obtain the i-th recognition model, the pre-trained model is obtained by training multiple third samples of multiple objects of different types, and the The precision is less than the precision threshold.
  • the object of the first sample and the object of the second sample belong to the same category, or the object of the first sample and the object of the second sample are the same object.
  • each data enhancement strategy in the iterative calculation process is evaluated by using samples of the same object or samples of the same type of object, so that the evaluation results of each data enhancement strategy are the same as the same object. or the same type of objects, therefore, the first target data enhancement strategy finally generated based on each data enhancement strategy corresponds to the same object or the same type of object, then the identification trained by this first target data enhancement strategy will be used subsequently. model, the recognition effect is optimal when recognizing the same object or the same type of object.
  • the i-th data enhancement strategy has multiple, and after the i-th data enhancement strategy is generated, the method further includes:
  • the evaluation values of the at least one i-th data enhancement operation strategy are respectively received from the plurality of evaluation devices.
  • the first target data enhancement strategy is further used to indicate the target operation probability of each data enhancement operation in the multiple data enhancement operations, and the target operation probability of any data enhancement operation is the probability of performing any one of the data augmentation operations on the first sample;
  • the performing the various data enhancement operations on the first sample within the operation intensity variation interval indicated by the target operation levels of the various data enhancement operations based on the first target data enhancement strategy includes:
  • the multiple data enhancement operations are performed on the first sample based on the target operation probability and target operation level of each data enhancement operation in the multiple data enhancement operations, wherein the value of any one of the data enhancement operations is
  • the target enhancement samples of the target operation probability have performed any one of the data enhancement operations.
  • the enhanced samples that have undergone various data enhancement operations account for a certain proportion of the multiple enhanced samples, so that the multiple enhanced samples are more in line with the real application scenario.
  • the performing the multiple data enhancement operations on the first sample based on the target operation probability and target operation level of each data enhancement operation in the multiple data enhancement operations includes: :
  • the first sample is subjected to any of the data augmentation operations.
  • the plurality of target enhancement samples are sent to the application node through the target interface.
  • a data enhancement apparatus comprising:
  • an acquisition module configured to acquire a first target data enhancement strategy, where the first target data enhancement strategy is used to indicate a target operation level of the data enhancement operation, and a target operation level is used to indicate an operation intensity variation interval;
  • the enhancement module is configured to, based on the first target data enhancement strategy, perform the various data enhancement operations on the first sample within the operation intensity variation interval indicated by the target operation levels of the various data enhancement operations, and obtain the result. multiple target enhancement samples of the first sample.
  • a change corresponding to any data enhancement operation occurs in the any data in the plurality of target enhancement samples.
  • a data enhancement operation is within the operation intensity variation interval indicated by the target intensity level of the data enhancement operation.
  • the first target data enhancement strategy is obtained by searching based on multiple operation levels of the configured multiple data enhancement operations.
  • Any data enhancement operation is configured with multiple operation levels, and one operation Levels are used to indicate a range of intensity variations.
  • the first target data enhancement strategy is determined based on multiple data enhancement strategies, and the multiple data enhancement strategies are obtained by searching through multiple operation levels of the configured multiple data enhancement operations, Any data enhancement operation is configured with a plurality of operation levels, and one operation level is used to indicate an intensity change interval.
  • the multiple data enhancement strategies are evaluated by multiple evaluation devices.
  • the objects of the second samples used in the evaluation of the multiple data augmentation strategies are the same object or objects of the same type.
  • the multiple operation levels configured by any one of the data enhancement operations are obtained based on the variation range of the target operation intensity of the any one of the data enhancement operations, and the target of the any one of the data enhancement operations
  • the operating intensity variation range is configured by the configuration device.
  • the apparatus further includes:
  • a division module configured to divide the target operation intensity variation range corresponding to any one of the multiple data enhancement operations into a plurality of operation intensity change intervals for any one of the multiple data enhancement operations
  • a configuration module configured to configure an operation level for each operation intensity change interval in the plurality of operation intensity change intervals.
  • the first target data enhancement strategy is further used to indicate the target operation probability of each data enhancement operation among the multiple data enhancement operations, and the target operation of any one of the data enhancement operations
  • the probability is the probability of performing any one of the data enhancement operations on the first sample
  • the acquisition module is used for:
  • the first target data enhancement strategy is determined based on a plurality of data enhancement strategies, and any data enhancement strategy is used to indicate the operation probability and operation level of each data enhancement operation among the multiple data enhancement operations, and any data enhancement strategy
  • the operation probability of the operation is the probability of performing any one of the data augmentation operations on the second sample.
  • the obtaining module includes:
  • the values are all higher than the evaluation values of the data enhancement strategies other than the plurality of second target data enhancement strategies in the plurality of data enhancement strategies, and the evaluation value of any one of the data enhancement strategies is used to indicate that the The pros and cons of the recognition model trained by the augmented samples obtained by the data augmentation strategy;
  • a generating submodule is configured to generate the first target data enhancement strategy based on the plurality of second target data enhancement strategies.
  • the generating submodule includes:
  • a determining unit configured to determine target operation probabilities and target operation levels of the multiple data enhancement operations based on the operation probabilities and operation levels of the multiple data enhancement operations in the multiple second target data enhancement strategies;
  • a generating unit configured to generate the first target data enhancement strategy based on the determined target operation probabilities and target operation levels of the multiple data enhancement operations.
  • the determining unit includes:
  • a clustering subunit configured to perform clustering on a plurality of operation levels of any one of the data enhancement operations in the plurality of second target data enhancement strategies for any one of the data enhancement operations, to obtain at least one operation class category;
  • a determining subunit configured to, for any operation level category in the at least one operation level category, based on the operation level in the any operation level category and the third target in the plurality of second target data enhancement strategies
  • the operation probability of any one of the data enhancement operations in the data enhancement strategy, determining the target operation intensity and target operation probability of the any one of the data enhancement operations, and the third target data enhancement strategy is the any operation level category
  • the second target data augmentation strategy to which the operational level of .
  • the determining subunit includes:
  • a first determining element configured to determine a target operation level of any one of the data enhancement operations based on the operation level in the any operation level category;
  • the second determining element is configured to determine the target of any one of the data enhancement operations based on the operation probability of the any one of the data enhancement operations in the third target data enhancement strategy in the plurality of second target data enhancement strategies Action probability.
  • the first determining element is used for:
  • Each operation level greater than or equal to the minimum operation level and less than or equal to the maximum operation level is determined as the target operation level of any one of the data enhancement operations.
  • the first determining element is used for:
  • Each operation level in the any operation level category is determined as the target operation level of the any one data enhancement operation.
  • the target operation probability of any one of the data enhancement operations is the probability of any one of the data enhancement operations in the third target data enhancement strategy in the plurality of second target data enhancement strategies Average probability of action probabilities.
  • the device also includes:
  • the iterative module is configured to perform iterative calculation based on the initial data enhancement strategy to obtain the multiple data enhancement strategies.
  • the iteration module is used to:
  • the plurality of data enhancement strategies are determined based on the data enhancement strategy determined in the previous i-1 iteration calculation process and each data enhancement strategy in the initial data enhancement strategy.
  • the i-th data enhancement strategy in wherein the i is an integer greater than or equal to 1 or less than or equal to N, and the i is the total number of iterative calculations.
  • the iteration module includes:
  • the first prediction submodule is used for any one of the data enhancement operations, based on multiple operation levels of the any one of the data enhancement strategies in the respective data enhancement strategies and multiple evaluations of the respective data enhancement strategies value, predicts the operation level of any data enhancement operation described in the i-th data enhancement strategy, and the evaluation value of any data enhancement strategy is used to indicate that the enhanced sample obtained based on the any data enhancement strategy is trained. Identify the pros and cons of the model;
  • the second prediction sub-module is configured to, for any one of the data enhancement operations, based on multiple operation probabilities of the any one of the data enhancement strategies in the respective data enhancement strategies and multiple evaluations of the respective data enhancement strategies value, predicting the operation probability of any data enhancement operation described in the i-th data enhancement strategy;
  • a generation sub-module is configured to generate the i-th data enhancement strategy based on the predicted operation levels and operation probabilities of the multiple data enhancement operations.
  • the first prediction submodule is used for:
  • the operation level that makes the acquisition function take the maximum value is determined as the operation level of any one of the data enhancement operations in the i-th data enhancement strategy .
  • the second prediction submodule is used for:
  • the operation probability that makes the acquisition function take the maximum value is determined as the operation probability of any one of the data enhancement operations in the i-th data enhancement strategy .
  • the device also includes:
  • an obtaining module configured to perform the multiple data enhancement operations on a plurality of the second samples based on the operation probability and the operation level of the multiple data enhancement operations in the i-th data enhancement strategy, to obtain the multiple data enhancement operations a plurality of augmented samples of the second sample;
  • a training module for obtaining the ith recognition model based on the multiple enhanced sample training
  • a target acquisition module configured to acquire the evaluation value of the i-th data enhancement strategy based on the i-th identification model.
  • the training module is used to:
  • the pre-trained model is trained to obtain the i-th recognition model, the pre-trained model is obtained by training multiple third samples of multiple objects of different types, and the The precision is less than the precision threshold.
  • the object of the first sample and the object of the second sample belong to the same category, or the object of the first sample and the object of the second sample are the same object.
  • the apparatus further includes:
  • a first sending module configured to send at least one i-th data enhancement operation strategy to a plurality of evaluation devices respectively;
  • the first receiving module is configured to respectively receive the evaluation value of the at least one i-th data enhancement operation strategy from the plurality of evaluation devices.
  • the first target data enhancement strategy is further used to indicate the target operation probability of each data enhancement operation in the multiple data enhancement operations, and the target operation probability of any data enhancement operation is the probability of performing any one of the data augmentation operations on the first sample;
  • the enhancement module is used to:
  • the multiple data enhancement operations are performed on the first sample based on the target operation probability and target operation level of each data enhancement operation in the multiple data enhancement operations, wherein the value of any one of the data enhancement operations is
  • the target enhancement samples of the target operation probability have performed any one of the data enhancement operations.
  • the enhancement module is used to:
  • the first sample is subjected to any of the data augmentation operations.
  • the apparatus further includes:
  • a second receiving module configured to obtain the first sample from the application node through the target interface
  • a second sending module configured to send the multiple target enhanced samples to the application node through the target interface.
  • a computing device in a third aspect, includes a processor and a memory, the memory stores at least one piece of program code, the program code is loaded and executed by the processor to realize the data in the above-mentioned first aspect The action performed by the enhancement method.
  • a computer-readable storage medium is provided, and at least one piece of program code is stored in the storage medium, and the program code is loaded and executed by a processor to implement the operations performed by the data enhancement method in the above-mentioned first aspect .
  • a computer program product or computer program includes program code
  • the program code is stored in a computer-readable storage medium
  • the processor of the computing device reads from the computer-readable storage medium Taking the program code, the processor executes the program code, so that the computer device executes the method provided in the first aspect or various optional implementation manners of the first aspect.
  • FIG. 1 is a schematic diagram of a data enhancement system provided by an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of a first node provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a data enhancement strategy provided by an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of another data enhancement system provided by an embodiment of the present application.
  • FIG. 5 is a working principle diagram of a second node provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a data enhancement system in a cloud scenario provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a computing device provided by an embodiment of the present application.
  • FIG. 9 is a flowchart of a data enhancement method provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a causal relationship between a sample label, an object, and a shooting method provided by an embodiment of the present application;
  • 11 is a graph showing the accuracy change of the recognition model in a training process provided by an embodiment of the present application.
  • FIG. 13 is a flowchart of a data enhancement method provided by an embodiment of the present application.
  • FIG. 14 is a schematic diagram of a data enhancement system provided by an embodiment of the present application.
  • 15 is a schematic diagram of a data change distribution provided by an embodiment of the present application.
  • 16 is a schematic diagram of distribution of specific pixel value changes when a sharpening operation changes according to an embodiment of the present application
  • FIG. 17 is a schematic structural diagram of a data enhancement apparatus provided by an embodiment of the present application.
  • FIG. 1 is a schematic diagram of a data enhancement system provided by an embodiment of the present application.
  • the data enhancement system 100 includes a first node 101 , a second node 102 and a third node 103 .
  • the first node 101 predicts a data enhancement strategy that can achieve better evaluation results based on the evaluation result of at least one data enhancement strategy, and feeds back the predicted data enhancement strategy to the second node 102, by
  • the second node 102 evaluates the predicted data enhancement strategy, and feeds back the evaluation result of the predicted data enhancement strategy to the first node 101, wherein the at least one data enhancement strategy is historically predicted by the first node Data Augmentation Strategies.
  • the data enhancement strategy predicted last time and has an evaluation result is a historically predicted data enhancement strategy, which means that each data enhancement strategy predicted by the first node 101 has been updated. . Then, based on the historically predicted evaluation results of each data enhancement strategy, the first node 101 predicts again a data enhancement strategy that can achieve a better evaluation result. By analogy, until each data enhancement strategy predicted by the first node 101 in the history meets the preset conditions, the first node 101 sends each data enhancement strategy predicted in the history to the third node 103, and the third node 103 sends the data enhancement strategy. At least one first target data enhancement strategy is generated based on the received respective data enhancement strategies.
  • the third node 103 provides the at least one first target data enhancement strategy to an external application node, and the application node adopts the any first target data enhancement strategy to perform data enhancement on multiple samples to obtain a large number of Enhanced samples.
  • the application node In order for the application node to use a large number of enhanced samples for model training in the future, in order to obtain a recognition model with higher accuracy.
  • each node is described below in combination with the physical architecture of each node.
  • the first node 101 includes an operation pool 1011 , a policy change space 1012 , and a policy optimizer 1013 .
  • the operation pool 1011 is used to provide the user with the configuration of data enhancement operations.
  • the user configures multiple data enhancement operations in the operation pool 1011 based on the requirements of the application scenario and the target operation intensity variation range of each data enhancement operation in the multiple data enhancement operations.
  • the multiple data enhancement operations are data enhancement operations required by the samples in the application scenario
  • the variation range of the target operation intensity of a data enhancement operation is: the intensity of the data enhancement operation performed on the samples in the application scenario variation range.
  • the first node is the target operation intensity variation range of the any data enhancement operation configured in the operation pool 1011 .
  • Any data enhancement operation is assigned multiple operation levels, and a probability change interval is assigned to any data enhancement operation, so that any data enhancement operation corresponds to multiple operation levels and a probability change interval.
  • the first node can also store the operation level and probability change interval corresponding to each data enhancement operation in the multiple data enhancement operations in the policy change space 1012 .
  • an operation level of any data enhancement operation is used to indicate an operation intensity change interval
  • the operation intensity change interval is a sub-interval of the target operation intensity change range of the data enhancement operation.
  • the strategy optimizer 1013 randomly selects an operation level and an operation probability in the probability change interval of the data enhancement operation from the strategy change space 1012 to achieve random search and A data augmentation strategy.
  • the data enhancement strategy includes the operation probability and the operation level of each data enhancement operation in the multiple data enhancement operations in the strategy change space 1012 .
  • FIG. 3 is a schematic diagram of a data enhancement strategy provided by an embodiment of the present application.
  • the data enhancement strategy shown in FIG. 3 includes S types of data enhancement operations, which are respectively data enhancement operations 1-S.
  • the data enhancement strategy also includes data enhancement operations.
  • the operation probabilities of the enhancement operations 1-S are respectively operation probabilities 1-S
  • the data enhancement strategy also includes the operation levels of the data enhancement operations 1-S, which are respectively the operation levels 1-S, where S is an integer greater than or equal to 1. It should be noted that any data enhancement operation in the data enhancement strategy shown in FIG. 3 has an operation level and an operation probability, while in other embodiments, any data enhancement operation in a data enhancement operation strategy An operation level of an operation can also have multiple operation levels.
  • the strategy optimizer 1013 After the strategy optimizer 1013 searches for the first data enhancement strategy, it sends the first data enhancement strategy to the second node 102, and the second node 102 feeds back the evaluation result of the first data enhancement strategy.
  • the policy optimizer 1013 searches in the policy change space 1012 based on the evaluation result of the first data augmentation policy to obtain the second data augmentation policy. Since the second data augmentation strategy is searched based on the evaluation result of the first data augmentation strategy, the second data augmentation strategy is unknown for the first data augmentation operation strategy.
  • the two data augmentation strategies are the data augmentation strategies predicted by the strategy optimizer 1013 to achieve better evaluation results.
  • the strategy optimizer 1013 sends the second data enhancement strategy to the second node 102, and the second node 102 feeds back the evaluation result of the second data enhancement strategy.
  • the strategy optimizer 1013 continues searching in the strategy change space 1012 based on the evaluation results of the first data enhancement strategy and the second data enhancement strategy to obtain a third data enhancement strategy.
  • the strategy optimizer 1013 sends each data enhancement strategy to the third node 103, and the third node 103 receives each data enhancement strategy based on the received data.
  • the enhancement strategy generates at least one first target data enhancement strategy.
  • the policy optimizer 1013 can also fuse the sample labels of the samples to predict a data augmentation policy.
  • the data augmentation strategy predicted by the strategy optimizer 1013 in the iterative process is oriented to samples with the same sample label, or oriented to samples belonging to the same type of sample label.
  • the sample label of an object is used to identify the object and the type of the object. If the objects in the samples in the training set belong to different types, the first node 101 can finally generate different types of target data enhancement strategies, wherein, Each type of target data augmentation strategy corresponds to a type of object. For example, in the iterative process of generating any type of target data augmentation strategy by the first node, the strategy optimizer 1013 sends the predicted data augmentation strategy and the either type to the second node 102, so that the second node 102 can base on the Samples with either type of sample label are evaluated on the received data augmentation strategy.
  • the strategy optimizer 1013 sends each data enhancement strategy of the type to the third node 103, and each third node 103 generates the data enhancement strategy based on the type of data enhancement strategy.
  • the application node can train the recognition of each object suitable for any type based on the enhanced samples obtained by the optimal data enhancement strategy of any type. Model.
  • the data enhancement system 100 shown in FIG. 1 is described by taking the first node 101 and the third node 103 as two independent nodes as an example, and in other embodiments, the first node 101 is the same node as the third node 103, the third node 103 may be used as a unit in the first node 101, and exists in the first node 101, for example, the first node 101 shown in FIG. 2 may also include a policy Generator 1014 , the policy generator 1014 is also the third node 103 .
  • the number of the second nodes 102 in the data enhancement system 100 may be one or more, and the second nodes 102 may be regarded as evaluation equipment.
  • the first node 101 can predict multiple data enhancement strategies in each iterative process (ie, the search process), if the second node 102 in the data enhancement system 100 has multiple data enhancement strategies In this case, the first node 101 may send the multiple data enhancement strategies predicted at each time to multiple second nodes 102 in a distributed manner.
  • Each second node 102 evaluates a part of the data enhancement strategies in the plurality of data enhancement strategies, and each second node 102 sends the evaluation results of the data enhancement strategies that it is responsible for evaluating to the first node 101 .
  • the first node 101 can obtain the evaluation results of multiple data enhancement strategies predicted in each iteration process, thereby realizing distributed evaluation.
  • the evaluation is performed in parallel by a plurality of second nodes 102 , thus improving the evaluation efficiency of the data enhancement system 100 .
  • FIG. 4 a schematic structural diagram of another data enhancement system provided by an embodiment of the present application, the first node sends multiple data enhancement strategies in each iteration process to multiple second nodes, and the multiple second nodes The node evaluates the multiple data augmentation strategies, and returns the evaluation results of each data augmentation strategy.
  • the first node After that, the first node generates a first target data enhancement strategy based on each historical data enhancement strategy, and sends the first target data enhancement strategy to the application node.
  • the application node adopts the first target data enhancement strategy to perform data enhancement on the samples obtained from the data server, and performs model training based on the enhanced samples obtained after the enhancement, and deploys the trained recognition model to the service.
  • the second node 102 adopts the any data enhancement strategy to perform a data enhancement operation on a plurality of samples to obtain a plurality of enhanced samples of the plurality of samples . Afterwards, the second node 102 uses the plurality of enhanced samples to train the initial model to obtain a recognition model. The multiple samples may be obtained by the second node 102 from the data server. Further, the second node 102 evaluates the recognition effect of the recognition model, and feeds back the evaluation result of the recognition model to the first node 101 as the evaluation result of any data enhancement strategy.
  • the second node 102 may first acquire a pre-training model, and the pre-training model may be a recognition model that is about to be trained. For example, the accuracy of the pre-training model is less than the accuracy threshold.
  • the second node 102 uses the multiple enhanced samples to train the pre-training model to obtain a recognition model. For example, as shown in FIG. 5, a working principle diagram of a second node provided by an embodiment of the present application is shown.
  • the second node uses the enhanced samples obtained by the data enhancement strategy to re-train the pre-training model.
  • a recognition model is obtained, and the recognition effect of the recognition model is evaluated to obtain the evaluation result of the data enhancement strategy.
  • the third node 103 selects multiple optimal data enhancement strategies (for example, multiple data enhancement strategies) from the multiple data enhancement strategies based on the received evaluation results of the multiple data enhancement strategies A data enhancement strategy with a higher evaluation result among the strategies), and based on the multiple optimal data enhancement strategies, a first target data enhancement strategy is generated.
  • multiple optimal data enhancement strategies for example, multiple data enhancement strategies
  • the third node 103 When the multiple data enhancement strategies are evaluated based on multiple samples of one object, or based on multiple samples of multiple objects under one type, then the third node 103 finally based on the multiple data enhancement strategies
  • the generated first target data enhancement strategy corresponds to the object or the type, and the application node trains a recognition model based on the enhanced samples obtained by the first target data enhancement strategy, and the recognition effect of the object or the object under this type is relatively high. good.
  • the data enhancement system 100 can be deployed on multiple computing devices, and the multiple computing devices are respectively used to implement the functions of the first node 101 , the second node 102 and the third node 103 .
  • the first node 101 , the second node 102 and the third node 103 are mutually independent computing devices.
  • some or all of the first node 101 , the second node 102 , and the third node 103 are integrated into a computing device, and then each node in the computing device is also a part of the computing device.
  • a unit, for example, the first node 101 also has the functions of the second node 102 and the third node 103 .
  • This embodiment of the present application does not limit the deployment manner of each node in the data enhancement system 100 .
  • the first node 101, the second node 102 and the third node 103 may also be cloud computing devices.
  • the data enhancement system 100 is deployed in the cloud.
  • FIG. 6 a schematic diagram of a data enhancement system in a cloud scenario provided by an embodiment of the present application, an application node uploads a sample to be enhanced (eg, customer data) to a first node deployed in the cloud through a target interface.
  • a sample to be enhanced eg, customer data
  • an uploading interface is displayed in the application node, and the user uploads the sample to be enhanced in the uploading interface.
  • the application node When the application node detects that the user has performed a confirmation operation in the uploading interface, the application node passes the target interface to the first A node sends samples to be enhanced for uploading. After the upload of the sample to be enhanced is completed, the first node randomly searches for a plurality of data enhancement strategies in the strategy change space, and sends at least one data enhancement strategy and the to-be-enhanced data enhancement strategy to a plurality of second nodes respectively. sample. After any second node receives the at least one data enhancement strategy and the sample to be enhanced, for any data enhancement strategy in the at least one data enhancement strategy, the any second node, based on the sample to be enhanced, The any data enhancement strategy is evaluated, and the evaluation result of the any data enhancement strategy is fed back to the first node.
  • the first node generates a plurality of new data enhancement strategies based on the evaluation results of the plurality of data enhancement strategies, and the plurality of new data enhancement strategies are again evaluated by the plurality of nodes.
  • the first node when there are enough historically generated data enhancement strategies, the first node generates a target data enhancement strategy based on each historically generated data enhancement strategy, and the first node adopts the target data enhancement strategy, and the sample to be enhanced Multiple data enhancement operations are performed to obtain multiple enhanced samples of the sample to be enhanced.
  • the application node downloads the plurality of enhanced samples through the target interface, so that the application node subsequently performs model training based on the downloaded plurality of enhanced samples, obtains a recognition model, and deploys the recognition model to a service.
  • the target interface is a software interface or a hardware interface for data interaction between the data enhancement system in the cloud and the application node.
  • the application node can also directly download the target data enhancement strategy through the target interface, and parse the target data enhancement strategy, and based on the parsed target data enhancement strategy, perform a variety of Data augmentation operation to obtain multiple augmented samples.
  • the first node includes a central processing unit (CPU), and the CPU implements each step performed by the first node, and the first node includes a graphics processing unit (GPU) , each step performed by the second node is implemented by the CPU.
  • CPU central processing unit
  • GPU graphics processing unit
  • the first node or the third node in the data enhancement system can also generate multiple target data enhancement strategies, and any target data enhancement strategy is based on the first node.
  • the data enhancement strategy finally generated by each data enhancement strategy generated in history, any target data enhancement strategy corresponds to an object type or an object; the node that generates the multiple target data enhancement strategies combines each target data enhancement strategy and each target data enhancement strategy.
  • the object type or object identifier corresponding to the target data enhancement strategy is uploaded to the cloud, so that the application node can download the corresponding target data enhancement strategy from the cloud according to the sample to be enhanced.
  • a recognition model for the sample to be enhanced or the object is uploaded to the cloud, so that the application node can download the corresponding target data enhancement strategy from the cloud according to the sample to be enhanced.
  • the first node may also train a recognition model based on each target data enhancement strategy, associate each recognition model with the object type or object identifier corresponding to each target data enhancement strategy, and upload it to the cloud,
  • the application node directly downloads the required recognition model from the cloud according to the needs of the application scenario.
  • the first node can also train a recognition model based on each target data enhancement strategy, generate multiple enhanced samples, and upload the multiple enhanced samples and the sample label of each enhanced sample to the cloud, and the application node According to the needs of the application scenario, download each required enhancement sample directly from the cloud.
  • the first node 101, the second node 102, and the third node 103 in this application may be servers, and the application nodes may be user equipment such as a terminal, a personal computer (PC), or an application server.
  • the implementation forms of the first node 101, the second node 102, the third node 103 and the application node are not limited.
  • the computing device 700 may be configured as any one of a first node, a second node, and a third node.
  • the computing device 700 may be configured as a There is a large difference due to different performance, which may include one or more processors 701 and one or more memories 702, wherein, the memory 702 stores at least one instruction, and the at least one instruction is loaded by the processor 701 and loaded. Execute to implement the methods provided by each method embodiment described below. For example, if the computing device 700 is configured as the first node, the at least one instruction is loaded and executed by the processor 701 to implement the steps performed by the first node in the following method embodiments.
  • the computing device 700 may also have components such as a wired or wireless network interface, a keyboard, and an input/output interface for input and output, and the computing device 700 may also include other components for implementing device functions, which will not be repeated here.
  • the processor 701 can be a CPU, a GPU, a tensor processing unit (TPU), a neural network processing unit (NPU), a brain processor (brain processing unit, BPU), a deep learning processor (deep learning processing unit, DPU), holographic image processor (holographic processing unit, HPU), vector processor (vector processing unit, VPU) and intelligent processor (intelligence processing unit, IPU) Any processor.
  • the processor 701 may use a general-purpose CPU, a microprocessor, an application specific integrated circuit (ASIC), a GPU, or one or more integrated circuits for executing relevant programs to implement the following data enhancement method.
  • ASIC application specific integrated circuit
  • GPU graphics processing unit
  • the processor 701 can also be an integrated circuit chip, which has signal processing capability. In the implementation process, the steps performed by any node in the data enhancement method of the present application may be completed by an integrated logic circuit of hardware in the processor 701 or instructions in the form of software.
  • the above-mentioned processor 701 can also be a general-purpose processor, a digital signal processor (digital signal processing, DSP), an ASI, an off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistors Logic devices, discrete hardware components.
  • DSP digital signal processing
  • ASI an off-the-shelf programmable gate array
  • FPGA field programmable gate array
  • the methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor.
  • Software modules can be located in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory or electrically erasable programmable memory, registers and other mature storage media in the field middle.
  • the storage medium is located in the memory 702, and the processor 701 reads the information in the memory 702 and, in combination with its hardware, completes the functions required to be performed by the modules included in the data enhancement apparatus of the embodiments of the present application, or performs the data enhancement of the method embodiments of the present application. method.
  • a computer-readable storage medium such as a memory including instructions, is also provided, and the instructions can be executed by a processor in the terminal to complete the data enhancement method in the following embodiments.
  • the computer-readable storage medium may be ROM, RAM, compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, optical data storage device, and the like.
  • Taking the first node to perform an initialization configuration for an application scenario before acquiring the optimal data enhancement strategy applied to the application scenario refer to the process of an initialization configuration provided by the embodiment of the present application shown in FIG. 8 . picture.
  • the first node configures multiple data enhancement operations and target operation intensity variation ranges of the multiple data enhancement operations.
  • the multiple data enhancement operations are data enhancement operations required for samples in a certain application scenario, and the application scenario is any one of an image recognition scenario, an audio video scenario, and a text recognition scenario.
  • the scene is not limited.
  • the sample consists of data collected in the application scenario, or it is considered that the sample consists of real data.
  • the application scenario is an image recognition scenario as an example for description.
  • the sample may be in the image Identify images captured by any device capable of capturing the scene.
  • data enhancement operations can be used to enhance the existing samples to obtain enhanced samples, and the enhanced samples are regarded as the real data collected in the application scenario for model training.
  • the various data enhancement operations include at least one of a flip operation, a rotation operation, a zoom operation, a crop operation, a translation operation, an operation of adding noise, and an operation of sharpening.
  • the multiple data enhancement operations may be adaptively configured according to specific application scenarios, and the embodiments of the present application do not limit the multiple data enhancement operations.
  • the change of the real data in this application scenario is a distribution, rather than a few specific change points.
  • the inclination and size of the cat in the captured images will have infinite changes, not just a specific number of degrees.
  • the images taken by different mobile phones will be different.
  • the first node is also configured with target operation intensity variation ranges of the various data enhancement operations, so that the samples are subsequently processed within the target operation intensity variation ranges of each data enhancement operation.
  • Each data operation is performed, so that each variation range of the finally obtained enhanced sample is within the target operation intensity variation range of each data enhancement operation, so as to simulate the variation distribution of real data in this application scenario.
  • the target operation intensity variation range of a data enhancement operation includes multiple operation intensities
  • the embodiment of the present application does not limit the operation intensity range of the target operation intensity variation range of each data enhancement operation, and can be adapted according to actual application scenarios. configuration.
  • a data enhancement operation is a rotation operation, and the operation intensity of the rotation operation can be represented by a rotation angle.
  • the first node can configure the target operation intensity change range for the rotation operation to be [0 degrees, 180 degrees] degrees], so that the image in the image recognition scene can be rotated in the range of [0 degrees, 180 degrees] to obtain multiple data-enhanced images (that is, multiple enhanced samples).
  • the objects in the multiple data-enhanced images are rotated within the range of [0 degrees, 180 degrees], so that the multiple data-enhanced images conform to the changing law of the real images in the image recognition scene, then the multiple data-enhanced images can be changed.
  • a data augmentation data is used as a real sample in the image recognition scene for model training.
  • the first node configures various data enhancement operations and target operation intensity variation ranges of the various data enhancement operations based on manual operations.
  • the first node displays a data enhancement operation configuration interface, and the user configures multiple data enhancement operations and a target operation intensity variation range of each data enhancement operation in the data enhancement operation configuration interface.
  • the first node stores a variety of data enhancement operations configured by the user in the data enhancement operation configuration interface and changes in the target operation intensity of each data enhancement operation Scope, the configuration is complete.
  • the process shown in this step 801 is performed by an operation pool in the first node, for example, the operation pool 1011 in the first node 101 shown in FIG. 2 .
  • this step 801 is performed by the first node, and the first node may be regarded as a configuration device. In some embodiments, this step 801 is not performed by the first node, but is performed by any configuration device other than the first node.
  • the first node divides the operation intensity of the any one of the data enhancement operations into a number of strength level.
  • An intensity level is used to indicate an intensity variation interval.
  • the intensity variation interval may be any subset of the target operation intensity variation range.
  • the first node divides the target operation intensity change range corresponding to any one of the data enhancement operations into a plurality of intensity change intervals, and there is no intersection between the plurality of intensity change intervals, and the first node is the plurality of operation intensity changes
  • Each operation intensity change interval in the interval configures the operation level.
  • the operation intensity in the plurality of intensity change intervals is positively correlated with the intensity level of the operation intensity change interval. That is, the operation intensity change interval with higher operation intensity among the plurality of intensity change intervals has a higher intensity level, and vice versa, the intensity level is lower.
  • any data enhancement operation is a rotation operation
  • the target operation intensity variation range of the rotation operation is [0 degrees, 180 degrees]
  • the first node divides the target operation intensity variation range [0 degrees, 180 degrees] into 10 intensity change intervals, in order [0 degrees, 18 degrees), [18 degrees, 36 degrees), ..., [162 degrees, 180 degrees], according to the level of operation intensity in the 10 intensity change intervals, the first node
  • the intensity levels corresponding to the 10 intensity change intervals are sequentially arranged as intensity levels 1, 2, . . . , 10, and the 10 intensity levels are sequentially increased.
  • the process shown in this step 802 is completed by the first node in the policy change space, for example, the policy change space 1012 in the first node 101 shown in FIG. 2 .
  • this step 802 is performed by the first node, and the first node may be regarded as a configuration device. In some embodiments, this step 802 is not performed by the first node, but is performed by any configuration device other than the first node.
  • the first node configures probability change intervals for the multiple data enhancement operations.
  • a probability change interval includes a plurality of operation probabilities, and an operation probability of any data enhancement operation is the probability of performing any one of the data enhancement operations on the sample.
  • the first node configures the same probability variation interval for each of the multiple data enhancement operations, or configures different probability variation intervals for the multiple data enhancement operations.
  • the probability variation interval configured by the first node for the various data enhancement operations is not limited.
  • the first node directly configures the multiple data augmentation operations with the same probability variation interval, eg [0, 1].
  • the process shown in this step 803 is completed by the first node in the policy change space, for example, the policy change space 1012 in the first node 101 shown in FIG. 2 .
  • step 803 there is no sequence in the execution of step 803 and step 802.
  • the first node may execute step 803 first and then execute the above step 802.
  • the embodiment of this application does not limit the execution sequence of steps 802-803.
  • the initialization configuration is completed.
  • multiple data augmentation strategies can be formed by combining different data augmentation operations to achieve More diverse data changes, and the first node can also obtain the first target data enhancement strategy based on multiple data enhancement strategies, so that the first target data enhancement strategy can be subsequently used to expand the sample.
  • FIG. 9 refers to the flowchart of a data enhancement method provided by an embodiment of the present application shown in FIG. 9 .
  • the first node acquires an initial data enhancement strategy, where the initial data enhancement strategy is used to indicate operation probabilities and intensity levels of various data enhancement operations.
  • each data enhancement in the multiple data enhancement operations may be identified by an operation identifier, and the operation identifiers of different data enhancement operations are different.
  • the initial data enhancement strategy includes operation identifiers, operation probabilities, and intensity levels of the multiple data enhancement operations, operation identifiers of each data enhancement operation, and operation intensity and operation intensity of each data enhancement operation. Probability corresponds.
  • the sum of the operation probabilities of the multiple data enhancement operations in the initial data enhancement strategy is 1, and each data enhancement operation in the initial data enhancement strategy has at least one intensity level, and each data enhancement operation has at least one intensity level.
  • the enhanced probability of operation is 1. It should be noted that, the embodiments of the present application are described by taking as an example that each data enhancement operation in the initial data enhancement strategy has one intensity level.
  • the initial data augmentation strategy obtained by the first node may be one or multiple.
  • the embodiments of the present application do not limit the number of initial data augmentation policies acquired by the first node. It should be noted that the embodiments of the present application take an example of acquiring an initial data augmentation policy by the first node for description.
  • the process for the first node to acquire an initial data enhancement policy is: for any data enhancement operation among the configured multiple data enhancement operations, the first node obtains an initial data enhancement strategy from the configured data enhancement operation. Randomly selects an intensity level from a plurality of intensity levels of various data enhancement operations, and randomly selects an operation probability from the configured probability variation interval of any one of the data enhancement operations; the first node is based on the multiple data enhancement operations.
  • the process shown in this step 901 is performed by the policy optimizer in the first node, for example, the policy optimizer 1013 in the first node 101 shown in FIG. 2 .
  • the first node Since there are many types of data enhancement operations, and different combinations of different types of data enhancement operations are actually another form of data enhancement operations, the first node needs to find a better data enhancement strategy from the acquired multiple data enhancement strategies. For the data enhancement strategy, it is necessary to evaluate each acquired data enhancement strategy, so as to obtain a better data enhancement strategy based on the obtained evaluation results of the multiple data enhancement strategies.
  • the initial data enhancement strategy is a data enhancement strategy obtained by the first node. In a possible implementation manner, the first node evaluates the initial data enhancement strategy through the following step 902 .
  • the first node evaluates the initial data enhancement strategy, and obtains an evaluation value of the initial data enhancement strategy, where the evaluation value of the initial data enhancement strategy is used to indicate the training of the enhanced samples obtained based on the initial data enhancement strategy. Identify how good or bad the model is.
  • the evaluation value of the initial data enhancement strategy is also the evaluation result of the initial data enhancement strategy by the first node.
  • the evaluation value of the initial data enhancement strategy is the accuracy of the recognition model trained on the enhanced samples obtained based on the initial data enhancement strategy.
  • the first node adopts the initial data enhancement strategy to perform data enhancement on the samples in the application scenario, and performs model training based on the enhanced samples after data enhancement, and evaluates the quality of the initial data enhancement strategy through the trained recognition model .
  • the process shown in this step 902 is implemented by the processes shown in the following steps 9021-9023.
  • Step 9021 Based on the operation probability and operation level of the multiple data enhancement operations in the initial data enhancement strategy, the first node performs the multiple data enhancement operations on a plurality of second samples, and obtains the multiple data enhancement operations of the multiple second samples. an enhanced sample.
  • the enhanced sample of the operation probability of the any one of the data enhancement operations in the multiple enhanced samples has performed the any one of the data enhancement operations .
  • Each second sample in the plurality of second samples is composed of data collected in the application scenario, and for any second sample in the plurality of second samples, the any second sample has a sample label,
  • the sample label of the any second sample is used to indicate the object of the any second sample.
  • the sample label of the any second sample includes an object identifier, and the object identifier is used to identify the object of the any second sample.
  • the object of the any second sample is the object collected when the any second sample is collected. For example, if the any second sample is an image of a cat, the cat is also the object of the any second sample.
  • the objects of the plurality of second samples are different, or the objects of some of the second samples of the plurality of second samples are different.
  • samples involved in this application are all composed of data collected in this application scenario, and are real data in this application scenario.
  • the enhanced samples involved in this application are all obtained by performing data enhancement operations on the real data in the application scenario, and are used to imitate the real data in the application scenario to expand the sample size in the application scenario.
  • the samples or enhanced samples involved in this application are ultimately used for model training, and the data changes of the samples used for model training depend on how these samples are generated.
  • the images of the same cat shot with different mobile phones will have different image attributes.
  • the color temperature of the images shot by some mobile phones is warmer, and some are cooler.
  • the image of a cat and the image of the bird will also have obvious differences in image attributes.
  • the image of the bird is relatively blurred, and the camera may automatically add more sharpening operations.
  • the cat image does not require automatic sharpening. For example, as shown in FIG.
  • FIG. 10 a schematic diagram of a causal relationship between sample labels, objects, and shooting methods provided by the embodiment of the present application, wherein the static flagpole and the dynamic hummingbird are used as objects, which determine the different shooting methods and the shooting methods.
  • the sample labels of the images are different, and the different shooting methods determine the change distribution of the captured images.
  • the captured images and the labels of the images form a training set for model training.
  • the data augmentation strategy is used to generate augmented samples with data distributed as similar as possible to the real data population, it is necessary to consider the impact of the sample generation method, and then calculate different data augmentation strategies for different sample generation methods.
  • the manner in which the sample is generated is generally related to the type of the object of the sample or the object of the sample itself. Therefore, in some embodiments, the objects of the plurality of second samples belong to the same type.
  • the objects of the plurality of second samples are any of birds, amphibians or reptiles.
  • the objects of the plurality of second samples are the same object, for example, the objects of the plurality of second samples are all cats or all birds.
  • the sample label of each second sample also includes a category identifier of the object of each second sample, if the objects of the plurality of second samples belong to the same category, the sample labels of the plurality of second samples belong to the same category. Included are the same species IDs. For the convenience of description, the category to which the objects of the multiple second samples belong is recorded as the target category; if the objects of the multiple second samples are the same object, the object identifiers included in the sample tags of the multiple second samples are the same.
  • the first node based on the operation probability of any one of the data augmentation operations, performs In the operation intensity variation interval indicated by the operation level of a data enhancement operation, any one of the data enhancement operations is performed on the plurality of second samples, thereby obtaining a plurality of enhanced samples that have undergone the any one of the data enhancement operations.
  • the first node is set with a required target number of enhanced samples of the any second sample, and the first node is based on the target number and the The operation probability of any kind of data enhancement operation, within the operation intensity variation interval indicated by the intensity level of the any kind of data enhancement operation, performing the any kind of data enhancement operation for any second sample for many times to obtain the A plurality of enhanced samples of any second sample that have undergone any one of the data enhancement operations.
  • the number of enhanced samples that have performed any one of the data enhancement operations is the difference between the target number and the operation probability of the any one of the data enhancement operations. product.
  • this step 9021 is described as follows: For the jth data enhancement operation in the S data enhancement operations and the rth second sample among the plurality of second samples, if the operation probability of the jth data enhancement operation is 0.2, and the set in the first node The required target number of enhancement samples of the rth second sample is 100, then within the intensity variation interval indicated by the intensity level of the jth data enhancement operation in the initial data enhancement strategy, the first node The j-th data enhancement operation is performed 20 times on the r-th second sample to obtain 20 enhanced samples of the r-th second sample.
  • the j is an integer greater than or equal to 1 or less than or equal to S
  • the r is an integer greater than or equal to 1 or less than or equal to M.
  • the first node performs any one of the data enhancement operations for the any second sample multiple times.
  • the process is as follows: the first node performs any one of the data enhancement operations multiple times on the second sample according to the operation intensity in the operation intensity variation interval indicated by the intensity level of the any one of the data enhancement operations.
  • the operation intensity change interval corresponding to the intensity level of the rotation operation in the initial data enhancement strategy is [0 degrees, 18 degrees), and it is necessary to perform the rotation operation for any second data enhancement strategy.
  • the sample performs 18 rotation operations, then the first node selects 18 rotation angles from the operation intensity variation interval [0, 18 degrees) with equal probability, which are 0, 1 degree, ..., 17 degrees, respectively.
  • the node performs a rotation operation on any second sample based on each rotation angle selected to obtain 18 enhanced samples of any second sample, so that the variation distribution of these 18 enhanced samples in the rotation angle is approximately at [0, 18 degrees).
  • each enhanced sample obtained by performing the data enhancement operation on any second sample is the same as the object of any second sample, and the first node can label the sample of any second sample.
  • the sample label of each enhanced sample obtained by performing the data enhancement operation on the any second sample is the same as the object of any second sample, and the first node can label the sample of any second sample.
  • Step 9022 The first node obtains an initial recognition model by training based on a plurality of enhanced samples of the plurality of second samples.
  • the initial recognition model is used to recognize the sample, and optionally, the initial recognition model is a neural network model, such as a deep neural network model.
  • the first node inputs the plurality of enhanced samples into an initial model, and performs model training on the initial model by using the plurality of enhanced samples to obtain the initial recognition model.
  • the initial model is a neural network model without any training.
  • the first node obtains a pre-training model first, and then trains the pre-training model based on the plurality of enhanced samples to obtain the initial recognition model.
  • the pre-training model is obtained by training multiple third samples of multiple objects of different types, the accuracy of the pre-training model is less than the accuracy threshold, and the accuracy threshold can be set according to the actual application scenario, the embodiment of the present application The accuracy threshold is not limited.
  • the plurality of third samples are composed of data collected in the application scenario. Since the accuracy of the pre-training model is less than the accuracy threshold, it is considered that the pre-training model is an untrained model. Then, the first node continues to train the pre-training model based on the plurality of enhanced samples, and the time to obtain the initial recognition model is much less than the time from training the initial model to the initial recognition model, thereby shortening the first recognition model.
  • the total time for a node to evaluate the initial data enhancement strategy improves the evaluation efficiency of the data enhancement operation. For example, as shown in FIG.
  • a graph showing the accuracy change of the recognition model in the training process provided by the embodiment of the present application, if the first node trains the initial model based on the multiple enhanced samples, if the accuracy is above 95% If the first node uses 115 minutes to train the pre-training model first, after acquiring the multiple enhanced samples, the pre-trained model will be trained based on the multiple enhanced samples.
  • the training time of the training model is 95 minutes to obtain an initial recognition model with an accuracy of more than 95%, which significantly shortens the time occupied by model training in the evaluation process and improves the evaluation efficiency of data augmentation operations.
  • the process of acquiring the pre-training model by the first node is as follows: the first node performs training on the initial model based on the plurality of third samples to obtain the pre-training model.
  • the pre-training model is trained by other devices other than the first node, and the first node obtains the pre-training model from other devices.
  • This embodiment of the present application does not limit the manner in which the first node obtains the pre-training model.
  • the first node only needs to acquire the pre-trained model once, and does not need to acquire it multiple times.
  • the pre-training model trained by multiple third samples of different types of objects is suitable for enhanced samples of different types of objects. Therefore, even if the objects of multiple enhanced samples belong to the same type or the same object, the first node By training the pre-trained model based on the plurality of enhanced samples, the initial recognition model can still be trained. Since the objects of the multiple enhanced samples belong to the same type or the same object, the trained initial recognition model has better recognition effect when recognizing the object of the target type or other samples of the objects of the multiple enhanced samples.
  • Step 9023 The first node obtains the evaluation value of the initial data enhancement strategy based on the initial identification model.
  • the first node uses a plurality of fourth samples to test the initial recognition model, and obtains the accuracy of the initial recognition model based on the test results, and the first node obtains the accuracy of the initial recognition model as the The evaluation value of the initial data augmentation strategy.
  • the plurality of fourth samples consist of data collected in the application scenario; if the types of objects in the plurality of second samples are different, the types of objects in the plurality of fourth samples are different; if the types of objects in the plurality of fourth samples are different; The objects of the two samples all belong to the target category, then the objects of the plurality of fourth samples also belong to the target category; if the objects of the plurality of second samples are the same object, the objects of the plurality of fourth samples are the same as those of the plurality of second samples. The objects of the samples are the same.
  • the initial recognition model is also a recognition model trained based on the enhanced samples obtained from the initial data enhancement strategy, and the accuracy of the initial recognition model is the accuracy rate at which the initial recognition model correctly identifies the objects of the plurality of fourth samples .
  • the first node inputs the plurality of fourth samples into the initial recognition model, the initial recognition model recognizes each of the input fourth samples, and outputs the recognition of each fourth sample result. If the object indicated by the recognition result of a fourth sample is the same as the object indicated by the sample label of the fourth sample, the fourth sample is correctly identified; otherwise, the fourth sample is incorrectly identified.
  • the first node counts the number of fourth samples correctly identified by the initial model, and divides the number of correctly identified fourth samples by the total number of the plurality of fourth samples to obtain the accuracy of the initial identification model, The first node uses the accuracy as an evaluation value for the initial data augmentation strategy.
  • the first node in addition to using the accuracy of the initial recognition model as the evaluation value of the initial data enhancement strategy, can also use other evaluation indicators of the initial recognition model as the initial data enhancement
  • the evaluation value of the strategy such as recall rate, F1 value, or area under the curve (AUC) value, here, the embodiment of the present application does not limit the other evaluation indicators.
  • the process shown in this step 902 is performed by the policy optimizer in the first node, for example, the policy optimizer 1013 in the first node 101 shown in FIG. 2 .
  • the first node performs iterative calculation based on the initial data enhancement strategy to obtain multiple data enhancement strategies.
  • any data enhancement strategy is used to indicate the operation probability and operation level of each data enhancement operation in the multiple data enhancement operations.
  • the first node determines the plurality of data enhancement strategies based on the data enhancement strategy determined in the previous i-1 iteration calculation process and each data enhancement strategy in the initial data enhancement strategy The ith data augmentation strategy in the strategy.
  • the i is an integer greater than or equal to 1 or less than or equal to N, where N is the total number of iterative calculations.
  • the i-th data enhancement strategy is the data enhancement strategy determined by the first node during the i-th iterative calculation process. Among them, the data enhancement strategy determined in the previous i-1 iteration calculation process and each data enhancement strategy in the initial data enhancement strategy, that is, each historical data enhancement strategy determined by the first node at the current moment Strategy.
  • i is equal to 1, that is, the first iterative calculation process, there is no previous i-1 iterative calculation process, and the first node determines the first data enhancement strategy based on the initial data enhancement strategy.
  • the first i-1 iteration calculation process exists at this time, and the first node is based on the data enhancement strategy determined in the previous i-1 iteration calculation process and the initial data enhancement strategy.
  • the i-1th data enhancement strategy among the plurality of data enhancement strategies is determined.
  • the first node can determine at least one i-th data enhancement strategy.
  • the first node first predicts the operation level and operation probability of each data enhancement operation in the i-th data enhancement strategy, and then based on the predicted data enhancement operation in the i-th data enhancement strategy The operation level and operation probability in the i-th data enhancement strategy are used to generate the i-th data enhancement strategy.
  • the process of the i-th iterative calculation is implemented by the processes shown in the following steps 9031-9033.
  • Step 9031 For any one of the multiple data enhancement operations, the first node is based on the multiple operation levels of the any one of the data enhancement strategies and the multiple levels of the respective data enhancement strategies. an evaluation value to predict the operation level of any one of the data augmentation operations in the i-th data augmentation strategy, and the evaluation value of any data augmentation strategy is used to indicate the identification trained by the augmented samples obtained based on the any data augmentation strategy The pros and cons of the model.
  • the respective data enhancement strategies are the data enhancement strategies determined in the previous i-1 iterative calculation process and the respective data enhancement strategies in the initial data enhancement strategy, that is, each historical data enhancement strategy.
  • each variation distribution of each sample collected in this application scenario conforms to a normal distribution, that is, a Gaussian distribution.
  • the first node fits the Gaussian distribution to which the multiple operation levels of each data enhancement operation in the respective data enhancement strategies obey the operation levels of each data enhancement operation in the respective data enhancement strategies.
  • multivariate Gaussian distribution to predict a new level of operation for each data augmentation operation.
  • the new operation level of each data enhancement operation is also the operation level of each data enhancement operation in the i-th data enhancement strategy.
  • this step 9031 is implemented by the following steps A1-A2.
  • Step A1 Based on the plurality of operation levels and the plurality of evaluation values, the first node determines a multivariate Gaussian distribution to which the operation level of any one of the data enhancement operations in the i-th data enhancement strategy complies.
  • the plurality of operation levels are in one-to-one correspondence with the plurality of evaluation values, wherein any operation level in the plurality of operation levels corresponds to the evaluation value of the data enhancement strategy to which the any operation level belongs.
  • Each operation level in the multiple operation levels obeys a Gaussian distribution, and the Gaussian distribution obeyed by each operation level is the value range of the evaluation value corresponding to each operation level. Treated as the mean value of the Gaussian distribution that each operating level follows.
  • the Gaussian process is the generalization of the multivariate Gaussian probability distribution. First, it is assumed that there is a prior distribution p(f
  • the intensity level of any one of the data enhancement operations in the strategy x 1 is the intensity level of any one of the data enhancement operations in the first data enhancement strategy
  • x i-1 is the i-1th data enhancement strategy.
  • the intensity level of the data enhancement operation; f is the set of function values of the mapping function, and the mapping function is the difference between the intensity level of the data enhancement operation and the evaluation value of the data enhancement strategy to which the data enhancement operation belongs.
  • k(x p , x q ) exp(- ⁇
  • is the Euclidean distance between x p and x q ,
  • is a smoothing factor, which is generally a decimal to reduce the adverse effects caused by extreme values.
  • the first node determines the multiple operation levels and the i-th data based on the multiple operation levels and the multiple evaluation values.
  • the operation levels of any one of the data enhancement strategies in the enhancement operation strategy obey the joint distribution shown in the following formula (2).
  • the first node predicts the multivariate Gaussian distribution obeyed by the operation level of any data enhancement operation in the i-th data enhancement strategy, that is, predicts the value of the evaluation value of the i-th data enhancement strategy. value range.
  • the multivariate Gaussian distribution obeyed by the operation level of any data enhancement operation in the predicted i-th data enhancement strategy As shown in the following formula (3).
  • ⁇ * is the mean value of the Gaussian distribution of the predicted evaluation value of the ith data enhancement strategy
  • ⁇ * is the variance of the Gaussian distribution of the predicted evaluation value of the ith data enhancement strategy.
  • Step A2 under the multivariate Gaussian distribution that the operation level of this any kind of data enhancement operation obeys, this first node determines the operation level that makes the acquisition function take the maximum value as this any kind of data enhancement in the i-th data enhancement strategy.
  • the operation level of the operation is the operation level of the operation.
  • the acquisition function is represented by the following formula (4).
  • the first node substitutes the intensity level of any one of the data enhancement operations in each historical data enhancement strategy and the evaluation value of each historical data enhancement strategy into the above formula (4) to obtain the configured any one of the data enhancement operations For each function value of the acquisition function under multiple intensity levels of able to make If the value is larger, the first node determines the intensity level at which the acquisition function takes the maximum value as the operation level of any one of the data enhancement operations in the first data enhancement strategy.
  • the number of multiple intensity levels of the configured data enhancement operation is relatively large, in order to quickly determine from the configured multiple intensity levels of the data enhancement operation that the acquisition function can be If the intensity level of the maximum value is taken, the first node calculates the operation level of the acquisition function to take the maximum value through the optimization algorithm.
  • the optimization algorithm is a quasi-Newton algorithm.
  • the quasi-Newton algorithm is implemented by the python package scipy.
  • the first node calls the scipy.optimize.minimize() function.
  • the first node will enhance the evaluation value of each data enhancement strategy And the operation level of any one of the data enhancement operations in the various historical data enhancement strategies, and - ⁇ (x) is input to the scipy.optimize.minimize() function, which can be obtained in the i-th data enhancement strategy.
  • the operation level of the data augmentation operation is input to the scipy.optimize.minimize() function, which can be obtained in the i-th data enhancement strategy.
  • step 9031 refers to the distribution fitting process of an operation level of a data enhancement operation provided by an embodiment of the present application shown in FIG. 12 .
  • x is the strength level
  • the evaluation value of the data enhancement strategy corresponding to x is y (that is, f(x)).
  • the first node First randomly select i intensity levels x (the black cross in Figure 12, that is, the operation level of any data enhancement operation in each historical data enhancement strategy), and the y corresponding to each x is understood as an x value (Curve 1 in Figure 12 is the estimated mean curve of this probability distribution, the gray area is the probability distribution based on a specific mean, and the curve 2 is the correspondence between the estimated intensity level x and the estimated value y
  • the function curve of the relationship since the y value corresponding to the black fork is a definite value, the variance of the y value will be very small, and it can be seen that the gray area above and below the black fork is almost absent.
  • Curve 1 is a curve composed of the mean value distribution of y values corresponding to all intensity levels x, and the gray area is the area covered by the variance distance above and below the mean value of the y value distribution corresponding to all intensity levels x points, that is, different
  • the y value corresponding to the intensity level x will appear in its corresponding gray area, and the probability of appearing on the curve 1 is the greatest.
  • the y value corresponding to the intensity level x must be on the mean line shown in curve 1, and since it has been determined that the variance will be very small, That is to say, the probability of the corresponding y taking a value other than the certain value is very small, so there is almost no gray area near the black cross.
  • the multivariate Gaussian distribution can be used to model the problem of 'what kind of x should be what kind of y value', that is, for the black cross It obeys the multivariate Gaussian distribution described in the above formula (1), which is based on the known multivariate Gaussian distribution composed of i black forks. And when the i+1th intensity level x is to be taken (that is, ) corresponding to When the value is distributed, the first node uses the multivariate Gaussian distribution shown in formula (1) to estimate the corresponding The distribution of , that is, the above formula (2) is obtained. then for a new K -1 , K ** , etc. can be calculated.
  • the first node can predict the new point corresponding The value probability distribution of .
  • the x value corresponding to the maximum value of the acquisition function ⁇ (x) shown in the above formula (4), where ⁇ is the control factor, which is used to control how much weight to consider The value within the variance range, when ⁇ 0, the first node only takes the value such that Take the x value with the largest mean in the value distribution, ignoring The possibility of taking a larger value within the variance range.
  • the subsequent first node can On the basis of , continue to evaluate to obtain the corresponding The real value of , the known x, y values will add another pair, which will make the y value calculated by the first node using the multivariate Gaussian distribution more and more accurate, and the estimated x * is more likely to be obtained. higher y values. Iterating continuously, the first node can get the x value corresponding to the higher and higher y value.
  • the x value corresponding to the highest y value is the desired x, that is, the optimal intensity level of any numerical enhancement operation.
  • Step 9032 for any one of the data enhancement operations, the first node predicts the first node based on the multiple operation probabilities of the any one of the data enhancement strategies and the multiple evaluation values of the respective data enhancement strategies.
  • the operation probability of any of the data augmentation operations in the data augmentation strategy is not limited to the data augmentation operations.
  • the first node fits the multivariate Gaussian distribution to which the operation probabilities of each data enhancement operation in the respective data enhancement strategies obey through the Gaussian distribution obeyed by the multiple operation probabilities of each data enhancement operation in the respective data enhancement strategies , to predict the new operation probability of each data augmentation operation.
  • the new operation probability of each data enhancement operation is also the operation probability of each data enhancement operation in the i-th data enhancement strategy.
  • this step 9032 is implemented by the following steps B1-B2.
  • Step B1 the first node determines, based on the plurality of operation probabilities and the plurality of evaluation values, a multivariate Gaussian distribution to which the operation probability of any one of the data enhancement operations obeys.
  • step B1 The process shown in this step B1 is the same as the process shown in the above-mentioned step A1, the difference is that this step B1 takes the operation probability of any one of the data enhancement operations as the random variable x, and the above-mentioned step A1 takes the operation probability of the any data enhancement operation as the random variable x.
  • the operation intensity of a data enhancement operation is taken as a random variable x, and the process shown in this step B1 will not be repeated in this embodiment of the present application.
  • Step B2 under the multivariate Gaussian distribution subject to the operation probability of any one of the data enhancement operations, the first node determines the operation probability of making the acquisition function take the maximum value as the i-th data enhancement strategy for any one of the data enhancement strategies.
  • the operation probability of the operation is the operation probability of the operation.
  • step B2 The process shown in this step B2 is the same as the process shown in the above-mentioned step A2, the difference is that this step B2 takes the operation probability of any one of the data enhancement operations as the random variable x, and the above-mentioned step A2 takes the operation probability of the any data enhancement operation as the random variable x.
  • the operation intensity of a data enhancement operation is taken as a random variable x, and the process shown in this step B2 is not repeated in this embodiment of the present application.
  • Step 9033 The first node generates the i-th data enhancement strategy based on the predicted operation levels and operation probabilities of the various data enhancement operations.
  • the first node performs the process shown in the above steps 9031-9032 for each of the multiple data enhancement operations, so that the first node can obtain the predicted operation level and operation of the multiple data enhancement operations probability.
  • the first node identifies the operation identifiers of the multiple data enhancement operations, the predicted operation levels and operation probabilities of the multiple data enhancement operations , packaged into the i-th data augmentation strategy.
  • the first node After the first node determines the i-th data enhancement strategy, the first node further acquires the evaluation value of the i-th data enhancement strategy.
  • the first node adopts the i-th data enhancement strategy to perform data enhancement for the samples in the application scenario, and performs model training based on the enhanced samples after data enhancement, and evaluates the i-th data enhancement strategy through the trained recognition model. good or bad.
  • the process of evaluating the value of the i-th data enhancement strategy by the first node is implemented by the process described in the following steps C1-C3.
  • Step C1 the first node performs the multiple data enhancement operations on multiple second samples based on the operation probability and operation level of the multiple data enhancement operations in the i-th data enhancement strategy, and obtains the multiple second samples. Multiple augmented samples.
  • step C1 is the same as the process shown in step 9011 , and this step C1 is not repeated in this embodiment of the present application.
  • Step C2 the first node obtains the ith recognition model based on the plurality of enhanced samples through training.
  • the ith recognition pattern is also the recognition model trained based on the enhanced samples obtained from the ith data enhancement strategy.
  • the process shown in this step C2 is the same as the process shown in step 9022, and here, this embodiment of the present application does not repeat the description of this step C2.
  • Step C3 the first node obtains the evaluation value of the ith data enhancement strategy based on the ith identification model.
  • step C3 is the same as the process shown in step 9023, and here, this embodiment of the present application does not repeat the description of this step C3.
  • the preset condition includes at least one of the following: the total number of data enhancement strategies determined in the previous i iterations reaches the target number; the average evaluation value of the data enhancement strategies determined in the previous i iterations reaches target evaluation value.
  • the first node terminates the iterative calculation.
  • the process shown in this step 903 is a process in which the first node acquires multiple data enhancement policies.
  • the process shown in this step 903 is performed by the policy optimizer in the first node, for example, the policy optimizer 1013 in the first node 101 shown in FIG. 2 .
  • the first node determines the first target data enhancement strategy based on multiple data enhancement strategies, where the first target data enhancement strategy includes a target operation probability and a target operation level of each data enhancement operation among the multiple data enhancement operations.
  • the first target data enhancement strategy is an optimal data enhancement strategy finally determined by the first node.
  • the first node uses the data enhancement strategy with the highest evaluation value among the plurality of data enhancement strategies as the first target data enhancement strategy. Since the multiple data enhancement strategies are obtained by searching through multiple operation levels of the configured multiple data enhancement operations, the first target data enhancement strategy is one of the multiple data enhancement strategies. Therefore, the first target data enhancement strategy is one of the multiple data enhancement strategies. The enhancement strategy is also obtained by searching through multiple operation levels based on multiple data enhancement operations configured.
  • the first node first selects a part of the data enhancement strategy from the plurality of data enhancement strategies, and then determines the first target data enhancement strategy based on the selected part of the data enhancement strategy .
  • the process shown in this step 904 is implemented by the processes shown in the following steps 9041-9042.
  • Step 9041 the first node selects a plurality of second target data enhancement strategies from the plurality of data enhancement strategies based on the evaluation values of the plurality of data enhancement strategies, and the evaluation values of the plurality of second target data enhancement strategies are all higher than the evaluation value of the data enhancement strategies in the plurality of data enhancement strategies other than the plurality of second target data enhancement strategies.
  • the first node sorts the evaluation values of the plurality of data enhancement strategies, and determines a second target data enhancement strategy among the plurality of data enhancement strategies based on the ranking result.
  • the first node sorts the evaluation values of the plurality of data enhancement strategies in descending order of the evaluation values to obtain a first evaluation value sequence, and the first node takes the first evaluation value sequence.
  • the evaluation values ranked at the former target position in the evaluation value sequence are used as the evaluation values of the second target data enhancement strategy, so that the first node selects a plurality of second target data enhancement strategies from the plurality of data enhancement strategies.
  • the first node sorts the evaluation values of the multiple data enhancement strategies in order of the evaluation values from small to large to obtain a second evaluation value sequence, and the first node applies the second evaluation value to the second evaluation value sequence.
  • the evaluation value ranked at the rear target position in the evaluation value sequence is used as the evaluation value of the second target data enhancement strategy, so that the first node selects a plurality of second target data enhancement strategies from the plurality of data enhancement strategies.
  • the target bit is an integer greater than 1, and the embodiment of the present application does not limit the number of target bits.
  • Step 9042 The first node generates the first target data enhancement strategy based on the plurality of second target data enhancement strategies.
  • step 9042 is implemented by the process shown in the following steps D1-D2.
  • Step D1 the first node determines target operation probabilities and target operation levels of the multiple data enhancement operations based on the operation probabilities and operation levels of the multiple data enhancement operations in the multiple second target data enhancement strategies.
  • the first node determines a target operation probability and a target operation level of any data enhancement operation based on the clustering result of the intensity level of any data enhancement operation.
  • this step D1 is implemented by the processes shown in the following steps D11-D12.
  • Step D11 for any one of the multiple data enhancement operations, the first node performs clustering on the multiple operation levels of the any one of the multiple second target data enhancement strategies, Get at least one operation class category.
  • the first node performs clustering on a plurality of operation levels of any one data enhancement operation in the plurality of second target data enhancement strategies based on a density clustering algorithm, to obtain the at least one Operational level categories, wherein one operational level category includes at least one operational level of any one of the data enhancement operations.
  • the density clustering algorithm is a density-based spatial clustering of applications with noise (DBSCAN), which is a density-based spatial clustering algorithm that can The area of is divided into clusters, and clusters of arbitrary shapes are found in the spatial database with noise.
  • the cluster is the largest set of densely connected points (ie, operation levels), and the operation level of any data enhancement operation in an operation level category That is, a cluster.
  • intensity levels of any one of the data augmentation operations in the plurality of second target data augmentation strategies which are intensity levels 1, 3, 5, 6, and 7, respectively.
  • the intensity classes are clustered to obtain operation class categories (ie, clusters) 1 and 2, wherein the operation class class 1 includes intensity classes 1 and 3, and the operation class class 2 includes intensity classes 5, 6, and 7.
  • the first node may also use other types of clustering algorithms other than DBSCAN to cluster multiple operation levels of any data enhancement operation.
  • Other types of clustering algorithms are not limited.
  • Step D12 for any operation level category in the at least one operation level category, the first node enhances the third target data based on the operation level in the any operation level category and the plurality of second target data enhancement strategies
  • the operation probability of any data enhancement operation in the strategy determine the target operation intensity and target operation probability of the any data enhancement operation
  • the third target data enhancement strategy is the operation level to which the operation level of any operation level category belongs. Two-target data augmentation strategy.
  • the first node determines the target operation level of any one of the data enhancement operations based on the operation level in the any operation level category.
  • the first node determines a minimum operation level and a maximum operation level in any operation level category; and determines each operation level greater than or equal to the minimum operation level and less than or equal to the maximum operation level as the The target operation level for any data augmentation operation.
  • the minimum operation level in the operation level category 1 is the operation level 1
  • the maximum operation level is the operation level 3
  • each operation level is greater than or equal to the minimum operation level and less than or equal to the maximum operation level.
  • the first node determines the operation levels 1-3 as the target operation levels of any one of the data enhancement operations.
  • the first node determines each operation level in the any operation level category as the target operation level of the any one data enhancement operation. Taking the above operation level category 1 as an example, the first node determines the operation level 1 and operation level 3 in the operation level category 1 as the target operation level of any data enhancement operation.
  • the first node determines, based on an operation probability of any one of the data enhancement operations in a third target data enhancement strategy among the plurality of second target data enhancement strategies, the any one of the data enhancement operations The target action probability for the action.
  • the first node determines the average probability of the operation probability of any one of the data enhancement operations in the third target data enhancement strategy in the plurality of second target data enhancement strategies as the any one of the data enhancement operations.
  • target operation probability For example, if operation level 1 includes operation levels 1 and 3, the operation probability of any data enhancement operation in the data enhancement strategy to which operation level 1 belongs is 0.6, and the data enhancement strategy to which operation level 3 belongs (that is, the third target data The operation probability of any data enhancement operation in the enhancement strategy) is 0.4. When the target operation level is operation level 1 and 3, the target operation probability of any data enhancement operation is the average probability between 0.6 and 0.4. 0.5.
  • the product of the target operation probability and the number of operation levels in any operation level category is the sum of the operation probabilities corresponding to each operation level in the any operation level category, wherein one operation level corresponds to the operation level.
  • the operation probability is the operation probability of any data enhancement operation in the data enhancement strategy to which the operation level belongs. For example, if operation level 1 includes operation levels 1 and 3, the operation probability of any data enhancement operation in the data enhancement strategy to which operation level 1 belongs is 0.6, and the operation probability of any data enhancement operation in the data enhancement strategy to which operation level 3 belongs is 0.6.
  • the operation probability of the operation is 0.3.
  • the target operation level is operation level 1, 2 and 3, and the operation level 2 belongs to any data enhancement strategy
  • the default operation probability corresponding to the operation level 2 is 0.
  • Step D2 the first node generates the first target data enhancement strategy based on the determined target operation probability and target operation level of the multiple data enhancement operations.
  • the first node packages the operation identifiers, target operation probabilities, and target operation probabilities of the multiple data enhancement operations into the first target data enhancement strategy.
  • step 904 is also the process of acquiring the first target data enhancement policy by the first node.
  • the process shown in this step 904 is performed by a policy optimizer in the first node, for example, the optimal policy generator 1014 in the first node 101 shown in FIG. 2 .
  • the first node Based on the first target data enhancement strategy, the first node performs the various data enhancement operations on the first sample within the operation intensity variation interval indicated by the target operation levels of the various data enhancement operations to obtain the first sample. Multiple target augmentation samples of the same sample.
  • the first sample is a sample composed of data collected in the application scenario.
  • the subject of the first sample is of the same class as the subject of the second sample, or the subject of the first sample and the subject of the second sample are the same subject.
  • the objects of the first sample and the objects of the second sample belong to different categories. The embodiments of the present application are described by taking an example that the object of the first sample and the object of the second sample belong to the same category, that is, the object of the first sample belongs to the target category.
  • the first node may also obtain the first sample.
  • the first node obtains the first sample uploaded by the application node through the target interface.
  • the first sample may also be a sample stored locally by the first node, and the first node obtains the first sample from the locally stored samples.
  • the first node performs the multiple data enhancement operations on the first sample based on the target operation probability and target operation level of each of the multiple data enhancement operations in the first target data enhancement strategy, and obtains the multiple data enhancement operations.
  • target augmentation samples wherein, the target enhancement sample of the target operation probability of any data enhancement operation among the plurality of target enhancement samples has performed any one of the data enhancement operations.
  • the first node For any one of the data enhancement operations, the first node, based on the target operation probability of the any one of the data enhancement operations, within the operation intensity variation interval indicated by the target operation level of the any one of the data enhancement operations, Perform any one of the data enhancement operations on a sample, and the process is the same as the operation indicated by the operation level of the any one of the data enhancement operations based on the operation probability of the first node in the above step 9021.
  • the process of performing any one of the data enhancement operations on the plurality of second samples is the same, and the process is not repeated in this embodiment of the present application.
  • the process shown in this step 905 is performed by a policy optimizer in the first node, for example, the optimal policy generator 1014 in the first node 101 shown in FIG. 2 .
  • the first node obtains a target recognition model by training based on a plurality of target enhancement samples.
  • the target recognition model is a model that is ultimately used to recognize samples in the application scenario. After acquiring the target recognition model, the first node associates the target recognition model with the type identifier of the target type, so that The target recognition model is subsequently used in the sample recognition of objects of this target type.
  • the process shown in this step 906 is similar to the process in the above-mentioned step 9022 that the first node obtains the initial recognition model by training based on the plurality of enhanced samples of the plurality of second samples, and this step 906 is not repeated in this embodiment of the present application.
  • the process shown in this step 906 is performed by a policy optimizer in the first node, for example, the optimal policy generator 1014 in the first node 101 shown in FIG. 2 .
  • the first node after the first node obtains the first target data enhancement strategy, the first node can also send the multiple target enhancement samples to the application node through the target interface, and the application node
  • the process shown in this step 906 is performed, for example, the cloud scenario shown in FIG. 6 .
  • the data enhancement method provided by the embodiments of the present application processes the samples through various data enhancement operations in the data enhancement strategy, and can obtain multiple target enhanced samples, thereby achieving the purpose of expanding the samples. Perform a variety of data enhancement operations on the samples within the operation intensity variation interval indicated by the target operation level of the enhancement operation, so that the obtained multiple target enhancement samples can be in the operation intensity variation interval indicated by the target operation level of the various data enhancement operations. variation within the sample, thus increasing the diversity of the sample.
  • the above-mentioned process shown in FIG. 9 is a process in which the first node independently completes the data augmentation under the condition that the first node has the functions of each node in the data augmentation system.
  • the data enhancement process is completed by each node in the data enhancement system collaboratively.
  • FIG. 13 To further illustrate the process, refer to the flowchart of a data enhancement method provided by the embodiment of the present application shown in FIG. 13 .
  • the first node acquires an initial data enhancement policy.
  • step 1301 is the same as the process shown in the above-mentioned step 901, and here, in this embodiment of the present application, this step 1301 is not repeated.
  • the process shown in this step 904 is performed by the policy optimizer in the first node, for example, the policy optimizer 1013 in the first node 101 shown in FIG. 2 .
  • the first node sends the initial data enhancement policy to the second node.
  • the second node evaluates the received initial data enhancement strategy to obtain an evaluation value of the initial data enhancement strategy.
  • step 1303 The process shown in this step 1303 is the same as the process shown in the above-mentioned step 902, and this step 1303 is not repeated in this embodiment of the present application.
  • the second node sends the evaluation value of the initial data enhancement policy to the first node.
  • the first node determines a first data enhancement strategy among the plurality of data enhancement strategies based on the initial data enhancement strategy.
  • step 1305 The process shown in this step 1305 is the same as the process shown in the above-mentioned steps 9031-9033.
  • this embodiment of the present application does not repeat the description of this step 1305.
  • the first node sends the first data enhancement policy to the second node.
  • the second node evaluates the received first data enhancement strategy to obtain an evaluation value of the first data enhancement strategy.
  • the process shown in this step 1307 is the same as the process shown in the above-mentioned step 902, and this step 1307 is not repeated in this embodiment of the present application.
  • the process shown in this step 904 is performed by the policy optimizer in the first node, for example, the policy optimizer 1013 in the first node 101 shown in FIG. 2 .
  • the second node sends the evaluation value of the first data enhancement policy to the first node.
  • the first node is based on the data enhancement strategy determined in the previous i-1 iteration calculation process and each of the initial data enhancement strategies.
  • Data augmentation strategy determine the i-th data augmentation strategy.
  • the process shown in this step 1309 is the same as the process shown in the above-mentioned steps 9031-9033.
  • this embodiment of the present application does not repeat the description of this step 1309.
  • the process shown in this step 904 is performed by the policy optimizer in the first node, for example, the policy optimizer 1013 in the first node 101 shown in FIG. 2 .
  • the first node sends the i-th data enhancement policy to the second node.
  • the second node evaluates the received i-th data enhancement strategy to obtain an evaluation value of the i-th data enhancement strategy.
  • step 1311 is the same as the process shown in the above-mentioned step 902, and here, this embodiment of the present application does not repeat the description of this step 1311.
  • the second node sends the evaluation value of the i-th data enhancement policy to the first node.
  • the first node after the first node acquires multiple data enhancement strategies in each iteration process, the first node sends the multiple data enhancement strategies to different The second node of , different data enhancement strategies are evaluated by different second nodes, so as to realize distributed evaluation.
  • the first node sends at least one i-th data enhancement strategy to multiple second nodes respectively, and each second node sends at least one i-th data enhancement strategy.
  • Each received i-th data enhancement strategy is evaluated based on the process shown in the following step 1311, so that the first node subsequently receives an evaluation value of at least one i-th data enhancement strategy from the plurality of second nodes respectively.
  • the second node is also an evaluation device, which is used to evaluate the data enhancement policy sent by the first node.
  • FIG. 14 a schematic diagram of a data enhancement system provided by an embodiment of the present application is shown.
  • the data enhancement strategy is to perform data enhancement on the samples obtained from the data server to obtain multiple enhanced samples, train and test the recognition model based on the multiple enhanced samples, and return the evaluation value of the data enhancement strategy to the first node.
  • a scheduling node is further set in the data enhancement system, and each time the second node evaluates a data enhancement strategy, the second node sends the evaluation value of the data enhancement strategy to the scheduling node; if The evaluation value of the data enhancement strategy is lower than the target evaluation value, and it is considered that the data enhancement strategy is not optimal, and the scheduling node instructs the second node not to return the evaluation value of the data enhancement strategy to the first node.
  • the first node discards the data enhancement strategy, so that the evaluation values of multiple data enhancement strategies finally determined by the subsequent first node are all greater than or equal to the target evaluation value, that is, the multiple data enhancement strategies finally determined by the first node are all better.
  • the scheduling node after the second node receives a data enhancement strategy, if the scheduling node does not obtain the evaluation value of the data enhancement strategy for a long time, it is considered that the second node is in the process of evaluating the data enhancement strategy There is a problem, for example, the second node cannot train the recognition model based on the enhanced samples obtained by the data enhancement strategy for a long time, then the scheduling node instructs the second node to end the evaluation process of the data enhancement strategy and proceed to the next data enhancement strategy. evaluation process. For example, the scheduling node in Figure 14.
  • the first node sends the multiple data enhancement strategies to the third node.
  • the first node does not send the plurality of data enhancement strategies to the third node after obtaining the plurality of data enhancement strategies, but every time the first node obtains a data enhancement strategy For sending to the third node, this embodiment of the present application does not limit the timing at which the first node sends the multiple data enhancement policies to the third node.
  • the first node sends the initial data enhancement policy to the third node in addition to the plurality of data enhancement policies to the third node.
  • the third node determines the first target data enhancement strategy based on the received multiple data enhancement strategies.
  • step 1314 is the same as the process shown in the above-mentioned step 904, and here, this embodiment of the present application does not repeat the description of this step 1314.
  • the third node sends the first target data enhancement policy to the application node.
  • the application node acquires a target recognition model based on the first target data enhancement strategy.
  • the application node performs multiple data enhancement operations on the first sample based on the first target data enhancement strategy to obtain multiple target enhancement samples of the first sample, and the application node is based on A plurality of target enhancement samples of at least one first sample are trained to obtain a target recognition model.
  • This process is the same as the process shown in the foregoing steps 905 and 906, and here, this embodiment of the present application does not describe this step 1316 in detail.
  • FIG. 15 is a schematic diagram of a data change distribution provided by an embodiment of the present application.
  • the horizontal axis is each rotation angle of the rotation operation
  • the vertical axis is the rotation operation of multiple enhanced samples through different rotation angles.
  • the proportion of enhanced samples, the distribution formed by curve 1 is the change distribution of the real data in the application scenario
  • the distribution formed by curve 2 is the change distribution of the data of the enhanced sample obtained by using the first target data enhancement operation, as shown in Figure 15
  • the vertical line of is the change distribution of the data of the enhanced sample obtained by using the data enhancement strategy provided by the related technology. It can be seen from Figure 15 that the vertical line can only cover a very small part of the real data change distribution in this application scenario, while the data change distribution of the enhanced sample obtained by using the first target data enhancement operation can cover a larger part of the real data change distribution. A wide range of. Therefore, the subsequent enhanced samples generated based on the first target data enhancement strategy can better simulate real data, and the recognition model trained by the enhanced samples based on the first target data enhancement strategy has stronger generalization ability and is more adaptable real application scenarios.
  • the change distribution of the data enhancement operation in the first target data enhancement strategy can also be expressed as the change distribution of the pixel values of the enhancement samples.
  • the data enhancement operation is a sharpening operation. When the sharpening operation is performed, the color contrast on both sides of the contour of the part related to the contour line of the object in the image (sample) will be improved.
  • the sample is an image of a bird
  • the image enhanced sample
  • the black and white pixels near the bird's beak in the original image The contrast is not strong, and the black and white contrast near the bird's beak in the image obtained after the strong sharpening operation will be stronger, and these contrasts can be reflected in the relevant changes of each pixel.
  • pixels are all composed of red, green, blue, RGB triples, which consist of three numbers ranging from 0 to 256, representing red, green, and blue, respectively. Therefore, the variation distribution of the data enhancement operation can be expressed as the variation distribution of the pixel value of the enhancement sample.
  • FIG. 16 a schematic diagram of distribution of specific pixel value changes when a sharpening operation is changed provided by an embodiment of the present application.
  • Fig. 16 shows the change distribution of the numerical value of a certain pixel point in an image enhanced by the data enhancement method of the present application, and shows the enhancement of the sharpening operation provided by the related art.
  • the horizontal axis is the value of the specific pixel after the image data is enhanced into several enhanced images (due to cartography, in order to show the contrast effect, the values on the horizontal axis are multiplied by 100), and the vertical axis is the image data.
  • the number of enhanced images with this value is taken on the specific pixel point, that is, the value distribution on the specific pixel point after the image is enhanced (that is, the change distribution of data enhancement).
  • the data enhancement in the related art can only bring about limited changes, while the data changes brought about by the data enhancement performed in the present application are presented as a distribution, and there are infinitely many change points on this distribution.
  • different data enhancement strategies are evaluated by using samples of different object types. Therefore, the data enhancement strategies corresponding to different object types are different. .
  • the distribution of pixel changes of birds and airplanes is different, and related technologies can only use the same data enhancement strategy for different object types, so regardless of birds or airplanes, the pixel changes of related technologies after data enhancement are the same of.
  • steps 1314-1316 the process shown in steps 1314-1316 is that the first node sends multiple data enhancement strategies to the third node, and the third node determines the first target data enhancement strategy based on the received multiple data enhancement strategies, and send it to the application node, and the application node obtains the first target data enhancement policy from the third node.
  • the first node determines the first target data enhancement strategy based on multiple data enhancement strategies, and sends the determined first target data enhancement strategy to the application node.
  • the application node Based on the first target data enhancement strategy, the application node performs data enhancement on multiple samples obtained from the data server to obtain enhanced samples, performs model training based on the multiple enhanced samples, and deploys the trained recognition model Services, such as the first node and the application node in the data augmentation system shown in FIG. 14 .
  • the process of data enhancement is completed through the cooperation of multiple nodes, and since the multiple nodes work in parallel, the efficiency of data enhancement performed by the data enhancement system is improved.
  • the data enhancement strategy provided in the related art and the data enhancement strategy provided in this application are used to respectively perform data enhancement on the samples in the public data set CIFAR10NEI.
  • the accuracy rate of the trained recognition model is 92.3% for the enhanced samples obtained by the data enhancement strategy provided by the related technology.
  • the accuracy rate of the recognition model trained by the enhanced samples obtained by the provided data enhancement strategy is 96.1%. It can be seen that the accuracy rate of the recognition model trained by the data enhancement strategy provided in this application can be increased by 4%.
  • the search time used to search for the final data enhancement strategy in the related art is 5000 GPU hours, while the present application performs distributed evaluation in the process of searching for the final data enhancement strategy, and the evaluation process is based on pre-training.
  • the search time used in this application to search for the final data enhancement strategy is 114 GPU hours, which is 50 times less than the search time in the related art.
  • the accuracy of the recognition model and the search time of the data augmentation strategy are compared, as shown in Table 1 below.
  • FIG. 17 is a schematic structural diagram of a data enhancement apparatus provided by an embodiment of the present application.
  • the apparatus 1700 includes:
  • the obtaining module 1701 is used to obtain a first target data enhancement strategy, the first target data enhancement strategy is used to indicate the target operation level of the data enhancement operation, and a target operation level is used to indicate an operation intensity variation interval;
  • the enhancement module 1702 is configured to perform the various data enhancement operations on the first sample within the operation intensity variation interval indicated by the target operation levels of the various data enhancement operations based on the first target data enhancement strategy, to obtain A plurality of target augmentation samples of the first sample.
  • a change corresponding to any data enhancement operation occurs in the any data in the plurality of target enhancement samples.
  • a data enhancement operation is within the operation intensity variation interval indicated by the target intensity level of the data enhancement operation.
  • the first target data enhancement strategy is obtained by searching based on multiple operation levels of the configured multiple data enhancement operations.
  • Any data enhancement operation is configured with multiple operation levels, and one operation Levels are used to indicate a range of intensity variations.
  • the first target data enhancement strategy is determined based on multiple data enhancement strategies, and the multiple data enhancement strategies are obtained by searching through multiple operation levels of the configured multiple data enhancement operations, Any data enhancement operation is configured with a plurality of operation levels, and one operation level is used to indicate an intensity change interval.
  • the multiple data enhancement strategies are evaluated by multiple evaluation devices.
  • the objects of the second samples used in the evaluation of the multiple data enhancement strategies are the same object or the same type of object.
  • the multiple operation levels configured by any one of the data enhancement operations are obtained based on the variation range of the target operation intensity of the any one of the data enhancement operations, and the target of the any one of the data enhancement operations
  • the operating intensity variation range is configured by the configuration device.
  • the apparatus 1700 further includes:
  • a division module configured to divide the target operation intensity variation range corresponding to any one of the multiple data enhancement operations into a plurality of operation intensity change intervals for any one of the multiple data enhancement operations
  • a configuration module configured to configure an operation level for each operation intensity change interval in the plurality of operation intensity change intervals.
  • the first target data enhancement strategy is also used to indicate the target operation probability of each data enhancement operation in the multiple data enhancement operations, and the target operation probability of any one data enhancement operation is the target operation probability of the data enhancement operation. the probability that the first sample performs any one of the data augmentation operations;
  • the obtaining module 1701 is used for:
  • the first target data enhancement strategy is determined based on a plurality of data enhancement strategies, and any data enhancement strategy is used to indicate the operation probability and operation level of each data enhancement operation among the multiple data enhancement operations, and any data enhancement strategy
  • the operation probability of the operation is the probability of performing any one of the data augmentation operations on the second sample.
  • the obtaining module 1701 includes:
  • the values are all higher than the evaluation values of the data enhancement strategies other than the plurality of second target data enhancement strategies in the plurality of data enhancement strategies, and the evaluation value of any one of the data enhancement strategies is used to indicate that the The pros and cons of the recognition model trained by the augmented samples obtained by the data augmentation strategy;
  • a generating submodule is configured to generate the first target data enhancement strategy based on the plurality of second target data enhancement strategies.
  • the generating submodule includes:
  • a determining unit configured to determine target operation probabilities and target operation levels of the multiple data enhancement operations based on the operation probabilities and operation levels of the multiple data enhancement operations in the multiple second target data enhancement strategies;
  • a generating unit configured to generate the first target data enhancement strategy based on the determined target operation probabilities and target operation levels of the multiple data enhancement operations.
  • the determining unit includes:
  • a clustering subunit configured to perform clustering on a plurality of operation levels of any one of the data enhancement operations in the plurality of second target data enhancement strategies for any one of the data enhancement operations, to obtain at least one operation class category;
  • a determining subunit configured to, for any operation level category in the at least one operation level category, based on the operation level in the any operation level category and the third target in the plurality of second target data enhancement strategies
  • the operation probability of any one of the data enhancement operations in the data enhancement strategy, determining the target operation intensity and target operation probability of the any one of the data enhancement operations, and the third target data enhancement strategy is the any operation level category
  • the second target data augmentation strategy to which the operational level of .
  • the determining subunit includes:
  • a first determining element configured to determine the target operation level of any one of the data enhancement operations based on the operation level in the any operation level category
  • the second determining element is configured to determine the target of any one of the data enhancement operations based on the operation probability of the any one of the data enhancement operations in the third target data enhancement strategy in the plurality of second target data enhancement strategies Action probability.
  • the first determining element is used for:
  • Each operation level greater than or equal to the minimum operation level and less than or equal to the maximum operation level is determined as the target operation level of any one of the data enhancement operations.
  • the first determining element is used for:
  • Each operation level in the any operation level category is determined as the target operation level of the any one data enhancement operation.
  • the target operation probability of any one of the data enhancement operations is the average probability of the operation probabilities of the any one of the data enhancement operations in the third target data enhancement strategy in the plurality of second target data enhancement strategies.
  • the apparatus 1700 also includes:
  • the iterative module is configured to perform iterative calculation based on the initial data enhancement strategy to obtain the multiple data enhancement strategies.
  • the iteration module is used to:
  • the plurality of data enhancement strategies are determined based on the data enhancement strategy determined in the previous i-1 iteration calculation process and each data enhancement strategy in the initial data enhancement strategy.
  • the i-th data enhancement strategy in wherein the i is an integer greater than or equal to 1 or less than or equal to N, and the i is the total number of iterative calculations.
  • the iteration module includes:
  • the first prediction submodule is used for any one of the data enhancement operations, based on multiple operation levels of the any one of the data enhancement strategies in the respective data enhancement strategies and multiple evaluations of the respective data enhancement strategies value, predicts the operation level of any data enhancement operation described in the i-th data enhancement strategy, and the evaluation value of any data enhancement strategy is used to indicate that the enhanced sample obtained based on the any data enhancement strategy is trained. Identify the pros and cons of the model;
  • the second prediction sub-module is configured to, for any one of the data enhancement operations, based on multiple operation probabilities of the any one of the data enhancement strategies in the respective data enhancement strategies and multiple evaluations of the respective data enhancement strategies value, predicting the operation probability of any data enhancement operation described in the i-th data enhancement strategy;
  • a generation sub-module is configured to generate the i-th data enhancement strategy based on the predicted operation levels and operation probabilities of the multiple data enhancement operations.
  • the first prediction submodule is used for:
  • the operation level that makes the acquisition function take the maximum value is determined as the operation level of any one of the data enhancement operations in the i-th data enhancement strategy .
  • the second prediction submodule is used for:
  • the operation probability that makes the acquisition function take the maximum value is determined as the operation probability of any one of the data enhancement operations in the i-th data enhancement strategy .
  • the apparatus 1700 also includes:
  • an obtaining module configured to perform the multiple data enhancement operations on a plurality of the second samples based on the operation probability and the operation level of the multiple data enhancement operations in the i-th data enhancement strategy, to obtain the multiple data enhancement operations a plurality of augmented samples of the second sample;
  • a training module for obtaining the ith recognition model based on the multiple enhanced sample training
  • a target acquisition module configured to acquire the evaluation value of the i-th data enhancement strategy based on the i-th identification model.
  • the training module is used to:
  • the pre-trained model is trained to obtain the i-th recognition model, the pre-trained model is obtained by training multiple third samples of multiple objects of different types, and the The precision is less than the precision threshold.
  • the object of the first sample and the object of the second sample belong to the same category, or the object of the first sample and the object of the second sample are the same object.
  • the apparatus 1700 further includes:
  • a first sending module configured to send at least one i-th data enhancement operation strategy to a plurality of evaluation devices respectively;
  • the first receiving module is configured to respectively receive the evaluation value of the at least one i-th data enhancement operation strategy from the plurality of evaluation devices.
  • the first target data enhancement strategy is also used to indicate the target operation probability of each data enhancement operation in the multiple data enhancement operations, and the target operation probability of any data enhancement operation is the target operation probability of the first data enhancement operation. the probability that the sample performs any of the data augmentation operations;
  • the enhancement module 1702 is used to:
  • the multiple data enhancement operations are performed on the first sample based on the target operation probability and target operation level of each data enhancement operation in the multiple data enhancement operations, wherein the value of any one of the data enhancement operations is
  • the target enhancement samples of the target operation probability have performed any one of the data enhancement operations.
  • the enhancement module 1702 is used to:
  • the first sample is subjected to any of the data augmentation operations.
  • the apparatus 1700 further includes:
  • a second receiving module configured to obtain the first sample from the application node through the target interface
  • a second sending module configured to send the multiple target enhanced samples to the application node through the target interface.
  • Embodiments of the present application also provide a computer program product or computer program, where the computer program product or computer program includes computer instructions, where the computer instructions are stored in a computer-readable storage medium, and the processor of the computing device is obtained from the computer-readable storage medium.
  • the computer instructions are read, and the processor executes the computer instructions, so that the computing device executes the data augmentation method described above.
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of modules in the apparatus or units in a module is only a logical function division. In actual implementation, there may be other division methods, such as multiple divisions. Modules or units may be combined or may be integrated into another system, or some features may be omitted, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.
  • modules or units described as separate components may or may not be physically separated, and components shown as modules or units may or may not be physical modules or physical units, that is, they may be located in one place, or may be distributed. to multiple computing devices. Some or all of the modules or units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present invention.
  • each functional module or unit in each embodiment of the present invention may be integrated into one target processing module, or each module or unit may exist physically alone, or two or more modules or units may be integrated into one in the target processing module.
  • the above-mentioned integrated modules or units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种数据增强方法、装置、计算设备以及计算机可读存储介质,属于人工智能技术领域。通过数据增强策略中的多种数据增强操作,对样本进行处理,能够得到多个目标增强样本,从而达到了扩充样本的目的,并且由于是在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对样本进行多种数据增强操作,从而得到的多个目标增强样本能够在多种数据增强操作的目标操作等级所指示的操作强度变化区间之内具有变化,提高了样本的多样性。

Description

数据增强方法、装置、计算设备以及计算机可读存储介质
本申请要求于2020年12月30日提交中国专利局、申请号为202011606784.4、发明名称为“数据增强方法、装置、计算设备以及计算机可读存储介质”的中国专利申请的优先权,以及于2020年11月09日提交的申请号为202011237954.6、发明名称为“实现数据增强策略的方法和系统”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能技术领域,特别涉及一种数据增强方法、装置、计算设备以及计算机可读存储介质。
背景技术
随着人工智能(artificial intelligence,AI)技术的发展,深度学习模型作为人工智能的主流算法模型,被广泛运用在计算机视觉、自然语言处理、语言识别等领域,而且都取得了优异的表现。比起传统机器学习模型,深度学习模型能从大量标注数据(即由标签标注的样本)中萃取阶层性的表征(representation)。这些通过学习得到的、分层次的表征正是深度学习模型取得优越表现(不管是分类或回归)的关键。
例如具有识别功能的深度学习模型(简称识别模型)在学习的过程中需要大量的样本,在很多情况下,样本的数量、质量对训练出的识别模型的表现起到至关重要甚至决定性的影响。然而现有样本(包括公开的或私有的样本)并不能完全满足某些业务场景在数量和质量方面的需求。例如,在很多特殊或复杂业务场景下的样本的数据量不足或样本类别单一,致使训练出的识别模型出现准确度低、泛化能力低等问题。
目前,一般通过数据增强技术来扩充样本,例如按照数据增强策略中每种数据增强操作的操作强度,对样本进行每种数据增强操作,将得到的增强样本作为应用场景下实际的样本,以扩充该应用场景的样本量。通过这种数据增强操作策略中每种数据增强操作的操作强度,对样本进行每种数据增强操作,所得到的增强样本的类别单一,缺乏多样性。
发明内容
本申请实施例提供了一种数据增强方法、装置、计算设备、芯片及计算机存储介质,能够提高样本的多样性。该技术方案如下:
第一方面,提供了一种数据增强方法,由第一节点执行,所述方法包括:
获取第一目标数据增强策略;基于所述第一目标数据增强策略,在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对第一样本进行所述多种数据增强操作,得到所述第一样本的多个目标增强样本,其中,所述第一目标数据增强策略用于指 示数据增强操作的目标操作等级,一个目标操作等级用于指示一个操作强度变化区间。
本方法通过数据增强策略所指示的多种数据增强操作,对样本进行处理,能够得到多个目标增强样本,从而达到了扩充样本的目的,并且由于是在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对样本进行多种数据增强操作,从而得到的多个目标增强样本能够在多种数据增强操作的目标操作等级所指示的操作强度变化区间之内具有变化,提高了样本的多样性。
在一种可能的实现方式中,对于所述第一样本中的任一数据,所述多个目标增强样本中所述任一数据出现任一种数据增强操作对应的变化,在所述任一种数据增强操作的目标强度等级所指示的操作强度变化区间内。以所述第一样本为图像,所述任一数据为所述图像中的任一像素点,所述第一样本的目标增强样本为增强图像,任一种数据增强操作为旋转操作为例。旋转操作的目标强度等级所指示的操作强度变化区间内可以有无数个旋转角度(也即是操作强度),部分增强图像是第一节点在该操作强度变化区间内,对所述图像进行不同旋转角度的旋转操作得到,因此,对于该图像中的任一像素点,该任一像素点在这部分增强图像中的位置发生了旋转变化,且旋转变化体现在旋转角度上,旋转角度的变化是在旋转操作的目标强度等级所指示的操作强度变化区间之内的。可以理解的是,对于该图像中的某一个固定位置,由于这部分增强图像是对该图像进行不同旋转角度的旋转操作得到,因此,这部分增强图像中该固定位置的像素点发生了变化。
在一种可能的实现方式中,所述第一目标数据增强策略基于配置的所述多种数据增强操作的多个操作等级搜索得到,任一种数据增强操作配置有多个操作等级,一个操作等级用于指示一个强度变化区间。
在一种可能的实现方式中,所述第一目标数据增强策略基于多个数据增强策略确定,所述多个数据增强策略基于配置的所述多种数据增强操作的多个操作等级搜索得到,任一种数据增强操作配置有多个操作等级,一个操作等级用于指示一个强度变化区间。
在一种可能的实现方式中,所述多个数据增强策略由多个评估设备进行评估。
在一种可能的实现方式中,所述多个数据增强策略评估时所采用的第二样本的对象为同一个对象,或者为同一类型的对象。
在一种可能的实现方式中,所述任一种数据增强操作配置的多个操作等级基于所述任一种数据增强操作的目标操作强度变化范围得到,所述任一种数据增强操作的目标操作强度变化范围由配置设备进行配置。
在一种可能的实现方式中,所述任一种数据增强操作的配置过程包括:
对于所述多种数据增强操作中的任一种数据增强操作,将所述任一种数据增强操作对应的目标操作强度变化范围划分为多个操作强度变化区间;为所述多个操作强度变化区间中的每个操作强度变化区间配置操作等级。
基于上述可能的实现方式,第一节点将任一种数据增强操作的目标操作强度变化范围划分为多个操作强度变化区间后,为每个操作强度变化区间配置一个操作等级,从而在数据增强策略中可以用强度等级来指示操作强度变化区间,简化了操作强度变化区区间的表示形式。
在一种可能的实现方式中,所述第一目标数据增强策略还用于指示所述多种数据增强操作中每种数据增强操作的目标操作概率,所述任一种数据增强操作的目标操作概率为对所述第一样本进行所述任一种数据增强操作的概率;所述获取第一目标数据增强策包括:
基于多个数据增强策略,确定所述第一目标数据增强策略,任一数据增强策略用于指示所述多种数据增强操作中每种数据增强操作的操作概率以及操作等级,任一种数据增强操作的操作概率为对第二样本进行所述任一种数据增强操作的概率。
基于上述可能的实现方式中,该第一节点基于多种数据增强策略生成一个最优的数据增强策略,也即是第一目标数据增强策略,以便后续基于最优的目标数据增强策略来扩充样本以及提高样本的多样性。
在一种可能的实现方式中,所述基于多个数据增强策略,确定所述第一目标数据增强策略包括:
基于所述多个数据增强策略的评估值,从所述多个数据增强策略中选择出多个第二目标数据增强策略,基于所述多个第二目标数据增强策略,生成所述第一目标数据增强策略;
其中,所述多个第二目标数据增强策略的评估值均高于所述多个数据增强策略中除所述多个第二目标数据增强策略以外的数据增强策略的评估值,所述任一数据增强策略的评估值用于指示基于所述任一数据增强策略得到的增强样本所训练出的识别模型的优劣程度。
基于上述可能的实现方式,该第一节点可以先根据所述多个数据增强策略的评估值,从所述多个数据增强策略中选取一些较优的数据增强策略,例如第二目标数据增强策略,再基于选取出的多个第二目标数据增强策略,生成最优的第一目标数据增强策略,从而能够保证第一目标数据增强策略的质量。
在一种可能的实现方式中,所述基于所述多个第二目标数据增强策略,生成所述第一目标数据增强策略包括:
基于所述多个第二目标数据增强策略中所述多种数据增强操作的操作概率以及操作等级,确定所述多种数据增强操作的目标操作概率以及目标操作等级;基于确定出的所述多种数据增强操作的目标操作概率以及目标操作等级,生成所述第一目标数据增强策略。
在一种可能的实现方式中,所述基于所述多个第二目标数据增强策略中所述多种数据增强操作的操作概率以及操作等级,确定所述多种数据增强操作的目标操作概率以及目标操作等级包括:
对于所述任一种数据增强操作,对所述多个第二目标数据增强策略中所述任一种数据增强操作的多个操作等级进行聚类,得到至少一个操作等级类别;对于所述至少一个操作等级类别中的任一操作等级类别,基于所述任一操作等级类别中的操作等级、所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作强度以及目标操作概率,所述第三目标数据增强策略为所述任一操作等级类别的操作等级所属的第二目标数据增强策略。
在一种可能的实现方式中,所述基于所述任一操作等级类别中的操作等级、所述多 个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作强度以及目标操作概率包括:
基于所述任一操作等级类别中的操作等级,确定所述任一种数据增强操作的目标操作等级;基于所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作概率。
在一种可能的实现方式中,所述基于所述任一操作等级类别中的操作等级,确定所述任一种数据增强操作的目标操作等级包括:
确定所述任一操作等级类别中的最小操作等级以及最大操作等级;将大于等于所述最小操作等级且小于等于所述最大操作等级的各个操作等级,确定为所述任一种数据增强操作的目标操作等级。
基于上述可能的实现方式中,第一节点将大于等于所述最小操作等级且小于等于所述最大操作等级的各个操作等级,确定为所述任一种数据增强操作的目标操作等级,从而使得所述任一种数据增强操作的多个目标操作等级是连续的,则连续的目标操作等级对应的强度变化区间也是连续的,从而后续该第一节点在该任一种数据增强操作的目标操作等级所指示的操作强度变化区间内,对第一样本进行该任一种数据增强操作,所得到的增强样本在连续的操作强度变化区间出现变化,从而进一步增加了样本的多样性,更加符合真实应用场景中样本的变化情况。
在一种可能的实现方式中,所述基于所述任一操作等级类别中的操作等级,确定所述任一种数据增强操作的目标操作等级包括:
将所述任一操作等级类别中的各个操作等级,确定为所述任一种数据增强操作的目标操作等级。
一种可能的实现方式中,所述任一种数据增强操作的目标操作概率为所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率的平均概率。
在一种可能的实现方式中,所述基于多个数据增强策略,确定所述第一目标数据增强策略之前,所述方法还包括:
基于初始数据增强策略进行迭代计算,得到所述多个数据增强策略。
在一种可能的实现方式中,所述基于初始数据增强策略进行迭代计算包括:
在第i次迭代计算的过程中,基于在前i-1次迭代计算过程中所确定出的数据增强策略以及所述初始数据增强策略中的各个数据增强策略,确定所述多个数据增强策略中的第i数据增强策略,其中,所述i为大于等于1或小于等于N的整数,所述i为迭代计算的总次数。
在一种可能的实现方式中,所述基于在前i-1次迭代计算过程中所确定出的数据增强策略以及所述初始数据增强策略中的各个数据增强策略,确定所述多个数据增强策略中的第i数据增强策略包括:
对于所述任一种数据增强操作,基于所述各个数据增强策略中所述任一种数据增强操作的多个操作等级以及所述各个数据增强策略的多个评估值,预测所述第i数据增强策略中所述任一种数据增强操作的操作等级,任一数据增强策略的评估值用于指示基于所述任一数据增强策略得到的增强样本所训练出的识别模型的优劣程度;
对于所述任一种数据增强操作,基于所述各个数据增强策略中所述任一种数据增强操作的多个操作等级以及所述各个数据增强策略的多个评估值,预测所述第i数据增强策略中所述任一种数据增强操作的操作等级,任一数据增强策略的评估值用于指示基于所述任一数据增强策略得到的增强样本所训练出的识别模型的优劣程度;
对于所述任一种数据增强操作,基于所述各个数据增强策略中所述任一种数据增强操作的多个操作概率以及所述各个数据增强策略的多个评估值,预测所述第i数据增强策略中所述任一种数据增强操作的操作概率;
基于预测出的所述多种数据增强操作的操作等级以及操作概率,生成所述第i个数据增强策略。
基于上述可能的实现方式,第一节点基于所述各个数据策略中任一种数据增强操作的操作等级、操作概率以及所述各个数据策略中评估值,来预测新的数据增强策略中任一种数据增强操作的操作等级以及操作概率,以便后续预测出的数据增强策略能够具有较高的评价值。
在一种可能的实现方式中,所述基于所述各个数据增强策略中所述任一种数据增强操作的多个操作等级以及所述各个数据增强策略的多个评估值,预测所述第i数据增强策略中所述任一种数据增强操作的操作等级包括:
基于所述多个操作等级以及所述多个评估值,确定所述任一种数据增强操作的操作等级所服从的多元高斯分布;
在所述任一种数据增强操作的操作等级所服从的多元高斯分布下,将令采集函数取最大值的操作等级确定为所述第i数据增强策略中所述任一种数据增强操作的操作等级。
在一种可能的实现方式中,所述基于所述各个数据增强策略中所述任一种数据增强操作的多个操作概率以及所述各个数据增强策略的多个评估值,预测所述第i数据增强策略中所述任一种数据增强操作的操作概率包括:
基于所述多个操作概率以及所述多个评估值,确定所述任一种数据增强操作的操作概率所服从的多元高斯分布;
在所述任一种数据增强操作的操作概率所服从的多元高斯分布下,将令采集函数取最大值的操作概率确定为所述第i数据增强策略中所述任一种数据增强操作的操作概率。
在一种可能的实现方式中,所述确定所述多个数据增强策略中的第i数据增强策略之后,所述方法还包括:
基于所述第i数据增强策略中所述多种数据增强操作的操作概率以及操作等级,对多个所述第二样本进行所述多种数据增强操作,得到所述多个第二样本的多个增强样本;
基于所述多个增强样本训练得到第i识别模型;
基于所述第i识别模型,获取所述第i数据增强策略的评估值。
在一种可能的实现方式中,所述基于所述多个增强样本训练得到第i识别模型包括:
基于所述多个增强样本,对预训练模型进行训练,得到所述第i识别模型,所述预训练模型由不同种类的多个对象的多个第三样本训练得到,所述预训练模型的精确度小 于精确度阈值。
在一种可能的实现方式中,所述第一样本的对象与所述第二样本的对象属于同一种类,或所述第一样本的对象与所述第二样本的对象为同一对象。
基于上述可能的实现方式,该迭代计算过程中的各个数据增强策略均是采用同一个对象的样本或同一类型的对象的样本所进行的评估,从而使得各个数据增强策略的评估结果与同一个对象或同一类型的对象相关,因此,基于各个数据增强策略所最终生成的第一目标数据增强策略与同一个对象或同一类型的对象对应,则后续采用这种第一目标数据增强策略所训练的识别模型,在识别同一个对象或同一类型的对象时识别效果最优。
在一种可能的实现方式中,所述第i数据增强策略具有多个,所述生成所述第i数据增强策略之后,所述方法还包括:
向多个评估设备分别发送至少一个第i数据增强操作策略;
从所述多个评估设备分别接收所述至少一个第i数据增强操作策略的评估值。
在一种可能的实现方式中,所述第一目标数据增强策略还用于指示所述多种数据增强操作中每种数据增强操作的目标操作概率,任一种数据增强操作的目标操作概率为对所述第一样本进行所述任一种数据增强操作的概率;
所述基于所述第一目标数据增强策略,在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对第一样本进行所述多种数据增强操作包括:
基于所述多种数据增强操作中每种数据增强操作的目标操作概率以及目标操作等级,对所述第一样本进行所述多种数据增强操作,其中,所述任一种数据增强操作的目标操作概率的目标增强样本进行过所述任一种数据增强操作。
基于上述可能的实现方式,进行过各种数据增强操作的增强样本在所述多个增强样本中均有一定的占比,从而使得所述多个增强样本更加符合真实的应用场景。
在一种可能的实现方式中,所述基于所述多种数据增强操作中每种数据增强操作的目标操作概率以及目标操作等级,对所述第一样本进行所述多种数据增强操作包括:
对于所述任一种数据增强操作,基于所述任一种数据增强操作的目标操作概率,在所述任一种数据增强操作的目标操作等级所指示的操作强度变化区间范围内,对所述第一样本进行所述任一种数据增强操作。
在一种可能的实现方式中,通过目标接口从应用节点获取所述第一样本;
通过所述目标接口向所述应用节点发送所述多个目标增强样本。
第二方面,提供了一种数据增强装置,所述装置包括:
获取模块,用于获取第一目标数据增强策略,所述第一目标数据增强策略用于指示数据增强操作的目标操作等级,一个目标操作等级用于指示一个操作强度变化区间;
增强模块,用于基于所述第一目标数据增强策略,在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对第一样本进行所述多种数据增强操作,得到所述第一样本的多个目标增强样本。
在一种可能的实现方式中,对于所述第一样本中的任一数据,所述多个目标增强样本中所述任一数据出现任一种数据增强操作对应的变化,在所述任一种数据增强操作的目标强度等级所指示的操作强度变化区间内。
在一种可能的实现方式中,所述第一目标数据增强策略基于配置的所述多种数据增强操作的多个操作等级搜索得到,任一种数据增强操作配置有多个操作等级,一个操作等级用于指示一个强度变化区间。
在一种可能的实现方式中,所述第一目标数据增强策略基于多个数据增强策略确定,所述多个数据增强策略基于配置的所述多种数据增强操作的多个操作等级搜索得到,任一种数据增强操作配置有多个操作等级,一个操作等级用于指示一个强度变化区间。
在一种可能的实现方式中,所述多个数据增强策略由多个评估设备进行评估。
在一种可能的实现方式中,所述多个数据增强策略评估时所采用的第二样本的对象为同一个对象,或者为同一类型的对象。
在一种可能的实现方式中,所述任一种数据增强操作配置的多个操作等级基于所述任一种数据增强操作的目标操作强度变化范围得到,所述任一种数据增强操作的目标操作强度变化范围由配置设备进行配置。
在一种可能的实现方式中,所述装置还包括:
划分模块,用于对于所述多种数据增强操作中的任一种数据增强操作,将所述任一种数据增强操作对应的目标操作强度变化范围划分为多个操作强度变化区间;
配置模块,用于为所述多个操作强度变化区间中的每个操作强度变化区间配置操作等级。
在一种可能的实现方式中,所述第一目标数据增强策略还用于指示所述多种数据增强操作中每种数据增强操作的目标操作概率,所述任一种数据增强操作的目标操作概率为对所述第一样本进行所述任一种数据增强操作的概率;
所述获取模块用于:
基于多个数据增强策略,确定所述第一目标数据增强策略,任一数据增强策略用于指示所述多种数据增强操作中每种数据增强操作的操作概率以及操作等级,任一种数据增强操作的操作概率为对第二样本进行所述任一种数据增强操作的概率。
在一种可能的实现方式中,所述获取模块包括:
选择子模块,用于基于所述多个数据增强策略的评估值,从所述多个数据增强策略中选择出多个第二目标数据增强策略,所述多个第二目标数据增强策略的评估值均高于所述多个数据增强策略中除所述多个第二目标数据增强策略以外的数据增强策略的评估值,所述任一数据增强策略的评估值用于指示基于所述任一数据增强策略得到的增强样本所训练出的识别模型的优劣程度;
生成子模块,用于基于所述多个第二目标数据增强策略,生成所述第一目标数据增强策略。
在一种可能的实现方式中,所述生成子模块包括:
确定单元,用于基于所述多个第二目标数据增强策略中所述多种数据增强操作的操作概率以及操作等级,确定所述多种数据增强操作的目标操作概率以及目标操作等级;
生成单元,用于基于确定出的所述多种数据增强操作的目标操作概率以及目标操作等级,生成所述第一目标数据增强策略。
在一种可能的实现方式中,所述确定单元包括:
聚类子单元,用于对于所述任一种数据增强操作,对所述多个第二目标数据增强策略中所述任一种数据增强操作的多个操作等级进行聚类,得到至少一个操作等级类别;
确定子单元,用于对于所述至少一个操作等级类别中的任一操作等级类别,基于所述任一操作等级类别中的操作等级、所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作强度以及目标操作概率,所述第三目标数据增强策略为所述任一操作等级类别的操作等级所属的第二目标数据增强策略。
在一种可能的实现方式中,所述确定子单元包括:
第一确定元件,用于基于所述任一操作等级类别中的操作等级,确定所述任一种数据增强操作的目标操作等级;
第二确定元件,用于基于所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作概率。
在一种可能的实现方式中,所述第一确定元件用于:
确定所述任一操作等级类别中的最小操作等级以及最大操作等级;
将大于等于所述最小操作等级且小于等于所述最大操作等级的各个操作等级,确定为所述任一种数据增强操作的目标操作等级。
在一种可能的实现方式中,所述第一确定元件用于:
将所述任一操作等级类别中的各个操作等级,确定为所述任一种数据增强操作的目标操作等级。
在一种可能的实现方式中,所述任一种数据增强操作的目标操作概率为所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率的平均概率。
所述装置还包括:
迭代模块,用于基于初始数据增强策略进行迭代计算,得到所述多个数据增强策略。
在一种可能的实现方式中,所述迭代模块用于:
在第i次迭代计算的过程中,基于在前i-1次迭代计算过程中所确定出的数据增强策略以及所述初始数据增强策略中的各个数据增强策略,确定所述多个数据增强策略中的第i数据增强策略,其中,所述i为大于等于1或小于等于N的整数,所述i为迭代计算的总次数。
在一种可能的实现方式中,所述迭代模块包括:
第一预测子模块,用于对于所述任一种数据增强操作,基于所述各个数据增强策略中所述任一种数据增强操作的多个操作等级以及所述各个数据增强策略的多个评估值,预测所述第i数据增强策略中所述任一种数据增强操作的操作等级,任一数据增强策略的评估值用于指示基于所述任一数据增强策略得到的增强样本所训练出的识别模型的优劣程度;
第二预测子模块,用于对于所述任一种数据增强操作,基于所述各个数据增强策略中所述任一种数据增强操作的多个操作概率以及所述各个数据增强策略的多个评估值,预测所述第i数据增强策略中所述任一种数据增强操作的操作概率;
生成子模块,用于基于预测出的所述多种数据增强操作的操作等级以及操作概率,生成所述第i数据增强策略。
在一种可能的实现方式中,所述第一预测子模块用于:
基于所述多个操作等级以及所述多个评估值,确定所述任一种数据增强操作的操作等级所服从的多元高斯分布;
在所述任一种数据增强操作的操作等级所服从的多元高斯分布下,将令采集函数取最大值的操作等级确定为所述第i数据增强策略中所述任一种数据增强操作的操作等级。
在一种可能的实现方式中,所述第二预测子模块用于:
基于所述多个操作概率以及所述多个评估值,确定所述任一种数据增强操作的操作概率所服从的多元高斯分布;
在所述任一种数据增强操作的操作概率所服从的多元高斯分布下,将令采集函数取最大值的操作概率确定为所述第i数据增强策略中所述任一种数据增强操作的操作概率。
所述装置还包括:
得到模块,用于基于所述第i数据增强策略中所述多种数据增强操作的操作概率以及操作等级,对多个所述第二样本进行所述多种数据增强操作,得到所述多个第二样本的多个增强样本;
训练模块,用于基于所述多个增强样本训练得到第i识别模型;
目标获取模块,用于基于所述第i识别模型,获取所述第i数据增强策略的评估值。
在一种可能的实现方式中,所述训练模块用于:
基于所述多个增强样本,对预训练模型进行训练,得到所述第i识别模型,所述预训练模型由不同种类的多个对象的多个第三样本训练得到,所述预训练模型的精确度小于精确度阈值。
在一种可能的实现方式中,所述第一样本的对象与所述第二样本的对象属于同一种类,或所述第一样本的对象与所述第二样本的对象为同一对象。
在一种可能的实现方式中,所述装置还包括:
第一发送模块,用于向多个评估设备分别发送至少一个第i数据增强操作策略;
第一接收模块,用于从所述多个评估设备分别接收所述至少一个第i数据增强操作策略的评估值。
在一种可能的实现方式中,所述第一目标数据增强策略还用于指示所述多种数据增强操作中每种数据增强操作的目标操作概率,任一种数据增强操作的目标操作概率为对所述第一样本进行所述任一种数据增强操作的概率;
所述增强模块用于:
基于所述多种数据增强操作中每种数据增强操作的目标操作概率以及目标操作等级,对所述第一样本进行所述多种数据增强操作,其中,所述任一种数据增强操作的目标操作概率的目标增强样本进行过所述任一种数据增强操作。
在一种可能的实现方式中,所述增强模块用于:
对于所述任一种数据增强操作,基于所述任一种数据增强操作的目标操作概率,在 所述任一种数据增强操作的目标操作等级所指示的操作强度变化区间范围内,对所述第一样本进行所述任一种数据增强操作。
在一种可能的实现方式中,所述装置还包括:
第二接收模块,用于通过目标接口从应用节点获取所述第一样本;
第二发送模块,用于通过所述目标接口向所述应用节点发送所述多个目标增强样本。
第三方面,提供了一种计算设备,该计算设备包括处理器和存储器,该存储器中存储有至少一条程序代码,该程序代码由该处理器加载并执行以实现如上述第一方面中的数据增强方法所执行的操作。
第四方面,提供了一种计算机可读存储介质,该存储介质中存储有至少一条程序代码,该程序代码由处理器加载并执行以实现如上述第一方面中的数据增强方法所执行的操作。
第五方面,提供了一种计算机程序产品或计算机程序,该计算机程序产品或计算机程序包括程序代码,该程序代码存储在计算机可读存储介质中,计算设备的处理器从计算机可读存储介质读取该程序代码,处理器执行该程序代码,使得该计算机设备执行上述第一方面或者第一方面的各种可选实现方式中提供的方法。
附图说明
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种数据增强系统的示意图;
图2是本申请实施例提供的一种第一节点的结构示意图;
图3是本申请实施例提供的一种数据增强策略的示意图;
图4是本申请实施例提供的另一种数据增强系统的结构示意图;
图5是本申请实施例提供的一种第二节点的工作原理图;
图6是本申请实施例提供的一种云端场景下的数据增强系统的示意图;
图7是本申请实施例提供的一种计算设备的结构示意图;
图8是本申请实施例提供的一种初始化配置的流程图;
图9是本申请实施例提供的一种数据增强方法的流程图;
图10是本申请实施例提供的一种样本标签、对象以及拍摄方式之间的因果关系示意图;
图11是本申请实施例提供的一种训练过程中识别模型的精确度变化曲线图;
图12是本申请实施例提供的一种数据增强操作的操作等级的分布拟合过程;
图13是本申请实施例提供的一种数据增强方法的流程图;
图14是本申请实施例提供的一种数据增强系统的示意;
图15是本申请实施例提供的一种数据变化分布的示意图;
图16是本申请实施例提供的一种锐化操作变化时特定像素值变化的分布示意图;
图17是本申请实施例提供的一种数据增强装置的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。
图1是本申请实施例提供的一种数据增强系统的示意图,参见图1,数据增强系统100包括第一节点101、第二节点102以及第三节点103。在该数据增强系统100中,第一节点101基于至少一个数据增强策略的评估结果,预测能够达到更好评估结果的数据增强策略,并将预测到的数据增强策略反馈给第二节点102,由第二节点102对预测到的数据增强策略进行评估,并将该预测到的数据增强策略的评估结果反馈给第一节点101,其中,该至少一个数据增强策略为该第一节点历史预测到的数据增强策略。
此时对于该第一节点101而言,上一次预测到的且具有评估结果的数据增强策略为一个历史预测到的数据增强策略,说明第一节点101历史预测到的各个数据增强策略出现了更新。则该第一节点101基于历史预测到的各个数据增强策略的评估结果,再次预测能够达到更好评估结果的数据增强策略。以此类推,直至第一节点101历史预测到的各个数据增强策略满足预设条件,则该第一节点101将历史预测到的各个数据增强策略发送给第三节点103,由该第三节点103基于接收到的各个数据增强策略,生成至少一个第一目标数据增强策略。
之后,第三节点103将该至少一个第一目标数据增强策略提供给外部的应用节点,由该应用节点采用该任一第一目标数据增强策略,对多个样本进行数据增强,以得到大量的增强样本。以便应用节点后续采用大量的增强样本进行模型训练,以得到精度较高的识别模型。
为了进一步说明数据增强系统100中各个节点的工作原理,下面结合各个节点的物理架构,对各个节点进行如下描述。
(1)第一节点101
参见图2所示的本申请实施例提供的一种第一节点的结构示意图,该第一节点101包括操作池子1011、策略变化空间1012以及策略优化器1013。该操作池子1011用于为用户提供数据增强操作的配置。在一种可能的实现方式中,用户基于应用场景的需求,在操作池子1011中配置多种数据增强操作以及该多种数据增强操作中每种数据增强操作的目标操作强度变化范围。其中,该多种数据增强操作为该应用场景下的样本所需要的数据增强操作,一种数据增强操作的目标操作强度变化范围为:对该应用场景下的样本进行该数据增强操作的强度的变化范围。在该策略变化空间1012中,对于该多种数据增强操作中的任一种数据增强操作,第一节点基于操作池子1011中配置的该任一种数据增强操作的目标操作强度变化范围,为该任一种数据增强操作分配多个操作等级,并为该任一种数据增强操作分配概率变化区间,从而该任一种数据增强操作对应多个操作等级以及一个概率变化区间。该第一节点还能够将该多种数据增强操作中每种数据增强操作对应的操作等级以及概率变化区间,存储在策略变化空间1012中。其中,该任一种数据增强操作的一个操作等级用于指示一个操作强度变化区间,该操作强度变化区间为该种数据增强操作的目标操作强度变化范围的一个子区间。
初始时,对于该任一种数据增强操作,策略优化器1013从策略变化空间1012中随机选取该任一种数据增强操作的一个操作等级以及概率变化区间中的一个操作概率,以实现随机搜索出一个数据增强策略。其中,该数据增强策略包括策略变化空间1012中的多种数据增强操作中每种数据增强操作的操作概率以及操作等级。例如图3所示的本申请实施例提供的一种数据增强策略的示意图,图3所示的数据增强策略包括S种数据增强操作,分别为数据增强操1-S,数据增强策略还包括数据增强操作1-S的操作概率,分别为操作概率1-S,该数据增强策略还包括数据增强操作1-S的操作等级,分别为操作等级1-S,其中S为大于等于1的整数。需要说明的是,图3所示的数据增强策略中的任一种数据增强操作具有一个操作等级以及一个操作概率,而在另外一些实施例中,一个数据增强操作策略中的任一种数据增强操作的操作等级也可以具有多个操作等级。
当策略优化器1013搜索到第一个数据增强策略后,将该第一个数据增强策略发送给第二节点102,由第二节点102反馈该第一个数据增强策略的评估结果。策略优化器1013基于第一个数据增强策略的评估结果,在该策略变化空间1012中进行搜索,得到第二个数据增强策。由于第二个数据增强策略是基于第一个数据增强策略的评估结果所搜索到的,那么,对于第一个数据增强操作策略而言,第二个数据增强策略是未知的,因此,该第二个数据增强策略也即是该策略优化器1013预测的能够达到更好评估结果的数据增强策略。该策略优化器1013将该第二数据增强策略发送给第二节点102,由第二节点102反馈该第二数据增强策略的评估结果。该策略优化器1013基于该第一个数增强策略以及该第二个数据增强策略的评估结果,在该策略变化空间1012中进行继续搜索,得到第三个数据增强策略。以此类推,直至该策略优化器1013搜索到的各个数据增强策略满足预设条件,该策略优化器1013将各个数据增强策略发送给第三节点103,由第三节点103基于接收到的各个数据增强策略,生成至少一个第一目标数据增强策略。
在一些实施例中,该策略优化器1013还能够融合样本的样本标签,来预测数据增强策略。在一种可能的实现方式中,该策略优化器1013在迭代过程中预测到的数据增强策略面向的是具有相同样本标签的样本,或者面向的是属于同一类型的样本标签的样本。
在一些实施例中,一个对象的样本标签用于标识该对象以及该对象的种类,若训练集中样本的对象属于不同种类,该第一节点101可以最终生成不同类型的目标数据增强策略,其中,每种类型的目标数据增强策略对应一种类型的对象。例如,第一节点在生成任一类型的目标数据增强策略的迭代过程中,该策略优化器1013将预测到的数据增强策略以及该任一类型发送给第二节点102,以便第二节点102基于具有该任一类型的样本标签的样本,对接收到的数据增强策略进行评估。当任一类型的数据增强策略满足预设条件时,该策略优化器1013将该类型的各个数据增强策略发送给第三节点103,由各个第三节点103基于该类型的各个数据增强策略,生成针对该任一类型的最优数据增强策略(即目标数据增强策略),以便应用节点基于该任一类型的最优数据增强策略所得到增强样本,训练出适合该任一类型的各个对象的识别模型。
需要说明的是,图1所示的数据增强系统100是以第一节点101和第三节点103为两个独立的节点为例进行说明的,而在另外一些实施例中,该第一节点101和第三节点 103为同一个节点,该第三节点103可以作为该第一节点101内的一个单元,存在于该第一节点101中,例如图2所示的第一节点101还可以包括策略生成器1014,该策略生成器1014也即是第三节点103。
(2)第二节点102
该数据增强系统100中第二节点102的个数可以是一个或多个,第二节点102可以视为评估设备。在一些实施例中,第一节点101在每一次的迭代过程(也即是搜索过程)中能够预测出多个数据增强策略,若在该数据增强系统100中的第二节点102有多个的情况下,该第一节点101可以将在每一次预测出的多个数据增强策略分散发送至多个第二节点102。由每个第二节点102对该多个数据增强策略中的部分数据增强策略进行评估,并由每个第二节点102将其负责评估的数据增强策略的评估结果均发送至第一节点101。如此,使得第一节点101能够获取每一次的迭代过程中所预测出的多个数据增强策略的评估结果,实现了分布式评估。在这种分布式评估的过程中,由多个第二节点102并行评估,因此提高了数据增强系统100的评估效率。例如图4所示的本申请实施例提供的另一种数据增强系统的结构示意图,第一节点将每一次迭代过程中的多个数据增强策略发送给多个第二节点,由多个第二节点对这多个数据增强策略进行评估,并返回各个数据增强策略的评估结果。之后,第一节点基于历史各个数据增强策略,生成第一目标数据增强策略,并将第一目标数据增强策略发送给应用节点。由应用节点采用该第一目标数据增强策略,对从数据服务器获取到样本进行数据增强,并基于增强后得到的增强样本进行模型训练,并将训练出的识别模型部署服务。
在一种可能的实现方式中,对于接收到的任一数据增强策略,第二节点102采用该任一数据增强策略,对多个样本进行数据增强操作,得到该多个样本的多个增强样本。之后,该第二节点102再使用该多个增强样本对初始模型训练,得到识别模型。其中,这多个样本可以是第二节点102从数据服务器获取的。进一步地,第二节点102对该识别模型的识别效果进行评估,将该识别模型的评估结果作为该任一数据增强策略的评估结果,反馈给第一节点101。
在一些实施例中,第二节点102可以先获取一个预训练模型,该预训练模型可以是快要训练完成的识别模型,例如表现为:该预训练模型的精确度小于精确度阈值,当该第二节点102基于一个数据增强策略得到的多个增强样本后,该第二节点102采用该多个增强样本对该预训练模型进行训练,得到识别模型。例如图5所示的本申请实施例提供的一种第二节点的工作原理图,第二节点接收到一个数据增强策略后,采用该数据增强策略所得到的增强样本,对预训练模型进行再训练微调,得到识别模型,并对该识别模型的识别效果进行评估,以得到该数据增强策略的评估结果。
(3)第三节点103
在一种可能的实现方式中,该第三节点103基于接收到的多个数据增强策略的评估结果,从该多个数据增强策略中选取多个较优的数据增强策略(例如多个数据增强策略中评估结果较高的数据增强策略),并基于该多个较优的数据增强策略,生成第一目标数据增强策略。
当该多个数据增强策略是基于一个对象的多个样本进行评估的,或者是基于一个类型下的多个对象的多个样本评估时,则该第三节点103基于该多个数据增强策略最终生 成的第一目标数据增强策略与该对象或与该类型对应,则应用节点基于第一目标数据增强策略所得到增强样本,训练出识别模型,对该对象或者该类型下的对象的识别效果较佳。
需要说明的是,该数据增强系统100可以部署在多个计算设备,该多个计算设备分别用于实现第一节点101、第二节点102以及第三节点103的功能,此时该第一节点101、第二节点102以及第三节点103为相互独立的计算设备。而在一些实施例中,该第一节点101、第二节点102以及第三节点103中的部分节点或全部节点集成在一个计算设备中,那么该计算设备中的各个节点也即是该设备的一个单元,例如第一节点101还具有第二节点102以及第三节点103的功能。本申请实施例对该数据增强系统100中各个节点的部署方式不作限定。
在一些实施例中,第一节点101、第二节点102以及第三节点103也可以是云计算设备。当第一节点101、第二节点102以及第三节点103为云计算设备时,该数据增强系统100部署在云端。例如图6所示的本申请实施例提供的一种云端场景下的数据增强系统的示意图,应用节点通过目标接口向部署在云端的第一节点上传待增强的样本(如客户数据)。例如,该应用节点内显示有上传界面,用户在该上传界面内上传该待增强的样本,当该应用节点检测到用户在该上传界面内进行了确认操作后,该应用节点通过目标接口向第一节点发送待增强的样本,以实现上传。当该待增强的样本上传完成后,该第一节点在策略变化空间中随机搜索出多个数据增强策略,并向多个第二节点分别发送搜索出的至少一个数据增强策略以及该待增强的样本。当任一第二节点接收到该至少一个数据增强策略以及待增强的样本后,对于该至少一个数据增强策略中的任一数据增强策略,该任一第二节点基于该待增强的样本,对该任一数据增强策略进行评估,并向第一节点反馈该任一数据增强策略的评估结果。之后,该第一节点基于该多个数据增强策略的评估结果,生成多个新的数据增强策略,并再次由多个节点对该多个新的数据增强策略进行评估。以此类推,当历史生成的数据增强策略足够多时,该第一节点基于历史生成的各个数据增强策略,生成目标数据增强策略,该第一节点采用该目标数据增强策略,对该待增强的样本进行多种数据增强操作,得到该待增强样本的多个增强样本。该应用节点通过该目标接口下载该多个增强样本,以便后续该应用节点基于下载的该多个增强样本进行模型训练,得到识别模型,并将识别模型部署服务。其中,该目标接口为处于云端的数据增强系统与应用节点之间进行数据交互的软件接口或硬件接口。
在一种可能的实现方式中,该应用节点还可以直接通过该目标接口下载该目标数据增强策略,并解析该目标数据增强策略,基于解析出的该目标数据增强策略,对待增强样本进行多种数据增强操作,得到多个增强样本。
在一些实施例中,该第一节点包括中央处理器(central processing unit,CPU),由该CPU实现第一节点所执行的各个步骤,该第一节点包括图形处理器(graphics processing unit,GPU),由该CPU实现第二节点所执行的各个步骤。
在一些实施例中,当数据增强系统未部署在云端时,该数据增强系统中的第一节点或第三节点也可以生成多个目标数据增强策略,任一目标数据增强策略为基于第一节点历史生成的各个数据增强策略所最终生成的数据增强策略,任一目标数据增强策略对应一种对象类型或者一个对象;生成该多个目标数据增强策略的节点将每个目标数据增强 策略以及每个目标数据增强策略所对应的对象类型或对象的标识上传至云端,以便应用节点根据待增强样本,从云端下载对应的目标数据增强策略,之后该应用节点可以采用下载的目标数据增强策略,获取适用于待增强样本或该对象的识别模型。或者,该第一节点还可以基于每个目标数据增强策略分别训练出一个识别模型,并将每个识别模型与每个目标数据增强策略对应的对象类型或对象的标识关联,并上传至云端,由应用节点根据应用场景的需求,直接从云端下载所需的识别模型。再或者,该第一节点还可以基于每个目标数据增强策略所训练出一个识别模型,生成多个增强样本,并将多个增强样本以及每个增强样本的样本标签上传至云端,由应用节点根据应用场景的需求,直接从云端下载所需的每个增强样本。
本申请中的第一节点101、第二节点102以及第三节点103可以是服务器,该应用节点可以是终端、个人计算机(personal computer,PC)或者应用服务器等用户设备,本申请实施例对第一节点101、第二节点102、第三节点103以及应用节点的实现形式不作限定。
本申请实施例还提供一种计算设备的结构示意图,参见图7,计算设备700可被配置为第一节点、第二节点以及第三节点中的任一节点,该计算设备700可因配置或性能不同而产生比较大的差异,可以包括一个或一个以上处理器701和一个或一个以上的存储器702,其中,该存储器702中存储有至少一条指令,该至少一条指令由该处理器701加载并执行以实现下述每个方法实施例提供的方法。例如,若该计算设备700被配置为第一节点,则该至少一个指令由该处理器701加载并执行以实现下述各个方法实施例中第一节点所执行的步骤。当然,该计算设备700还可以具有有线或无线网络接口、键盘以及输入输出接口等部件,以便进行输入输出,该计算设备700还可以包括其他用于实现设备功能的部件,在此不做赘述。该处理器701可以是CPU、GPU、张量处理器(tensor processing unit,TPU)、神经网络处理器(neural network processing unit,NPU)、大脑处理器(brain processing unit,BPU)、深度学习处理器(deep learning processing unit,DPU)、全息图像处理器(holographic processing unit,HPU)、矢量处理器(vector processing unit,VPU)以及智能处理器(intelligence processing unit,IPU)等任一处理器。
处理器701可以采用通用的CPU、微处理器、应用专用集成电路(application specific integrated circuit,ASIC),GPU或者一个或多个集成电路,用于执行相关程序,以实现下述的数据增强方法。
处理器701还可以是一种集成电路芯片,具有信号的处理能力。在实现过程中,本申请的数据增强方法中任一节点所执行的步骤可以通过处理器701中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器701还可以是通用处理器、数字信号处理器(digital signal processing,DSP)、ASI、现成可编程门阵列(field programmable gate array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成,或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器(random access memory,RAM),闪存、只读存储器(read-only memory,ROM),可编程只读存储器或者电可擦写可编程存 储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器702,处理器701读取存储器702中的信息,结合其硬件完成本申请实施例的数据增强装置中包括的模块所需执行的功能,或者执行本申请方法实施例的数据增强方法。
在示例性实施例中,还提供了一种计算机可读存储介质,例如包括指令的存储器,上述指令可由终端中的处理器执行以完成下述实施例中的数据增强方法。例如,该计算机可读存储介质可以是ROM、RAM、只读光盘(compact disc read-only memory,CD-ROM)、磁带、软盘和光数据存储设备等。
以第一节点在获取应用于某一应用场景的最优数据增强策略之前,先针对该应用场景进行一下初始化配置为例,参见图8所示的本申请实施例提供的一种初始化配置的流程图。
801、第一节点配置多种数据增强操作以及该多种数据增强操作的目标操作强度变化范围。
该多种数据增强操作为某一应用场景下的样本所需要的数据增强操作,该应用场景为图像识别场景、音频视频场景、文本识别场景中的任一个,在此本申请实施例对该应用场景不作限定。
该样本由在该应用场景下采集到的数据组成,或者认为该样本是由真实数据组成,在本申请实施例中以该应用场景为图像识别场景为例进行说明,该样本可以是在该图像识别场景下具有拍摄功能的任一设备所拍摄到的图像。
为了扩充已有的样本,可以采用数据增强操作对已有的样本进行数据增强,得到增强样本,并将增强样本视为该应用场景中采集到的真实数据,以进行模型训练。但是在该应用场景下采集到的真实数据之间可能存在各种各样的变化,为了使得后续得到的增强样本能够高度模拟该应用场景下的真实数据,需要采用多种数据增强操作对已有的样本进行数据增强。例如在图像识别场景下,该多种数据增强操作包括翻转操作、旋转操作、缩放操作、剪裁操作、平移操作、添加噪声操作以及锐化操作中的至少一个,当然,还可以包括其他种类的数据增强操作。该多种数据增强操作可根据具体的应用场景进行适应性配置,在此,本申请实施例对该多种数据增强操作不作限定。
需要说明的是,在该应用场景中的真实数据的变化是一个分布,而不是特定的几个变化点。比如不同的人去拍摄同一只小猫,由于每个人拿手机的姿势不同,拍摄出的图像中猫的倾斜度和大小是会有无限多的变化的,而不仅是倾斜特定几个度数。当然,不仅是图像中猫的倾斜度的不同变化,在色彩饱和度,色温等方面,不同手机拍出来的图像都会有所不同。因此,为了模拟出真实数据的变化,需要对图像采取不同强度变化范围的数据增强操作,以便能够得到更多的增强样本,同时保证这些增强样本的变化分布能尽可能地与该应用场景中实际样本的变化分布一致。因此,该第一节点除了配置多种数据增强操作以外,还配置有该多种数据增强操作的目标操作强度变化范围,以便后续在每种数据增强操作的目标操作强度变化范围内,对样本进行每种数据操作,从而使得最终得到的增强样本的每种变化范围均在每种数据增强操作的目标操作强度变化范围内,以模拟该应用场景下的真实数据的变化分布。
其中,一种数据增强操作的目标操作强度变化范围包括多个操作强度,本申请实施例对每种数据增强操作的目标操作强度变化范围的操作强度范围不作限定,可根据实际 应用场景进行适应性配置。以图像识别场景为例,一种数据增强操作为旋转操作,旋转操作的操作强度可以用旋转角度来表示。若该图像识别场景下的多个图像中的对象会发生旋转,且旋转范围为[0度,180度],则第一节点可以给旋转操作配置的目标操作强度变化范围为[0度,180度],以便后续在[0度,180度]范围内,对该图像识别场景下的图像进行旋转操作,得到多个数据增强图像(也即是多个增强样本)。且该多个数据增强图像中的对象在[0度,180度]范围内发生了旋转,从而使得该多个数据增强图像符合该图像识别场景下的真实图像的变化规律,则可以将该多个数据增强数据作为该图像识别场景下的真实样本,以进行模型训练。
在一些实施例中,该第一节点基于人工操作,配置多种数据增强操作以及该多种数据增强操作的目标操作强度变化范围。在一种可能的实现方式中,该第一节点显示数据增强操作配置界面,由用户在数据增强操作配置界面中配置多种数据增强操作以及每种数据增强操作的目标操作强度变化范围。当检测到该用户在该数据增强操作配置界面进行了确认操作后,该第一节点存储用户在该数据增强操作配置界面所配置的多种数据增强操作以及每种数据增强操作的目标操作强度变化范围,配置完成。
在一些实施例中,本步骤801所示的过程由该第一节点中的操作池子来执行,例如图2所示的第一节点101中的操作池子1011。
需要说明的是,本步骤801由第一节点来执行,该第一节点可以视为配置设备。在一些实施例中,本步骤801不是由该第一节点来执行,而是由该第一节点以外的任一配置设备来执行。
802、对于该多种数据操作中的任一种数据增强操作,该第一节点基于该任一种数据增强操作的目标操作强度变化范围,将该任一种数据增强操作的操作强度划分为多个强度等级。
一个强度等级用于指示一个强度变化区间。该强度变化区间可以是该目标操作强度变化范围的任一子集。
该第一节点将该任一种数据增强操作对应的目标操作强度变化范围划分成多个强度变化区间,该多个强度变化区间之间不存在交集,该第一节点为该多个操作强度变化区间中的每个操作强度变化区间配置操作等级。其中,该多个强度变化区间内的操作强度与操作强度变化区间的强度等级正相关。也即多个强度变化区间中操作强度越高的操作强度变化区间的强度等级越高,反之,强度等级越低。
例如该任一种数据增强操作为旋转操作,该旋转操作的目标操作强度变化范围为[0度,180度],该第一节点将该目标操作强度变化范围[0度,180度]划分为10个强度变化区间,依次为[0度,18度)、[18度,36度)、…、[162度,180度],按照10个强度变化区间中的操作强度的高低,第一节点将该10个强度变化区间对应的强度等级依次配置为强度等级1、2、…、10,且这个10强度等级依次增高。
在一些实施例中,本步骤802所示的过程由该第一节点在策略变化空间中完成的,例如图2所示的第一节点101中的策略变化空间1012。
需要说明的是,本步骤802由第一节点来执行,该第一节点可以视为配置设备。在一些实施例中,本步骤802不是由该第一节点来执行,而是由该第一节点以外的任一配置设备来执行。
803、该第一节点为该多种数据增强操作配置概率变化区间。
一个概率变化区间包括多个操作概率,任一种数据增强操作的一个操作概率为对样本进行该任一种数据增强操作的概率。
在一些实施例中,该第一节点为该多种数据增强操作中的每种数据增强操作配置相同的概率变化区间,或者为该多种数据增强操作配置不同的概率变化区间,本申请实施例对该第一节点为该多种数据增强操作配置的概率变化区间不作限定。
在一些实施例中,该第一节点直接将该多种数据增强操作配置相同的概率变化区间,例如[0,1]。
在一些实施例中,本步骤803所示的过程由该第一节点在策略变化空间中完成的,例如图2所示的第一节点101中的策略变化空间1012。
需要说明的是,本步骤803与步骤802的执行没有先后顺序,第一节点可以先执行本步骤803再执行上述步骤802,在此,本申请实施例对步骤802-803的执行顺序不作限定。
当该第一节点配置完该多种数据增强操作的强度等级以及概率变化区间后,初始化配置完成。而除了单独使用数据增强操作对样本实施变化,以增加样本的多样性并模拟真实场景所遇到的数据变化,还可以通过对不同的数据增强操作进行组合,形成多个数据增强策略,以达到更多样性的数据变化,并且第一节点还能够基于多个数据增强策略,来获取第一目标数据增强策略,以便后续采用这个第一目标数据增强策略,来扩充样本。为了进一步说明这个过程,参见图9所示的本申请实施例提供的一种数据增强方法的流程图。
901、第一节点获取初始数据增强策略,该初始数据增强策略用于指示多种数据增强操作的操作概率以及强度等级。
示例性地,该多种数据增强操作中的每种数据增强可以由一个操作标识来标识,不同数据增强操作的操作标识不同。在一种可能的实现方式中,该初始数据增强策略包括该多种数据增强操作的操作标识、操作概率和强度等级,每种数据增强操作的操作标识与每种数据增强操作的操作强度以及操作概率对应。在一种可能的实现方式中,该初始数据增强策略中该多种数据增强操作的操作概率之和为1,该初始数据增强策略中每种数据增强操作的强度等级具有至少一个,每种数据增强的操作概率为1个。需要说明的是,本申请实施例以初始数据增策略中每种数据增强操作具有一个强度等级为例进行说明。
在一些实施例中,该第一节点获取到的初始数据增策略可以是一个,也可以是多个。在此,本申请实施例对第一节点获取的初始数据增强策略的个数不作限定,需要说明的是,本申请实施例以第一节点获取一个初始数据增策略为例进行说明。
在一种可能的实现方式中,该第一节点获取一个初始数据增强策略的过程是:对于配置的多种数据增强操作中的任一种数据增强操作,该第一节点从配置的该任一种数据增强操作的多个强度等级中随机选择出一个强度等级,并从配置的该任一种数据增强操作的概率变化区间内随机选择出一个操作概率;该第一节点基于该多种数据增强操作的操作标识、所选择出的该多种数据增强操作中每种数据增强操作的强度等级以及操作概率,生成一个初始数据增强策略。
在一些实施例中,本步骤901所示的过程由该第一节点中的策略优化器来执行,例如图2所示的第一节点101中的策略优化器1013。
由于数据增强操作的种类比较多,而不同种类的数据增强操作的不同组合其实又是另一形式的数据增强操作,因此,第一节点要从获取到的多个数据增强策略中找到较优的数据增强策略,则需要对获取到的每个数据增强策略进行评估,以便基于获取到的多个数据增强策略的评估结果,获取较优的数据增强策略。其中,该初始数据增强策略为第一节点获取到的一个数据增强策略。在一种可能的实现方式中,该第一节点通过下述步骤902对该初始数据增强策略进行评估。
902、该第一节点对该初始数据增强策略进行评估,得到该初始数据增强策略的评估值,该初始数据增强策略的评估值用于指示基于该初始数据增强策略得到的增强样本所训练出的识别模型的优劣程度。
该初始数据增强策略的评估值也即是该第一节点对该初始数据增强策略的评估结果。可选地,该初始数据增强策略的评估值为基于该初始数据增强策略得到的增强样本所训练出的识别模型的精确度。
该第一节点采用该初始数据增强策略,对该应用场景下的样本进行数据增强,并基于数据增强后的增强样本进行模型训练,通过训练出的识别模型来评估该初始数据增强策略的好坏。在一种可能的实现方式中,本步骤902所示的过程由下述步骤9021-9023所示的过程来实现。
步骤9021、该第一节点基于该初始数据增强策略中该多种数据增强操作的操作概率以及操作等级,对多个第二样本进行该多种数据增强操作,得到该多个第二样本的多个增强样本。
对于该初始数据增强策略内该多种数据增强操作中的任一种数据增强操作,该多个增强样本中该任一种数据增强操作的操作概率的增强样本进行过该任一种数据增强操作。
该多个第二样本中的每个第二样本均由该应用场景下采集的数据组成,对于该多个第二样本中的任一第二样本,该任一第二样本具有一个样本标签,该任一第二样本的样本标签用于指示该任一第二样本的对象。可选地,该任一第二样本的样本标签包括对象标识,该对象标识用于标识该任一第二样本的对象。其中,该任一第二样本的对象是采集该任一第二样本时被采集的对象。例如,该任一第二样本是一只猫的图像,则这只猫也即是该任一第二样本的对象。在一些实施例中,多个第二样本的对象不同,或者该多个第二样本中部分第二样本的对象不同。
需要说明的是,本申请中所涉及到的样本均是在该应用场景下所采集的数据组成,为该应用场景下的真实数据。例如,上述的第二样本,下述的第一样本、第三样本以及第四样本等。本申请中所涉及到的增强样本均是对该应用场景下的真实数据进行数据增强操作所得到的,是用于模仿该应用场景下的真实数据的,以扩充该应用场景下的样本量。
本申请所涉及到的样本或者增强样本最终是用于模型训练的,而模型训练所用的样本的数据变化取决于产生这些样本的方式。例如,用不同手机拍摄的同一只猫,拍摄出来的图像在图像属性上都会有所不同,比如有的手机拍出来的图像色温偏暖,有些偏冷。 又比如,拍静态的猫,和拍动态的鸟,猫的图和鸟的图像在图像属性上也会有明显的差别,鸟的图像比较模糊,照相机可能会自动加了更多的锐化操作,而猫的图像则不需要自动加锐化操作。例如图10所示的本申请实施例提供的一种样本标签、对象以及拍摄方式之间的因果关系示意图,其中,静态的旗杆和动态的蜂鸟作为对象,决定了拍摄方式的不同以及所拍摄的图像的样本标签不同,而拍摄方式不同决定了拍摄出的图像的变化分布不同,拍摄出的图像以及图像的标签来形成训练集,以进行模型训练。
而采用数据增强策略如果要生成具有与真实数据总体尽可能相似分布的数据的增强样本,就需要考虑到样本产生方式所带来的影响,然后针对不同样本产生方式计算出不同的数据增强策略。而样本产生的方式一般和样本的对象的种类或者样本的对象本身有关,因此,在一些实施例中,该多个第二样本的对象属于同一种类。例如,该多个第二样本的对象均为鸟类、两栖类或者爬行类型中的任一类。或者,在一些实施例中,该多个第二样本的对象为同一对象,例如该多个第二样本的对象均为猫或者均为鸟。可选地,该每个第二样本的样本标签还包括每个第二样本的对象的种类标识,若该多个第二样本的对象属于同一种类,则该多个第二样本的样本标签所包括的种类标识相同。为了便于描述将多个第二样本的对象所属的种类记为目标种类;若多个第二样本的对象为同一对象,则该多个第二样本的样本标签所包括的对象标识相同。
在一种可能的实现方式中,对于该初始数据增强策略内的多种数据增强操作中的任一种数据增强操作,该第一节点基于该任一种数据增强操作的操作概率,在该任一种数据增强操作的操作等级所指示的操作强度变化区间内,对该多个第二样本进行该任一种数据增强操作,从而得到进行过该任一种数据增强操作的多个增强样本。
对于该多个第二样本中的任一第二样本,该第一节点内设置有所需的该任一第二样本的增强样本的目标个数,该第一节点基于该目标个数以及该任一种数据增强操作的操作概率,在该任一种数据增强操作的强度等级所指示的操作强度变化区间内,对该任一第二样本多次进行该任一种数据增强操作,得到该任一第二样本的进行过该任一种数据增强操作的多个增强样本。其中,在该任一第二样本的多个增强样本中,进行过该任一种数据增强操作的增强样本的个数为该目标个数与该任一种数据增强操作的操作概率之间的积。
为了便于理解该过程,以多个第二样本的个数为M,该初始数据增强策略为图3所示的数据增强策略为例,对本步骤9021进行如下说明:对于该初始数据增强策略内的S种数据增强操作中的第j种数据增强操作以及该多个第二样本中的第r个第二样本,若第j种数据增强操作的操作概率为0.2,且该第一节点内设置的所需的该第r个第二样本的增强样本的目标个数为100,则在该初始数据增强策略中该第j种数据增强操作的强度等级所指示的强度变化区间内,该第一节点对该第r个第二样本进行20次第j种数据增强操作,得到该第r个第二样本的20个增强样本。其中,该j为大于等于1或小于等于S的整数,该r为大于等于1或小于等于M的整数。
在一种可能的实现方式中,在该任一种数据增强操作的强度等级所指示的操作强度变化区间内,第一节点对该任一第二样本多次进行该任一种数据增强操作的过程为:该第一节点等概率的按照该任一种数据增强操作的强度等级所指示的操作强度变化区间中的操作强度,对该任一第二样本多次进行该任一种数据增强操作。例如,若该任一种 数据增强操作为旋转操作,该初始数据增强策略中该旋转操作的强度等级所对应的操作强度变化区间为[0度,18度),且需要对该任一第二样本进行18次旋转操作,则该第一节点等概率的从该操作强度变化区间[0,18度)中选取出18个旋转角度,分别为0,1度,…,17度,该第一节点基于选取出的每个旋转角度,对该任一第二样本分别进行一次旋转操作,得到该任一第二样本的18个增强样本,从而这18个增强样本在旋转角度上的变化分布近似于[0,18度)。
需要说明的是,对该任一第二样本进行数据增强操作所得到的各个增强样本的对象与该任一第二样本的对象相同,该第一节点可以将该任一第二样本的样本标签作为对该任一第二样本进行数据增强操作所得到的各个增强样本的样本标签。
步骤9022、该第一节点基于该多个第二样本的多个增强样本训练得到初始识别模型。
该初始识别模型用于对样本进行识别,可选地,该初始识别模型是神经网络模型,例如深度神经网络模型。
在一些实施例中,该第一节点将该多个增强样本输入初始模型,采用该多个增强样本对该初始模型进行模型训练,得到该初始识别模型。其中,该初始模型为未经过任何训练的神经网络模型。
在另外一些实施例中,该第一节点先获取预训练模型,再基于该多个增强样本,对预训练模型进行训练,得到该初始识别模型。其中,该预训练模型由不同种类的多个对象的多个第三样本训练得到,该预训练模型的精确度小于精确度阈值,该精确度阈值可根据实际应用场景来设置,本申请实施例对该精确度阈值不作限定。
该多个第三样本由在该应用场景下的采集的数据组成,由于该预训练模型的精确度小于精确度阈值,则认为该预训练模型是未训练完成的模型。那么,第一节点基于该多个增强样本,继续对该预训练模型进行训练,得到该初始识别模型的时间,相对于从初始模型训练到该初始识别模型的时间少很多,从而能够缩短该第一节点评估该初始数据增强策略的总时长,提高了数据增强操作的评估效率。例如图11所示的本申请实施例提供的一种训练过程中识别模型的精确度变化曲线图,若该第一节点基于该多个增强样本对初始模型训练,若想得到精确度在95%以上的初始识别模型,则训练时长要200分钟,若该第一节点提前用115分钟的时间先训练出预训练模型,当获取到该多个增强样本后,再基于该多个增强样本对该预训练模型进行训练,得到精确度在95%以上的初始识别模型的训练时长为95分钟,明显缩短了评估过程中模型训练所占用的时长,提高了数据增强操作的评估效率。
示例性地,该第一节点获取预训练模型的过程为:该第一节点基于该多个第三样本对初始模型进行训练,得到该预训练模型。或者,该预训练模型由除该第一节点以外的其他设备来训练,该第一节点从其他设备处获取该预训练模型。本申请实施例对第一节点获取预训练模型的方式不作限定。另外,该第一节点获取一次预训练模型即可,无需多次获取。
由不同种类的对象的多个第三样本所训练出的预训练模型,适用于不同种类的对象的增强样本,因此,即使多个增强样本的对象属于同一种类或者是同一个对象,第一节点基于该多个增强样本对该预训练模型进行训练,仍然能够训练出初始识别模型。由于 多个增强样本的对象属于同一种类或者是同一个对象,因此,所训练的初始识别模型在识别目标类型的对象或该多个增强样本的对象的其他样本时识别效果较优。
步骤9023、该第一节点基于该初始识别模型,获取该初始数据增强策略的评估值。
示例性地,该第一节点采用多个第四样本对该初始识别模型进行测试,并基于测试结果获取该初始识别模型的精确度,该第一节点将该初始识别模型的精确度获取为该初始数据增强策略的评估值。其中,该多个第四样本由在该应用场景下采集的数据组成;若该多个第二样本的对象的种类不同,则该多个第四样本的对象的种类不同;若该多个第二样本的对象均属于目标种类,则多个第四样本的对象也均属于目标种类;若该多个第二样本的对象为同一对象,则多个第四样本的对象与该多个第二样本的对象相同。该初始识别模型也即是基于该初始数据增强策略得到的增强样本所训练出的识别模型,该初始识别模型的精确度为该初始识别模型正确识别出该多个第四样本的对象的准确率。
在一种可能的实现方式中,该第一节点将该多个第四样本输入该初始识别模型,该初始识别模型对输入的每个第四样本进行识别,并输出每个第四样本的识别结果。若一个第四样本的识别结果所指示的对象与该第四样本的样本标签所指示的对象相同,则该第四样本识别正确,否则该第四样本识别错误。该第一节点统计该初始模型正确识别的第四样本的个数,将正确识别的第四样本的个数除以该多个第四样本的总个数,得到该初始识别模型的精确度,该第一节点并将该精确度作为该初始数据增强策略的评估值。
在一些实施例中,该第一节点除了将该初始识别模型的精确度作为该初始数据增强策略的评估值以外,该第一节点还能够将该初始识别模型的其他评价指标作为该初始数据增强策略的评估值,例如召回率、F1值或线下面积(area under the curve,AUC)值,在此,本申请实施例对该其他评价指标不作限定。
在一些实施例中,本步骤902所示的过程由该第一节点中的策略优化器来执行,例如图2所示的第一节点101中的策略优化器1013。
903、该第一节点基于该初始数据增强策略进行迭代计算,得到多个数据增强策略。
任一数据增强策略用于指示该多种数据增强操作中每种数据增强操作的操作概率以及操作等级。第i次迭代计算的过程中,该第一节点基于在前i-1次迭代计算过程中所确定出的数据增强策略以及该初始数据增强策略中的各个数据增强策略,确定该多个数据增强策略中的第i数据增强策略。其中,该i为大于等于1或小于等于N的整数,该N为迭代计算的总次数。该第i数据增强策略为在该第i次迭代计算的过程中,该第一节点所确定出的数据增强策略。其中,在前i-1次迭代计算过程中所确定出的数据增强策略以及该初始数据增强策略中的各个数据增强策略,也即是在当前时刻该第一节点所确定出的各个历史数据增强策略。
当i等于1时,也即是第一次迭代计算的过程,则不存在前i-1次迭代计算过程,该第一节点基于该初始数据增强策略,确定该多个数据增强策略中的第i数据增强策略。
当i不等于1时,此时存在前i-1次迭代计算过程,则该第一节点基于在前i-1次迭代计算过程中所确定出的数据增强策略以及该初始数据增强策略中的各个数据增强策略,确定该多个数据增强策略中的第i-1数据增强策略。
在该第i次迭代计算的过程中,该第一节点能够确定出至少一个第i数据增强策略。在确定一个第i数据增强策略的过程中,该第一节点先预测每种数据增强操作在该第i数 据增强策略中的操作等级以及操作概率,再基于预测出的每种数据增强操作在该第i数据增强策略中的操作等级以及操作概率,生成该第i数据增强策略。在一种可能的实现方式中,该第i次迭代计算的过程由下述步骤9031-9033所示的过程来实现。
步骤9031、对于该多种数据增强操作中的任一种数据增强操作,该第一节点基于该各个数据增强策略中该任一种数据增强操作的多个操作等级以及该各个数据增强策略的多个评估值,预测该第i数据增强策略中该任一种数据增强操作的操作等级,任一数据增强策略的评估值用于指示基于该任一数据增强策略得到的增强样本所训练出的识别模型的优劣程度。
其中,该各个数据增强策略为在前i-1次迭代计算过程中所确定出的数据增强策略以及该初始数据增强策略中的各个数据增强策略,也即是各个历史数据增强策略。
为了使得最终获取到的增强样本的每种变化分布均符合在该应用场景下所采集到的各个样本的每种变化分布,则预测的第i数据增强策略中的每种数据增强操作的操作强度的变化分布以及操作概率的变化分布均要符合该应用场景下所采集到的各个样本的变化分布。一般认为该应用场景下所采集到的各个样本的每种变化分布均符合正态分布,也即是高斯分布。可选地,该第一节点通过该各个数据增强策略中每种数据增强操作的多个操作等级所服从的高斯分布,拟合出该各个数据增强策略中每种数据增强操作的操作等级所服从的多元高斯分布,以预测出每种数据增强操作新的操作等级。其中,每种数据增强操作新的操作等级也即是第i数据增强策略中每种数据增强操作的操作等级。
在一种可能的实现方式中,本步骤9031由下述步骤A1-A2来实现。
步骤A1、该第一节点基于该多个操作等级以及该多个评估值,确定该第i数据增强策略中该任一种数据增强操作的操作等级所服从的多元高斯分布。
该多个操作等级与该多个评估值一一对应,其中,该多个操作等级中的任一操作等级与该任一操作等级所属的数据增强策略的评估值对应。该多个操作等级中的每个操作等级均服从一个高斯分布,而每个操作等级所服从的高斯分布为每个操作等级对应的评估值的取值范围,将每个操作等级对应的评估值视为每个操作等级所服从的高斯分布的平均值。
高斯过程是多元高斯概率分布的泛化,先假设该多个数据增强策略中该任一种数据增强操作的强度等级存在一个先验分布p(f|X),如下述公式(1)所示。
Figure PCTCN2021103015-appb-000001
其中,X为该多个数据增强策略中任一种数据增强操作的强度等级的集合,表示为X={x 0,x 1,...,x i-1},x 0为初始数据增强策略中该任一种数据增强操作的强度等级,x 1为第一数据增强策略中该任一种数据增强操作的强度等级,x i-1为第i-1数据增强策略中该任一种数据增强操作的强度等级;f为映射函数的函数值的集合,该映射函数为该任一种数据增强操作的强度等级与该任一种数据增强操作所属的数据增强策略的评估值之间的函数,或者认为:该任一种数据增强操作所属的数据增强策略的评估值也即是该映射函数的函数值,则f表示为f={f(x 0),f(x 1),...,f(x i-1)};μ为X的均值,K是由k pq所组成的协方差矩阵,其中,k pq=k(x p,x q),用于表示该X中x p与x q之间的相似距离(也即是相似程度),x p为X中的第p个强度等级,x q为该X中的第q个强度等级, 0≤p≤i-1,0≤q≤i-1。可选地,k(x p,x q)=exp(-γ|||x p-x q||2),||x p-x q||为x p与x q之间的欧式距离,γ为平滑因子,一般为小数,以减少极值带来的不良影响。
由于高斯过程是一个随机变量的集合,任意有限个随机变量都满足一个联合高斯分布,则该第一节点基于该多个操作等级以及该多个评估值,确定该多个操作等级与第i数据增强操作策略中该任一种数据增强策略的操作等级之间服从下述公式(2)所示的联合分布。
Figure PCTCN2021103015-appb-000002
其中,Y为观测到的未知函数的函数值集合,也即是各个历史数据增强策略的评估值的集合,则Y={f(x 0),f(x 1),...,f(x i-1)},
Figure PCTCN2021103015-appb-000003
为预测的第i数据增强策略的评估值,
Figure PCTCN2021103015-appb-000004
为预测的第i数据增强策略中该任一种数据增强策略的强度等级,m(X)为X的均值函数,
Figure PCTCN2021103015-appb-000005
Figure PCTCN2021103015-appb-000006
均值的均值函数,K **
Figure PCTCN2021103015-appb-000007
Figure PCTCN2021103015-appb-000008
之间的相似距离,K *
Figure PCTCN2021103015-appb-000009
与X中各个x之间相似距离,
Figure PCTCN2021103015-appb-000010
为X中的各个x与
Figure PCTCN2021103015-appb-000011
的相似距离。
该第一节点基于上述公式(2),预测第i数据增强策略中该任一种数据增强操作的操作等级所服从的多元高斯分布,也即是预测该第i数据增强策略的评估值的取值范围。其中,预测的第i数据增强策略中该任一种数据增强操作的操作等级所服从的多元高斯分布
Figure PCTCN2021103015-appb-000012
如下述公式(3)所示。
Figure PCTCN2021103015-appb-000013
Figure PCTCN2021103015-appb-000014
Figure PCTCN2021103015-appb-000015
(3)
其中,μ *为预测的该第i数据增强策略的评估值的高斯分布的均值,Σ *为预测的该第i数据增强策略的评估值的高斯分布的方差。
步骤A2、在该任一种数据增强操作的操作等级所服从的多元高斯分布下,该第一节点将令采集函数取最大值的操作等级确定为该第i数据增强策略中该任一种数据增强操作的操作等级。
在一种可能的实现方式中,该采集函数由下述公式(4)来表示。
α(x)=μ *+βΣ *    (4
其中,β为控制因子。该第一节点将各个历史数据增强策略中该任一种数据增强操作的强度等级以及各个历史数据增强策略的评估值,代入上述公式(4),以获取在配置的该任一种数据增强操作的多个强度等级下该采集函数的各个函数值,由于令采集函数取最大值的x会有更大的概率使得y取最大值,因此把采集函数取最大值的x作为
Figure PCTCN2021103015-appb-000016
能够使得
Figure PCTCN2021103015-appb-000017
取更大值,则该第一节点将令该采集函数取最大值的强度等级确定为该第数据增强策略中该任一种数据增强操作的操作等级。
在一些实施例中,配置的该任一种数据增强操作的多个强度等级的个数比较多,为了快速从配置的该任一种数据增强操作的多个强度等级确定出能够令该采集函数取最大值的强度等级,则该第一节点通过优化算法计算出令采集函数的取最大值的操作等级。可选地,该优化算法为拟牛顿算法。该拟牛顿算法由python包scipy来实现。在一种可能的实现方式中,该第一节点调用scipy.optimize.minimize()函数,由于拟牛顿算法用于求解最小化对应的x值,则第一节点将该各个数据增强策略的评估值以及该各个历 史数据增强策略中该任一种数据增强操作的操作等级,以及-α(x)输入scipy.optimize.minimize()函数,即可求的该第i数据增强策略中该任一种数据增强操作的操作等级。
为了进一步说明本步骤9031所述的过程,参见图12所示的本申请实施例提供的一种数据增强操作的操作等级的分布拟合过程。假设搜索的数据增强策略只有一种随机变量x,x为强度等级,x对应的数据增强策略的评估值是y(也即是f(x)),对于任一种数据增强操作,第一节点先随机取i个强度等级x(图12中的黑色叉叉,也即是各个历史数据增强策略中任一种数据增强操作的操作等级),每个x对应的y都理解为一个x取值的概率分布(图12中的曲线1是这个概率分布预估的均值曲线,灰色区域是以特定均值为基础的概率分布,曲线2是表示评估出的强度等级x与评估值y之间的对应关系的函数曲线),在黑色叉叉处对应的y值由于是确定值,y值的方差会非常小,可见黑色叉叉上下的灰色区域几乎是没有的。
对于图12中的曲线1和灰色区域,假设当取某个强度等级x时,y的取值范围会在一定的高斯分布范围内变化,其中这个高斯分布的均值就是y取值的概率最高的值,比均值点高的y点的取值概率会逐渐降低,比均值低的y点的取值概率也是同样的逐渐降低,也就是每个强度等级x值对应的y的取值都是一个这样的高斯分布,区别在于它们的均值和方差不一样。曲线1是所有强度等级x对应的y的取值分布的均值所组成的曲线,灰色区域是所有强度等级x点对应的y值分布的均值上下的方差距离所覆盖的区域,也即是,不同强度等级x对应的y值会在其对应的灰色区域中出现,而在曲线1上出现的概率最大。而对于黑色叉叉,由于是已经确定了的强度等级x和y值,所以该强度等级x所对应的y值必然在曲线1所示的均值线上,同时由于已经确定其方差会非常小,也就是对应的y取确定值以外的值的概率是非常小的,所以黑色叉叉附近的灰色区域也几乎是没有的。
由于每个强度等级x对应的y的取值都是一个高斯分布,则可以用多元高斯分布对‘什么样的x该取什么样的y值’这个问题进行建模,也就是对于黑色叉叉服从上述公式(1)所述的多元高斯分布,这是基于已知的i个黑色叉叉组成的多元高斯分布。而当要取第i+1个强度等级x(也即是
Figure PCTCN2021103015-appb-000018
)所对应的
Figure PCTCN2021103015-appb-000019
值的分布时,第一节点通过公式(1)所示的多元高斯分布,预估出对应的
Figure PCTCN2021103015-appb-000020
的分布,也即是得到上述公式(2)。那么对于一个新的
Figure PCTCN2021103015-appb-000021
K -1,K **等都是能够计算出。因此,通过上述公式(3),第一节点能够预测出新点
Figure PCTCN2021103015-appb-000022
对应的
Figure PCTCN2021103015-appb-000023
的取值概率分布。
Figure PCTCN2021103015-appb-000024
为上述公式(4)所示的采集函数α(x)取最大值所对应的x值,其中,β为控制因子,用于控制有多大比重考虑
Figure PCTCN2021103015-appb-000025
在方差范围内的取值,当β=0时,第一节点仅取使得
Figure PCTCN2021103015-appb-000026
取值分布中均值最大的x值,而忽略
Figure PCTCN2021103015-appb-000027
在方差范围内取更大值的可能性,当β越大时,代表第一节点越考虑
Figure PCTCN2021103015-appb-000028
在方差范围内取更大值的可能性,后续第一节点能够在确定出的
Figure PCTCN2021103015-appb-000029
的基础上,继续评估求得对应的
Figure PCTCN2021103015-appb-000030
的真实值,已知的x,y值就会又增加一对,这会使得第一节点利用多元高斯分布计算的y值会越来越准,所预估出来的x *就更有可能取得更高的y值。不断迭代,第一节点就能够得到越来越高的y值所对应的x值。而最高的y值所对应的x值就是所希望取得的x,也就是该任一数值增强操作的最优的强度等级。
步骤9032、对于该任一种数据增强操作,该第一节点基于该各个数据增强策略中该 任一种数据增强操作的多个操作概率以及该各个数据增强策略的多个评估值,预测该第数据增强策略中该任一种数据增强操作的操作概率。
该第一节点通过该各个数据增强策略中每种数据增强操作的多个操作概率所服从的高斯分布,拟合出该各个数据增强策略中每种数据增强操作的操作概率所服从的多元高斯分布,以预测出每种数据增强操作新的操作概率。其中,每种数据增强操作新的操作概率也即是第i数据增强策略中每种数据增强操作的操作概率。
在一种可能的实现方式中,本步骤9032由下述步骤B1-B2来实现。
步骤B1、该第一节点基于该多个操作概率以及该多个评估值,确定该任一种数据增强操作的操作概率所服从的多元高斯分布。
本步骤B1所示的过程与上述步骤A1所示的过程同理,其区别在于,本步骤B1是以该任一种数据增强操作的操作概率作为随机变量x,而上述步骤A1是以该任一种数据增强操作的操作强度作为随机变量x,在此,本申请实施例对本步骤B1所示的过程不作赘述。
步骤B2、在该任一种数据增强操作的操作概率所服从的多元高斯分布下,该第一节点将令采集函数取最大值的操作概率确定为该第i数据增强策略中该任一种数据增强操作的操作概率。
本步骤B2所示的过程与上述步骤A2所示的过程同理,其区别在于,本步骤B2是以该任一种数据增强操作的操作概率作为随机变量x,而上述步骤A2是以该任一种数据增强操作的操作强度作为随机变量x,在此,本申请实施例对本步骤B2所示的过程不作赘述。
步骤9033、该第一节点基于预测出的该多种数据增强操作的操作等级以及操作概率,生成该第i数据增强策略。
该第一节点对该多种数据增强操作中的每种数据增强操作均执行上述步骤9031-9032所示的过程,从而该第一节点能够得到预测的该多种数据增强操作的操作等级以及操作概率。当该第一节点预测到该多种数据增强操作的操作等级以及操作概率后,该第一节点将该多种数据增强操作的操作标识、预测的该多种数据增强操作的操作等级以及操作概率,打包成该第i数据增强策略。
当该第一节点确定出该第i数据增强策略后,该第一节点还要再获取第i数据增强策略的评估值。该第一节点采用该第i数据增强策略,对该应用场景下的样本进行数据增强,并基于数据增强后的增强样本进行模型训练,通过训练出的识别模型来评估该第i数据增强策略的好坏。其中,该第一节点对第i数据增强策略进行的评估值的过程由下述步骤C1-C3所述的过程来实现。
步骤C1、该第一节点基于该第i数据增强策略中该多种数据增强操作的操作概率以及操作等级,对多个第二样本进行该多种数据增强操作,得到该多个第二样本的多个增强样本。
本步骤C1所示的过程与步骤9011所示的过程同理,在此,本申请实施例对本步骤C1不作赘述。
步骤C2、该第一节点基于该多个增强样本训练得到第i识别模型。
该第i识别模式也即是基于该第i数据增强策略得到的增强样本所训练出的识别模 型。本步骤C2所示的过程与步骤9022所示的过程同理,在此,本申请实施例对本步骤C2不作赘述。
步骤C3、该第一节点基于该第i识别模型,获取该第i数据增强策略的评估值。
本步骤C3所示的过程与步骤9023所示的过程同理,在此,本申请实施例对本步骤C3不作赘述。
当该第一节点获取到该第i数据增强策略的评估值后,若前i次迭代过程所确定出的数据增强策略满足预设条件,则该第一节点终止迭代计算,否则该第一节点进入第i+1次迭代计算的过程。其中,该预设条件包括下述至少一个:前i次迭代过程所确定出的数据增强策略的总个数达到目标个数;前i次迭代过程所确定出的数据增强策略的平均评估值达到目标评估值。
若前i次迭代过程所确定出的数据增强策略总个数达到目标个数,则说明该第一节点确定出的数据增强策略足够多,若前i次迭代过程所确定出的数据增强策略的平均评估值达到目标评估值,则说明该前i次迭代过程所确定出的数据增强策略足够优质,在数据增强策略已经足够多和/或足够优质的情况下,该第一节点终止迭代计算。
需要说明的是,本步骤903所示的过程为第一节点获取多个数据增强策略的过程。在一些实施例中,本步骤903所示的过程由该第一节点中的策略优化器来执行,例如图2所示的第一节点101中的策略优化器1013。
904、该第一节点基于多个数据增强策略,确定该第一目标数据增强策略,该第一目标数据增强策略包括多种数据增强操作中每种数据增强操作的目标操作概率以及目标操作等级。
其中,该第一目标数据增强策略为该第一节点最终确定出的一个最优的数据增强策略。
在一种可能的实现方式中,该第一节点将该多个数据增强策略中评估值最高的数据增强策略作为该第一目标数据增强策略。由于该多个数据增强策略均是基于配置的多种数据增强操作多个操作等级搜索得到,此时该第一目标数据增强策略为该多个数据增强策略中一个,因此,该第一目标数据增强策略也是基于配置的多种数据增强操作多个操作等级搜索得到。
在另一种可能的实现方式中,该第一节点先从该多个数据增强策略中选取出部分数据增强策略,再基于选取出的这部分数据增强策略,确定出该第一目标数据增强策略。可选地,本步骤904所示的过程由下述步骤9041-9042所示的过程来实现。
步骤9041、该第一节点基于该多个数据增强策略的评估值,从该多个数据增强策略中选择出多个第二目标数据增强策略,该多个第二目标数据增强策略的评估值均高于该多个数据增强策略中除该多个第二目标数据增强策略之外的数据增强策略的评估值。
示例性地,该第一节点对该多个数据增强策略的评估值进行排序,基于排序结果确定该多个数据增强策略中的第二目标数据增强策略。
在一种可能的实现方式中,按照评估值从大到小的顺序,该第一节点对该多个数据增强策略的评估值进行排序,得到第一评估值序列,该第一节点将该第一评估值序列中排序位于前目标位的评估值作为第二目标数据增强策略的评估值,从而该第一节点从该多个数据增强策略中选取出多个第二目标数据增强策略。
在一种可能的实现方式中,按照评估值从小到大的顺序,该第一节点对该多个数据增强策略的评估值进行排序,得到第二评估值序列,该第一节点将该第二评估值序列中排序位于后目标位的评估值作为第二目标数据增强策略的评估值,从而该第一节点从该多个数据增强策略中选取出多个第二目标数据增强策略。
其中,目标位为大于1的整数,本申请实施例对目标位的个数不作限定。
步骤9042、该第一节点基于该多个第二目标数据增强策略,生成该第一目标数据增强策略。
在一种可能的实现方式中,本步骤9042所示的过程由下述步骤D1-D2所示的过程来实现。
步骤D1、该第一节点基于该多个第二目标数据增强策略中该多种数据增强操作的操作概率以及操作等级,确定该多种数据增强操作的目标操作概率以及目标操作等级。
该第一节点基于对任一种数据增强操作的强度等级的聚类结果,来确定该任一种数据增强操作的目标操作概率以及目标操作等级。在一种可能的实现方式中,本步骤D1由下述步骤D11-D12所示的过程来实现。
步骤D11、对于该多种数据增强操作中的任一种数据增强操作,该第一节点对该多个第二目标数据增强策略中该任一种数据增强操作的多个操作等级进行聚类,得到至少一个操作等级类别。
在一种可能的实现方式中,该第一节点基于密度聚类算法,对该多个第二目标数据增强策略中该任一种数据增强操作的多个操作等级进行聚类,得到该至少一个操作等级类别,其中,一个操作等级类别包括该任一种数据增强操作的至少一个操作等级。
可选地,该密度聚类算法为具有噪声的基于密度的聚类方法(density-based spatial clustering of applications with noise,DBSCAN),DBSCAN是一种基于密度的空间聚类算法,能够将具有足够密度的区域划分为簇,并在具有噪声的空间数据库中发现任意形状的簇,簇为密度相连的点(即操作等级)的最大集合,一个操作等级类别中该任一种数据增强操作的操作等级也即是一簇。
例如该多个第二目标数据增强策略中该任一种数据增强操作的强度等级有5个,分别为强度等级1、3、5、6以及7,通过DBSCAN,该第一节点对这5个强度等级进行聚类,得到操作等级类别(即簇)1和2,其中,操作等级类别1包括强度等级1和3,操作等级类别2包括强度等级5、6以及7。
需要说明的是,该第一节点还可以采用除DBSCAN以外的其他类型的聚类算法,对该任一种数据增强操作的多个操作等级进行聚类,在此,本申请实施例对除DBSCAN以外的其他类型的聚类算法不作限定。
步骤D12、对于该至少一个操作等级类别中的任一操作等级类别,该第一节点基于该任一操作等级类别中的操作等级、该多个第二目标数据增强策略中的第三目标数据增强策略内该任一种数据增强操作的操作概率,确定该任一种数据增强操作的目标操作强度以及目标操作概率,该第三目标数据增强策略为该任一操作等级类别的操作等级所属的第二目标数据增强策略。
在一种可能的实现方式中,该第一节点基于该任一操作等级类别中的操作等级,确定该任一种数据增强操作的目标操作等级。
在一些实施例中,该第一节点确定该任一操作等级类别中的最小操作等级以及最大操作等级;并将大于等于该最小操作等级且小于等于该最大操作等级的各个操作等级,确定为该任一种数据增强操作的目标操作等级。以上述的操作等级类别1为例,操作等级类别1中的最小操作等级为操作等级1,最大操作等级为操作等级3,则大于等于该最小操作等级且小于等于该最大操作等级的各个操作等级包括操作等级1、2以及3,该第一节点将该操作等级1-3确定为该任一种数据增强操作的目标操作等级。
在一些实施例中,该第一节点将该任一操作等级类别中的各个操作等级,确定为该任一种数据增强操作的目标操作等级。以上述的操作等级类别1为例,该第一节点将该操作等级类别1中的操作等级1和操作等级3,确定为该任一种数据增强操作的目标操作等级。
在一种可能的实现方式中,该第一节点基于该多个第二目标数据增强策略中的第三目标数据增强策略内该任一种数据增强操作的操作概率,确定该任一种数据增强操作的目标操作概率。
可选地,该第一节点将该多个第二目标数据增强策略中的第三目标数据增强策略内该任一种数据增强操作的操作概率的平均概率,确定为该任一种数据增强操作的目标操作概率。例如若操作等级1包括操作等级1和3,操作等级1所属的数据增强策略中该任一种数据增强操作的操概率为0.6,操作等级3所属的数据增强策略(也即是第三目标数据增强策略)中该任一种数据增强操作的操概率为0.4,当目标操作等级为操作等级1和3时,则该任一种数据增强操作的目标操作概率为0.6和0.4之间的平均概率0.5。
可选地,该目标操作概率与该任一种操作等级类别中操作等级的个数之积为该任一种操作等级类别中各个操作等级对应的操作概率之和,其中,一个操作等级对应的操作概率为该操作等级所属的数据增强策略中该任一种数据增强操作的操作概率。例如,若操作等级1包括操作等级1和3,操作等级1所属的数据增强策略中该任一种数据增强操作的操概率为0.6,操作等级3所属的数据增强策略中该任一种数据增强操作的操概率为0.3,当目标操作等级为操作等级1、2和3,且操作等级2并属于任何数据增强策略时,则默认该操作等级2对应的操作概率为0,则该任一种数据增强操作的目标操作概率=(0.6+0.3)/3=0.3。
步骤D2、该第一节点基于确定出的该多种数据增强操作的目标操作概率以及目标操作等级,生成该第一目标数据增强策略。
示例性地,该第一节点将该多种数据增强操作的操作标识、目标操作概率以及目标操作概率,打包为该第一目标数据增强策略。
需要说明的是,本步骤904所示的过程也即是第一节点获取第一目标数据增强策略的过程。
在一些实施例中,本步骤904所示的过程由该第一节点中的策略优化器来执行,例如图2所示的第一节点101中的最优策略生成器1014。
905、该第一节点基于该第一目标数据增强策略,在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对第一样本进行该多种数据增强操作,得到该第一样本的多个目标增强样本。
该第一样本为在该应用场景下采集的数据所组成的样本。在一些实施例中,该第一 样本的对象与该第二样本的对象属于同一种类,或该第一样本的对象与该第二样本的对象为同一对象。在另外一些实施例中,该第一样本的对象与该第二样本的对象属于不同的种类。本申请实施例以该第一样本的对象与该第二样本的对象属于同一种类为例进行说明,也即是该第一样本的对象属于目标种类。
该第一节点在执行本步骤903之前,还可以获取该第一样本,例如在图6所示的云端场景下,该第一节点通过目标接口获取应用节点上传的该第一样本。当然,在一些实施例中,该第一样本还可以是第一节点本地存储的样本,该第一节点从本地存储的样本中获取该第一样本。
该第一节点基于该第一目标数据增强策略中多种数据增强操作中每种数据增强操作的目标操作概率以及目标操作等级,对该第一样本进行该多种数据增强操作,得到该多个目标增强样本。其中,该多个目标增强样本中任一种数据增强操作的目标操作概率的目标增强样本进行过该任一种数据增强操作。
对于该任一种数据增强操作,该第一节点基于该任一种数据增强操作的目标操作概率,在该任一种数据增强操作的目标操作等级所指示的操作强度变化区间内,对该第一样本进行该任一种数据增强操作,该过程与上述步骤9021中该第一节点基于该任一种数据增强操作的操作概率,在该任一种数据增强操作的操作等级所指示的操作强度变化区间内,对该多个第二样本进行该任一种数据增强操作的过程同理,在此,本申请实施例对该过程不作赘述。
在一些实施例中,本步骤905所示的过程由该第一节点中的策略优化器来执行,例如图2所示的第一节点101中的最优策略生成器1014。
906、该第一节点基于多个目标增强样本训练得到目标识别模型。
该目标识别模型为最终用于对该应用场景下的样本进行识别的模型,当获取到该目标识别模型后,该第一节点将该目标识别模型与该目标种类的类型标识进行关联存储,以便后续将该目标识别模型用于在该目标种类的对象的样本识别中。
本步骤906所示的过程与上述步骤9022中该第一节点基于该多个第二样本的多个增强样本训练得到初始识别模型过程同理,在此本申请实施例对本步骤906不作赘述。在一些实施例中,本步骤906所示的过程由该第一节点中的策略优化器来执行,例如图2所示的第一节点101中的最优策略生成器1014。
需要说明的是,在云端场景下,该第一节点在获取到该第一目标数据增强策略后,该第一节点还可以通过目标接口向应用节点发送该多个目标增强样本,由该应用节点执行本步骤906所示的过程,例如图6所示的云端场景。
本申请实施例提供的数据增强方法,通过数据增强策略中的多种数据增强操作,对样本进行处理,能够得到多个目标增强样本,从而达到了扩充样本的目的,并且由于是在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对样本进行多种数据增强操作,从而得到的多个目标增强样本能够在多种数据增强操作的目标操作等级所指示的操作强度变化区间之内具有变化,因此提高了样本的多样性。
上述图9所示的过程为第一节点具有数据增强系统中各个节点的功能的情况下,由第一节点独自完成数据增强的过程。在另外一些实施例中,数据增强的过程是由数据增强系统中的各个节点协同完成,为了进一步说明该过程,参见图13所示的本申请实施 例提供的一种数据增强方法的流程图。
1301、第一节点获取初始数据增强策略。
本步骤1301所示的过程与上述步骤901所示的过程同理,在此,本申请实施例对本步骤1301不作赘述。
在一些实施例中,本步骤904所示的过程由该第一节点中的策略优化器来执行,例如图2所示的第一节点101中的策略优化器1013。
1302、该第一节点向第二节点发送该初始数据增强策略。
1303、该第二节点对接收到的该初始数据增强策略进行评估,得到该初始数据增强策略的评估值。
本步骤1303所示的过程与上述步骤902所示的过程同理,在此,本申请实施例对本步骤1303不作赘述。
1304、该第二节点向该第一节点发送该初始数据增强策略的评估值。
1305、当i等于1时,在第一次迭代计算的过程中,该第一节点基于该初始数据增强策略,确定该多个数据增强策略中的第一数据增强策略。
本步骤1305所示的过程与上述步骤9031-9033所示的过程同理,在此,本申请实施例对本步骤1305不作赘述。
1306、该第一节点向第二节点发送该第一数据增强策略。
1307、该第二节点对接收到的该第一数据增强策略进行评估,得到该第一数据增强策略的评估值。
本步骤1307所示的过程与上述步骤902所示的过程同理,在此,本申请实施例对本步骤1307不作赘述。在一些实施例中,本步骤904所示的过程由该第一节点中的策略优化器来执行,例如图2所示的第一节点101中的策略优化器1013。
1308、该第二节点向该第一节点发送该第一数据增强策略的评估值。
1309、当i不等于1时,在第i次迭代计算的过程中,该第一节点基于在前i-1次迭代计算过程中所确定出的数据增强策略以及该初始数据增强策略中的各个数据增强策略,确定第i数据增强策略。
本步骤1309所示的过程与上述步骤9031-9033所示的过程同理,在此,本申请实施例对本步骤1309不作赘述。在一些实施例中,本步骤904所示的过程由该第一节点中的策略优化器来执行,例如图2所示的第一节点101中的策略优化器1013。
1310、该第一节点向第二节点发送该第i数据增强策略。
1311、该第二节点对接收到的该第i数据增强策略进行评估,得到该第i数据增强策略的评估值。
本步骤1311所示的过程与上述步骤902所示的过程同理,在此,本申请实施例对本步骤1311不作赘述。
1312、第二节点向第一节点发送该第i数据增强策略的评估值。
对于上述1309-1312所示的过程,在一些实施例中,当第一节点在每次的迭代过程中获取到多个数据增强策略后,该第一节点将该多个数据增强策略发送到不同的第二节点,由不同的第二节点对不同的数据增强策略进行评估,以实现分布式评估。
在一种可能的实现方式中,若第一节点获取的第i数据增强策略有多个,该第一节 点向多个第二节点分别发送至少一个第i数据增强策略,由每个第二节点基于下述步骤1311所示的过程对接收到的每个第i数据增强策略进行评估,以便后续该第一节点从该多个第二节点分别接收至少一个第i数据增强策略的评估值。此时第二节点也即是评估设备,用于对第一节点发送的数据增强策略进行评估。
例如图14所示的本申请实施例提供的一种数据增强系统的示意图,该第一节点每生成一个数据增强策略,就将该数据增强策略发送给第二节点,由第二节点采用接收到的数据增强策略,对从数据服务器获取的样本进行数据增强,得到多个增强样本,并基于多个增强样本训练和测试识别模型,并向第一节点返回该数据增强策略的评估值。
在一些实施例中,该数据增强系统内还设置有调度节点,第二节点每对一个数据增强策略进行评估的过程中,该第二节点将该数据增强策略的评估值发送给调度节点;若该数据增强策略的评估值低于目标评估值,认为这个数据增强策略不优,则该调度节点指示第二节点不向该第一节点返回该数据增强策略的评估值。之后若第一节点从第二节点接收不到该数据增强策略的评估值,则该第一节点舍弃该数据增强策略,以便后续第一节点最终确定出的多个数据增强策略的评估值均大于或等于目标评估值,也即是第一节点最终确定出的多个数据增强策略都比较优。在一些实施例中,在该第二节点接收到一个数据增强策略后,若该调度节点长时间未获取到该数据增强策略的评估值,认为该第二节点在评估该数据增强策略的过程中出现了问题,例如第二节点基于该数据增强策略得到的增强样本长时间训练不出识别模型,则该调度节点指示第二节点结束对该数据增强策略的评估过程,进行下一个数据增强策略的评估过程。例如图14中的调度节点。
1313、当多次迭代过程后所确定出的多个数据增强策略满足预设条件时,该第一节点向第三节点发送该多个数据增强策略。
在一些实施例中,该第一节点不是在获取到该多个数据增强策略后,才向该第三节点发送该多个数据增强策略,而是该第一节点每获取到一个数据增强策略就向该第三节点发送,本申请实施例对该第一节点向该第三节点发送该多个数据增强策略的时机不作限定。
在一些实施例中,该第一节点除了向该第三节点发送该多个数据增强策略以外,还会向该第三节点发送该初始数据增强策略。
1314、该第三节点基于接收到的多个数据增强策略,确定该第一目标数据增强策略。
本步骤1314所示的过程与上述步骤904所示的过程同理,在此,本申请实施例对本步骤1314不作赘述。
1315、该第三节点向应用节点发送该第一目标数据增强策略。
1316、该应用节点基于该第一目标数据增强策略,获取目标识别模型。
在一种可能的实现方式中,该应用节点基于该第一目标数据增强策略,对第一样本进行多种数据增强操作,得到该第一样本的多个目标增强样本,该应用节点基于至少一个第一样本的多个目标增强样本,训练得到目标识别模型。此过程与上述步骤905和906所示的过程同理,在此,本申请实施例对本步骤1316不作赘述。
由于该第一目标数据增强策略中任一种数据增强操作的目标强度等级可能有多个,且每个目标强度等级对应一个强度变化区间,因此,基于该第一目标数据增强策略所得到的增强样本之间会呈现变化分布。例如图15所示的本申请实施例提供的一种数据变 化分布的示意图,在图15中,横轴为旋转操作的各个旋转角度,纵轴为多个增强样本经过不同旋转角度的旋转操作的增强样本的占比,曲线1构成的分布是该应用场景下的真实数据的变化分布,曲线2构成的分布是采用该第一目标数据增强操作所得到增强样本的数据的变化分布,图15中的竖线是采用相关技术提供的数据增强策略所得到的增强样本的数据的变化分布。从图15可知,竖线仅能覆盖该应用场景下的真实数据变化分布的极小一部分,而采用该第一目标数据增强操作所得到增强样本的数据的变化分布能够覆盖真实数据变化分布的更大范围。因此,后续基于该第一目标数据增强策略所生成的增强样本,更能模拟真实数据,且基于该第一目标数据增强策略得到增强样本训练出的识别模型,泛化能力更强,更能适应真实的应用场景。
另外,在图像应用场景下,该第一目标数据增强策略中数据增强操作的变化分布,还可以表现为增强样本的像素值的变化分布。例如数据增强操作为锐化操作,在实施锐化操作时,会提升图像(样本)中对象的轮廓线条相关部位的轮廓两侧的颜色对比度。比如若样本为小鸟的图像,对个图像进行强锐化操作后,以强锐化操作后所得到的图像(增强样本)中鸟嘴附近的像素为例,原图像中鸟嘴附近的黑白对比并不强烈,强锐化操作后所得到的图像中鸟嘴附近的黑白对比会较为强烈,而这些对比可以反映在相关的每个像素的变化上。更进一步,因为像素都是由红绿蓝(red、green、blue,RGB)三元组组成,该三元组中有三个范围从0到256变化的数字组成,分别代表红、绿、蓝三种颜色在该像素位置上的强度,因此,数据增强操作的变化分布可以表现为增强样本的像素值的变化分布。
例如图16所示的本申请实施例提供的一种锐化操作变化时特定像素值变化的分布示意图。以锐化操作为例,在图16中显示了通过本申请的数据增强方法所增强的一张图像中某一特定像素点的数值的变化分布,并显示了相关技术提供的锐化操作所增强的这张图像中该特定像素点的数值的变化分布。横轴是将这张图像数据增强成若干张增强图像后该特定像素点的数值(由于制图原因,为了呈现出对比效果,横轴上的数值均乘100),纵轴是将这张图像数据增强成若干张增强图像后,在该特定像素点上取了该数值的增强图像的个数,也就是该图增强后在该特定像素点上的取值分布(即数据增强的变化分布)。可见,相关技术进行数据增强后只能带来有限的变化,而通过本申请所进行的数据增强带来的数据变化呈现为一个分布,在这个分布上有无限多个变化点。另一方面,由于本申请实施例中在对数据增强策略进行评估的过程中,不同数据增强策略是采用不同对象类型的样本进行评估的,因此,不同对象类型对应的数据增强策略是不一样的。如图16中,鸟和飞机的像素变化的分布是不同的,而相关技术对不同对象类型只能用相同的数据增强策略,所以无论鸟或飞机,相关技术数据增强后的像素变化都是相同的。
需要说明的是,步骤1314-1316所示的过程是第一节点将多个数据增强策略发送给第三节点,由第三节点基于接收到多个数据增强策略,确定第一目标数据增强策略,并发送给应用节点,由应用节点从第三节点获取第一目标数据增强策略。而在另外一些实施例中,第一节点基于多个数据增强策略,确定第一目标数据增强策略,并将确定出的第一目标数据增强策略发送给应用节点。由应用节点基于该第一目标数据增强策略,对从数据服务器器获取到的多个样本进行数据增强,得到增强样本,并基于该多个增强样 本进行模型训练,并将训练出的识别模型部署服务,例如图14所示的数据增强系统中的第一节点以及应用节点。
本申请实施例提供的数据增强方法,通过多个节点协同完成数据增强的过程,由于多个节点并行工作,从而提高数据增强系统进行数据增强的效率。
需要说明的是,采用相关技术中提供的数据增强策略以及本申请提供的数据增强策略,分别对公共数据集CIFAR10NEI中的样本进行数据增强。在训练出的识别模型的准确率的人工基线均为92.1%的情形下,通过相关技术提供的数据增强策略得到的增强样本,所训练出的识别模型的准确率为92.3%,而通过本申请提供的数据增强策略得到的增强样本,所训练出的识别模型的准确率为96.1%,可见,通过本申请提供的数据增强策略所训练出的识别模型的准确率能够提高4%。并且,相关技术中搜索到最终的数据增强策略所使用的搜索时间为5000GPU小时,而本申请在搜索最终的数据增强策略的过程中,进行了分布式评估,且在评估过程中是基于预训练模型进行训练的,本申请搜索到最终的数据增强策略所使用搜索时间为114GPU小时,相比相关技术中的搜索时间节省了50倍。其中,识别模型的准确率和数据增强策略的搜索时间对比,如下表1所示。
表1
对比参数 人工基线 相关技术 本申请
准确率 92.1% 92.3% 96.1%
搜索时间   5000 GPU小时 114 GPU小时
图17是本申请实施例提供的一种数据增强装置的结构示意图,该装置1700包括:
获取模块1701,用于获取第一目标数据增强策略,所述第一目标数据增强策略用于指示数据增强操作的目标操作等级,一个目标操作等级用于指示一个操作强度变化区间;
增强模块1702,用于基于所述第一目标数据增强策略,在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对第一样本进行所述多种数据增强操作,得到所述第一样本的多个目标增强样本。
在一种可能的实现方式中,对于所述第一样本中的任一数据,所述多个目标增强样本中所述任一数据出现任一种数据增强操作对应的变化,在所述任一种数据增强操作的目标强度等级所指示的操作强度变化区间内。
在一种可能的实现方式中,所述第一目标数据增强策略基于配置的所述多种数据增强操作的多个操作等级搜索得到,任一种数据增强操作配置有多个操作等级,一个操作等级用于指示一个强度变化区间。
在一种可能的实现方式中,所述第一目标数据增强策略基于多个数据增强策略确定,所述多个数据增强策略基于配置的所述多种数据增强操作的多个操作等级搜索得到,任一种数据增强操作配置有多个操作等级,一个操作等级用于指示一个强度变化区间。
在一种可能的实现方式中,所述多个数据增强策略由多个评估设备进行评估。
在一种可能的实现方式中,所述多个数据增强策略评估时所采用的第二样本的对象 为同一个对象,或者为同一类型的对象。
在一种可能的实现方式中,所述任一种数据增强操作配置的多个操作等级基于所述任一种数据增强操作的目标操作强度变化范围得到,所述任一种数据增强操作的目标操作强度变化范围由配置设备进行配置。
可选地,所述装置1700还包括:
划分模块,用于对于所述多种数据增强操作中的任一种数据增强操作,将所述任一种数据增强操作对应的目标操作强度变化范围划分为多个操作强度变化区间;
配置模块,用于为所述多个操作强度变化区间中的每个操作强度变化区间配置操作等级。
可选地,所述第一目标数据增强策略还用于指示所述多种数据增强操作中每种数据增强操作的目标操作概率,所述任一种数据增强操作的目标操作概率为对所述第一样本进行所述任一种数据增强操作的概率;
所述获取模块1701用于:
基于多个数据增强策略,确定所述第一目标数据增强策略,任一数据增强策略用于指示所述多种数据增强操作中每种数据增强操作的操作概率以及操作等级,任一种数据增强操作的操作概率为对第二样本进行所述任一种数据增强操作的概率。
可选地,所述获取模块1701包括:
选择子模块,用于基于所述多个数据增强策略的评估值,从所述多个数据增强策略中选择出多个第二目标数据增强策略,所述多个第二目标数据增强策略的评估值均高于所述多个数据增强策略中除所述多个第二目标数据增强策略以外的数据增强策略的评估值,所述任一数据增强策略的评估值用于指示基于所述任一数据增强策略得到的增强样本所训练出的识别模型的优劣程度;
生成子模块,用于基于所述多个第二目标数据增强策略,生成所述第一目标数据增强策略。
可选地,所述生成子模块包括:
确定单元,用于基于所述多个第二目标数据增强策略中所述多种数据增强操作的操作概率以及操作等级,确定所述多种数据增强操作的目标操作概率以及目标操作等级;
生成单元,用于基于确定出的所述多种数据增强操作的目标操作概率以及目标操作等级,生成所述第一目标数据增强策略。
可选地,所述确定单元包括:
聚类子单元,用于对于所述任一种数据增强操作,对所述多个第二目标数据增强策略中所述任一种数据增强操作的多个操作等级进行聚类,得到至少一个操作等级类别;
确定子单元,用于对于所述至少一个操作等级类别中的任一操作等级类别,基于所述任一操作等级类别中的操作等级、所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作强度以及目标操作概率,所述第三目标数据增强策略为所述任一操作等级类别的操作等级所属的第二目标数据增强策略。
可选地,所述确定子单元包括:
第一确定元件,用于基于所述任一操作等级类别中的操作等级,确定所述任一种数 据增强操作的目标操作等级;
第二确定元件,用于基于所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作概率。
可选地,所述第一确定元件用于:
确定所述任一操作等级类别中的最小操作等级以及最大操作等级;
将大于等于所述最小操作等级且小于等于所述最大操作等级的各个操作等级,确定为所述任一种数据增强操作的目标操作等级。
可选地,所述第一确定元件用于:
将所述任一操作等级类别中的各个操作等级,确定为所述任一种数据增强操作的目标操作等级。
可选地,所述任一种数据增强操作的目标操作概率为所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率的平均概率。
所述装置1700还包括:
迭代模块,用于基于初始数据增强策略进行迭代计算,得到所述多个数据增强策略。
可选地,所述迭代模块用于:
在第i次迭代计算的过程中,基于在前i-1次迭代计算过程中所确定出的数据增强策略以及所述初始数据增强策略中的各个数据增强策略,确定所述多个数据增强策略中的第i数据增强策略,其中,所述i为大于等于1或小于等于N的整数,所述i为迭代计算的总次数。
可选地,所述迭代模块包括:
第一预测子模块,用于对于所述任一种数据增强操作,基于所述各个数据增强策略中所述任一种数据增强操作的多个操作等级以及所述各个数据增强策略的多个评估值,预测所述第i数据增强策略中所述任一种数据增强操作的操作等级,任一数据增强策略的评估值用于指示基于所述任一数据增强策略得到的增强样本所训练出的识别模型的优劣程度;
第二预测子模块,用于对于所述任一种数据增强操作,基于所述各个数据增强策略中所述任一种数据增强操作的多个操作概率以及所述各个数据增强策略的多个评估值,预测所述第i数据增强策略中所述任一种数据增强操作的操作概率;
生成子模块,用于基于预测出的所述多种数据增强操作的操作等级以及操作概率,生成所述第i数据增强策略。
可选地,所述第一预测子模块用于:
基于所述多个操作等级以及所述多个评估值,确定所述任一种数据增强操作的操作等级所服从的多元高斯分布;
在所述任一种数据增强操作的操作等级所服从的多元高斯分布下,将令采集函数取最大值的操作等级确定为所述第i数据增强策略中所述任一种数据增强操作的操作等级。
可选地,所述第二预测子模块用于:
基于所述多个操作概率以及所述多个评估值,确定所述任一种数据增强操作的操作 概率所服从的多元高斯分布;
在所述任一种数据增强操作的操作概率所服从的多元高斯分布下,将令采集函数取最大值的操作概率确定为所述第i数据增强策略中所述任一种数据增强操作的操作概率。
所述装置1700还包括:
得到模块,用于基于所述第i数据增强策略中所述多种数据增强操作的操作概率以及操作等级,对多个所述第二样本进行所述多种数据增强操作,得到所述多个第二样本的多个增强样本;
训练模块,用于基于所述多个增强样本训练得到第i识别模型;
目标获取模块,用于基于所述第i识别模型,获取所述第i数据增强策略的评估值。
可选地,所述训练模块用于:
基于所述多个增强样本,对预训练模型进行训练,得到所述第i识别模型,所述预训练模型由不同种类的多个对象的多个第三样本训练得到,所述预训练模型的精确度小于精确度阈值。
可选地,所述第一样本的对象与所述第二样本的对象属于同一种类,或所述第一样本的对象与所述第二样本的对象为同一对象。
可选地,所述装置1700还包括:
第一发送模块,用于向多个评估设备分别发送至少一个第i数据增强操作策略;
第一接收模块,用于从所述多个评估设备分别接收所述至少一个第i数据增强操作策略的评估值。
可选地,所述第一目标数据增强策略还用于指示所述多种数据增强操作中每种数据增强操作的目标操作概率,任一种数据增强操作的目标操作概率为对所述第一样本进行所述任一种数据增强操作的概率;
所述增强模块1702用于:
基于所述多种数据增强操作中每种数据增强操作的目标操作概率以及目标操作等级,对所述第一样本进行所述多种数据增强操作,其中,所述任一种数据增强操作的目标操作概率的目标增强样本进行过所述任一种数据增强操作。
可选地,所述增强模块1702用于:
对于所述任一种数据增强操作,基于所述任一种数据增强操作的目标操作概率,在所述任一种数据增强操作的目标操作等级所指示的操作强度变化区间范围内,对所述第一样本进行所述任一种数据增强操作。
可选地,所述装置1700还包括:
第二接收模块,用于通过目标接口从应用节点获取所述第一样本;
第二发送模块,用于通过所述目标接口向所述应用节点发送所述多个目标增强样本。
上述所有可选技术方案,可以采用任意结合形成本公开的可选实施例,在此不再一一赘述。
需要说明的是:上述实施例提供的数据增强装置在进行数据增强时,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功 能模块完成,即将装置的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。另外,上述实施例提供的数据增强方法实施例属于同一构思,其具体实现过程详见方法实施例,这里不再赘述。
本申请实施例还提供了一种计算机程序产品或计算机程序,该计算机程序产品或计算机程序包括计算机指令,该计算机指令存储在计算机可读存储介质中,计算设备的处理器从计算机可读存储介质读取该计算机指令,处理器执行该计算机指令,使得该计算设备执行上述数据增强方法。
需要说明的是,本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元、模块、芯片及方法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本发明的范围。
所属领域的技术人员可以清楚地了解到,为了描述的方便和简洁,上述描述的系统、装置、模块或者单元的具体工作过程,可以参考数据增强方法实施例中的对应过程,在此不再赘述。
在本申请所提供的几个实施例中,应该理解到,所揭露的系统、装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,装置内的模块或者模块中的单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个模块或单元可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另外,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接,也可以是电的,机械的或其它的形式连接。
所述作为分离部件说明的模块或者单元可以是或者也可以不是物理上分开的,作为模块或单元显示的部件可以是或者也可以不是物理模块或者物理单元,即可以位于一个地方,或者也可以分布到多个计算设备。可以根据实际的需要选择其中的部分或者全部模块或单元来实现本发明实施例方案的目的。
另外,在本发明各个实施例中的各功能模块或者单元可以集成在一个目标处理模块中,也可以是各个模块或者单元单独物理存在,也可以是两个或两个以上模块或者单元集成在一个目标处理模块中。上述集成的模块或者单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成,也可以通过程序指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,上述提到的存储介质可以是只读存储器,磁盘或光盘等。
以上所述仅为本申请的可选实施例,并不用以限制本申请,凡在本申请的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。

Claims (21)

  1. 一种数据增强方法,其特征在于,由第一节点执行,所述方法包括:
    获取第一目标数据增强策略,所述第一目标数据增强策略用于指示数据增强操作的目标操作等级,一个目标操作等级用于指示一个操作强度变化区间;
    基于所述第一目标数据增强策略,在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对第一样本进行所述多种数据增强操作,得到所述第一样本的多个目标增强样本。
  2. 根据权利要求1所述的方法,其特征在于,所述第一目标数据增强策略基于配置的所述多种数据增强操作的多个操作等级搜索得到,任一种数据增强操作配置有多个操作等级,一个操作等级用于指示一个强度变化区间。
  3. 根据权利要求1所述的方法,其特征在于,所述第一目标数据增强策略基于多个数据增强策略确定,所述多个数据增强策略基于配置的所述多种数据增强操作的多个操作等级搜索得到,任一种数据增强操作配置有多个操作等级,一个操作等级用于指示一个强度变化区间。
  4. 根据权利要求3所述的方法,其特征在于,所述多个数据增强策略由多个评估设备进行评估。
  5. 根据权利要求4所述的方法,其特征在于,所述多个数据增强策略评估时所采用的第二样本的对象为同一个对象,或者为同一类型的对象。
  6. 根据权利要求2-5任一项权利要求所述的方法,其特征在于,所述任一种数据增强操作配置的多个操作等级基于所述任一种数据增强操作的目标操作强度变化范围得到,所述任一种数据增强操作的目标操作强度变化范围由配置设备进行配置。
  7. 根据权利要求6所述的方法,其特征在于,所述任一种数据增强操作的配置过程包括:
    将所述任一种数据增强操作的目标操作强度变化范围划分为多个操作强度变化区间;
    为所述多个操作强度变化区间中的每个操作强度变化区间配置操作等级。
  8. 根据权利要求3所述的方法,其特征在于,所述第一目标数据增强策略还用于指示所述多种数据增强操作中每种数据增强操作的目标操作概率,所述任一种数据增强操作的目标操作概率为对所述第一样本进行所述任一种数据增强操作的概率;
    所述多个数据增强策略中的任一数据增强策略用于指示所述多种数据增强操作中每种数据增强操作的操作概率以及操作等级,任一种数据增强操作的操作概率为对第二样本进行所述任一种数据增强操作的概率。
  9. 根据权利要求8所述的方法,其特征在于,基于所述多个数据增强策略,确定所述第一目标数据增强策略包括:
    基于所述多个数据增强策略的评估值,从所述多个数据增强策略中选择出多个第二目标数据增强策略,所述多个第二目标数据增强策略的评估值均高于所述多个数据增强策略中除所述多个第二目标数据增强策略以外的数据增强策略的评估值,所述任一数据增强策略的评估值用于指示基于所述任一数据增强策略得到的增强样本所训练出的识别模型的优劣程度;
    基于所述多个第二目标数据增强策略,生成所述第一目标数据增强策略。
  10. 根据权利要求9所述的方法,其特征在于,所述基于所述多个第二目标数据增强策略,生成所述第一目标数据增强策略包括:
    基于所述多个第二目标数据增强策略中所述多种数据增强操作的操作概率以及操作等级,确定所述多种数据增强操作的目标操作概率以及目标操作等级;
    基于确定出的所述多种数据增强操作的目标操作概率以及目标操作等级,生成所述第一目标数据增强策略。
  11. 根据权利要求10所述的方法,其特征在于,所述基于所述多个第二目标数据增强策略中所述多种数据增强操作的操作概率以及操作等级,确定所述多种数据增强操作的目标操作概率以及目标操作等级包括:
    对于所述任一种数据增强操作,对所述多个第二目标数据增强策略中所述任一种数据增强操作的多个操作等级进行聚类,得到至少一个操作等级类别;
    对于所述至少一个操作等级类别中的任一操作等级类别,基于所述任一操作等级类别中的操作等级、所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作强度以及目标操作概率,所述第三目标数据增强策略为所述任一操作等级类别的操作等级所属的第二目标数据增强策略。
  12. 根据权利要求11所述的方法,其特征在于,所述基于所述任一操作等级类别中的操作等级、所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作强度以及目标操作概率包括:
    基于所述任一操作等级类别中的操作等级,确定所述任一种数据增强操作的目标操作等级;
    基于所述多个第二目标数据增强策略中的第三目标数据增强策略内所述任一种数据增强操作的操作概率,确定所述任一种数据增强操作的目标操作概率。
  13. 根据权利要求8-12任一项权利要求所述的方法,其特征在于,所述基于所述多个数据增强策略,确定所述第一目标数据增强策略之前,所述方法还包括:
    基于初始数据增强策略进行迭代计算,得到所述多个数据增强策略。
  14. 根据权利要求13所述的方法,其特征在于,所述基于初始数据增强策略进行迭代计算包括:
    在第i次迭代计算的过程中,基于在前i-1次迭代计算过程中所确定出的数据增强策略以及所述初始数据增强策略中的各个数据增强策略,确定所述多个数据增强策略中的第i数据增强策略,其中,所述i为大于等于1或小于等于N的整数,所述i为迭代计算的总次数。
  15. 根据权利要求14任一项权利要求所述的方法,其特征在于,所述确定所述多个数据增强策略中的第i数据增强策略之后,所述方法还包括:
    基于所述第i数据增强策略中所述多种数据增强操作的操作概率以及操作等级,对多个所述第二样本进行所述多种数据增强操作,得到所述多个第二样本的多个增强样本;
    基于所述多个增强样本训练得到第i识别模型;
    基于所述第i识别模型,获取所述第i数据增强策略的评估值。
  16. 根据权利要求15所述的方法,其特征在于,所述基于所述多个增强样本训练得到第i识别模型包括:
    基于所述多个增强样本,对预训练模型进行训练,得到所述第i识别模型,所述预训练模型由不同种类的多个对象的多个第三样本训练得到,所述预训练模型的精确度小于精确度阈值。
  17. 根据权利要求1-16任一项权利要求所述的方法,其特征在于,所述第一目标数据增强策略还用于指示所述多种数据增强操作中每种数据增强操作的目标操作概率,任一种数据增强操作的目标操作概率为对所述第一样本进行所述任一种数据增强操作的概率;
    所述基于所述第一目标数据增强策略,在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对第一样本进行所述多种数据增强操作包括:
    基于所述多种数据增强操作中每种数据增强操作的目标操作概率以及目标操作等级,对所述第一样本进行所述多种数据增强操作,其中,所述任一种数据增强操作的目标操作概率的目标增强样本进行过所述任一种数据增强操作。
  18. 根据权利要求1-17任一项权利要求所述的方法,其特征在于,所述方法还包括:
    通过目标接口从应用节点获取所述第一样本;
    通过所述目标接口向所述应用节点发送所述多个目标增强样本。
  19. 一种数据增强装置,其特征在于,所述装置包括:
    获取模块,用于获取第一目标数据增强策略,所述第一目标数据增强策略用于指示数据增强操作的目标操作等级,一个目标操作等级用于指示一个操作强度变化区间;
    增强模块,用于基于所述第一目标数据增强策略,在多种数据增强操作的目标操作等级所指示的操作强度变化区间内,对第一样本进行所述多种数据增强操作,得到所述第一样本的多个目标增强样本。
  20. 一种计算设备,其特征在于,所述计算设备包括处理器,所述处理器用于执行程序代码,使得所述计算设备执行如权利要求1至权利要求18中任一项所述的方法。
  21. 一种计算机可读存储介质,其特征在于,所述存储介质中存储有至少一条程序代码,所述至少一条程序代码由处理器读取以使计算设备执行如权利要求1至权利要求18中任一项所述的方法。
PCT/CN2021/103015 2020-11-09 2021-06-29 数据增强方法、装置、计算设备以及计算机可读存储介质 WO2022095476A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21888178.7A EP4235525A4 (en) 2020-11-09 2021-06-29 METHOD AND APPARATUS FOR DATA ENHANCEMENT, COMPUTER DEVICE AND COMPUTER-READABLE STORAGE MEDIUM
US18/313,869 US20230273914A1 (en) 2020-11-09 2023-05-08 Data augmentation method and apparatus, computing device, and computer-readable storage medium

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202011237954.6 2020-11-09
CN202011237954 2020-11-09
CN202011606784.4A CN114462628A (zh) 2020-11-09 2020-12-30 数据增强方法、装置、计算设备以及计算机可读存储介质
CN202011606784.4 2020-12-30

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/313,869 Continuation US20230273914A1 (en) 2020-11-09 2023-05-08 Data augmentation method and apparatus, computing device, and computer-readable storage medium

Publications (1)

Publication Number Publication Date
WO2022095476A1 true WO2022095476A1 (zh) 2022-05-12

Family

ID=81404373

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/103015 WO2022095476A1 (zh) 2020-11-09 2021-06-29 数据增强方法、装置、计算设备以及计算机可读存储介质

Country Status (4)

Country Link
US (1) US20230273914A1 (zh)
EP (1) EP4235525A4 (zh)
CN (1) CN114462628A (zh)
WO (1) WO2022095476A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116503737B (zh) * 2023-05-10 2024-01-09 中国人民解放军61646部队 基于空间光学图像的船舶检测方法和装置
CN117424813B (zh) * 2023-12-11 2024-03-22 北京云成金融信息服务有限公司 一种用于区块链的节点扩充方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190122120A1 (en) * 2017-10-20 2019-04-25 Dalei Wu Self-training method and system for semi-supervised learning with generative adversarial networks
CN110889457A (zh) * 2019-12-03 2020-03-17 深圳奇迹智慧网络有限公司 样本图像分类训练方法、装置、计算机设备和存储介质
CN111611790A (zh) * 2020-04-13 2020-09-01 华为技术有限公司 数据处理的方法与装置
CN111626342A (zh) * 2020-05-13 2020-09-04 Oppo广东移动通信有限公司 一种图像样本处理方法、装置及存储介质
CN111695624A (zh) * 2020-06-09 2020-09-22 北京市商汤科技开发有限公司 数据增强策略的更新方法、装置、设备及存储介质
CN111754494A (zh) * 2020-06-28 2020-10-09 深圳壹账通智能科技有限公司 小样本图像扩充方法、电子设备及存储介质
CN111832511A (zh) * 2020-07-21 2020-10-27 中国石油大学(华东) 一种增强样本数据的无监督行人重识别方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190122120A1 (en) * 2017-10-20 2019-04-25 Dalei Wu Self-training method and system for semi-supervised learning with generative adversarial networks
CN110889457A (zh) * 2019-12-03 2020-03-17 深圳奇迹智慧网络有限公司 样本图像分类训练方法、装置、计算机设备和存储介质
CN111611790A (zh) * 2020-04-13 2020-09-01 华为技术有限公司 数据处理的方法与装置
CN111626342A (zh) * 2020-05-13 2020-09-04 Oppo广东移动通信有限公司 一种图像样本处理方法、装置及存储介质
CN111695624A (zh) * 2020-06-09 2020-09-22 北京市商汤科技开发有限公司 数据增强策略的更新方法、装置、设备及存储介质
CN111754494A (zh) * 2020-06-28 2020-10-09 深圳壹账通智能科技有限公司 小样本图像扩充方法、电子设备及存储介质
CN111832511A (zh) * 2020-07-21 2020-10-27 中国石油大学(华东) 一种增强样本数据的无监督行人重识别方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4235525A4

Also Published As

Publication number Publication date
EP4235525A1 (en) 2023-08-30
US20230273914A1 (en) 2023-08-31
EP4235525A4 (en) 2024-04-24
CN114462628A (zh) 2022-05-10

Similar Documents

Publication Publication Date Title
WO2019100723A1 (zh) 训练多标签分类模型的方法和装置
CN111523621B (zh) 图像识别方法、装置、计算机设备和存储介质
WO2019100724A1 (zh) 训练多标签分类模型的方法和装置
Kao et al. Visual aesthetic quality assessment with a regression model
CN111738357B (zh) 垃圾图片的识别方法、装置及设备
WO2022095476A1 (zh) 数据增强方法、装置、计算设备以及计算机可读存储介质
WO2021027157A1 (zh) 基于图片识别的车险理赔识别方法、装置、计算机设备及存储介质
CN113095370A (zh) 图像识别方法、装置、电子设备及存储介质
US20210089825A1 (en) Systems and methods for cleaning data
CN113283368B (zh) 一种模型训练方法、人脸属性分析方法、装置及介质
CN110610143A (zh) 多任务联合训练的人群计数网络方法、系统、介质及终端
CN111291773A (zh) 特征识别的方法及装置
CN113743426A (zh) 一种训练方法、装置、设备以及计算机可读存储介质
Huo et al. Semisupervised learning based on a novel iterative optimization model for saliency detection
CN112884648A (zh) 多类模糊图像超分辨率重建的方法和系统
US11495020B2 (en) Systems and methods for stream recognition
Goyal et al. Leaf Bagging: A novel meta heuristic optimization based framework for leaf identification
CN113902944A (zh) 模型的训练及场景识别方法、装置、设备及介质
WO2024040941A1 (zh) 神经网络结构搜索方法、装置及存储介质
García-Lamont et al. Color image segmentation using saturated RGB colors and decoupling the intensity from the hue
CN113012030A (zh) 图像拼接方法、装置及设备
CN111414922B (zh) 特征提取方法、图像处理方法、模型训练方法及装置
CN111091198B (zh) 一种数据处理方法及装置
CN114298173A (zh) 一种数据处理方法、装置及设备
Ong et al. A flower recognition system using deep neural network coupled with visual geometry group 19 architecture

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21888178

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021888178

Country of ref document: EP

Effective date: 20230522

NENP Non-entry into the national phase

Ref country code: DE