CN111259939B - Tuning management method, device, equipment and medium for deep learning model - Google Patents
Tuning management method, device, equipment and medium for deep learning model Download PDFInfo
- Publication number
- CN111259939B CN111259939B CN202010024479.8A CN202010024479A CN111259939B CN 111259939 B CN111259939 B CN 111259939B CN 202010024479 A CN202010024479 A CN 202010024479A CN 111259939 B CN111259939 B CN 111259939B
- Authority
- CN
- China
- Prior art keywords
- data
- prediction
- deep learning
- learning model
- prediction data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013136 deep learning model Methods 0.000 title claims abstract description 83
- 238000007726 management method Methods 0.000 title claims abstract description 40
- 238000001914 filtration Methods 0.000 claims abstract description 21
- 230000004044 response Effects 0.000 claims abstract description 21
- 238000000034 method Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 abstract description 9
- 238000005457 optimization Methods 0.000 abstract description 5
- 238000012549 training Methods 0.000 description 8
- 241000282414 Homo sapiens Species 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000012937 correction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Development Economics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Game Theory and Decision Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an optimization management method of a deep learning model, which comprises the following steps: in response to the fact that the number of pictures in a data set needing to be predicted is larger than a set number threshold, averagely dividing the data set into a plurality of sub data sets, and predicting each sub data set through a deep learning model to obtain first prediction data; obtaining judgment data of each subdata set through manual prediction judgment; performing Kalman filtering on the first prediction data and the judgment data of each subdata set to obtain second prediction data of each subdata set; obtaining the error rate of the deep learning model according to the first prediction data and the second prediction data; in response to the error rate exceeding the error rate threshold, triggering the deep learning model to retrain. The invention also discloses a device, equipment and a medium. The principle of the invention is easy to understand, easy to operate and implement, the precision can be ensured, and the function of automatic retraining and tuning of the model is optimized.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to a method, an apparatus, a device, and a medium for tuning and managing a deep learning model.
Background
At present, retraining management of a prediction model can be divided into the following two types: firstly, comparing a model prediction result with an artificial labeling result, accumulating a prediction error result which is artificially identified, namely the number of pictures, and when the prediction error number reaches a threshold value, using all positive and negative sample sets accumulated by the previous prediction as a training set to retrain a model (for example, the threshold value is 200 prediction error pictures, 100 prediction errors are performed on 1000 pictures for the first time, 150 prediction errors are performed on 2000 pictures for the second time, and then 3000 pictures are used for training an optimization model after the second prediction is finished); secondly, the accuracy of the prediction is obtained by comparing the result of the artificial identification during each batch prediction, and if the accuracy is lower than a threshold, the training is carried out again (for example, the threshold is 0.9, 50 pictures are not predicted for the first 1000 pictures, 150 pictures are not predicted for the second 1000 pictures, and the previous 2000 pictures are used for training the tuning model after the second time is finished).
For the management of the retraining model, the retraining period and the triggering condition are important, the retraining times are too many, which indicates that the model is difficult to meet the accuracy requirement of the manager, the retraining takes time and calculation power, the retraining times are too few, the model is not updated for a long time, the prediction accuracy is not high for the new condition that the model is not learned, and the triggering of the retraining is affected when the predicted total number of the prediction is too large or too small each time.
Disclosure of Invention
In view of this, an object of the embodiments of the present invention is to provide a tuning management method, apparatus, device and medium for a deep learning model, which are mainly used for optimizing a triggering situation of model retraining tuning.
Based on the above object, in one aspect, the present invention provides an optimization management method for a deep learning model, where the method includes: in response to the fact that the number of pictures in a data set needing to be predicted is larger than a set number threshold, averagely dividing the data set into a plurality of sub data sets, and predicting each sub data set through a deep learning model to obtain first prediction data; obtaining judgment data of each subdata set through manual prediction judgment; performing Kalman filtering on the first prediction data and the judgment data of each sub data set to obtain second prediction data of each sub data set; obtaining the error rate of the deep learning model according to the first prediction data and the second prediction data; in response to the error rate exceeding the error rate threshold, triggering the deep learning model to retrain.
In some embodiments of the tuning management method of the deep learning model of the present invention, the method further comprises: in response to the fact that the number of pictures of the data set needing to be predicted is not larger than a set number threshold, predicting the data set through a deep learning model to obtain first prediction data, judging through manual prediction to obtain judgment data of the data set, and performing accumulated counting on the data with the first prediction data inconsistent with the judgment data; triggering retraining of the deep learning model in response to an accumulated value of the accumulated count exceeding an accumulated threshold.
In some embodiments of the tuning management method for a deep learning model according to the present invention, the first prediction data, the judgment data, and the second prediction data each use a target sample set as a judgment condition, and the target sample set includes a positive example sample set or a negative example sample set.
In some embodiments of the tuning management method for a deep learning model according to the present invention, the first prediction data includes prediction data of the number of pictures and a prediction accuracy, and the determination data includes determination data of the number of pictures and a determination accuracy.
In some embodiments of the tuning management method for a deep learning model of the present invention, obtaining the error rate of the deep learning model according to the first prediction data and the second prediction data further includes: and dividing the difference between the first prediction data and the second prediction data by the number of pictures of the subdata set to obtain the error rate of the deep learning model.
In another aspect of the embodiments of the present invention, there is also provided an tuning management apparatus for a deep learning model, the apparatus including: the large data set model prediction module is configured to respond that the number of pictures in the data set needing to be predicted is larger than a set number threshold value, averagely divide the data set into a plurality of sub data sets, and predict each sub data set through a deep learning model to obtain first prediction data; the artificial prediction module is configured to obtain judgment data of each subdata set through artificial prediction judgment; a Kalman filtering module configured to perform Kalman filtering on the first prediction data and the judgment data of each sub data set to obtain second prediction data of each sub data set; the error rate calculation module is configured to obtain the error rate of the deep learning model according to the first prediction data and the second prediction data; a retraining determination module configured to trigger retraining of the deep learning model in response to the error rate exceeding an error rate threshold.
In some embodiments of the tuning management apparatus for a deep learning model of the present invention, the apparatus further includes a small data set retraining determination module, and the small data set retraining determination module is configured to: in response to the fact that the number of pictures of the data set needing to be predicted is not larger than a set number threshold, predicting the data set through a deep learning model to obtain first prediction data, judging through manual prediction to obtain judgment data of the data set, and performing accumulated counting on the data with the first prediction data inconsistent with the judgment data; triggering retraining of the deep learning model in response to an accumulated value of the accumulated count exceeding an accumulated threshold.
In some embodiments of the tuning management device for a deep learning model according to the present invention, the first prediction data includes prediction data of the number of pictures and a prediction accuracy, and the determination data includes determination data of the number of pictures and a determination accuracy.
In another aspect of the embodiments of the present invention, there is also provided a computer device, including: at least one processor; and the memory stores a computer program which can run on the processor, and the processor executes the tuning management method of the deep learning model when executing the program.
In another aspect of the embodiments of the present invention, a computer-readable storage medium is further provided, where a computer program is stored, where the computer program is executed by a processor to perform the foregoing tuning management method for a deep learning model.
The invention has at least the following beneficial technical effects: according to the method, through data set classification and a Kalman filtering algorithm, a model retraining and tuning triggering mechanism of the deep learning platform is optimized, and the management function of the platform for automatically retraining the deep learning model is realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
Fig. 1 shows a schematic diagram of an embodiment of a tuning management method of a deep learning model according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it is understood that "first" and "second" are only used for convenience of description and should not be construed as limiting the embodiments of the present invention, and the descriptions thereof in the following embodiments are omitted.
In view of the above, in a first aspect of the embodiments of the present invention, an embodiment of a tuning management method for a deep learning model is provided. Fig. 1 is a schematic diagram illustrating an embodiment of a tuning management method of a deep learning model according to the present invention. In the embodiment shown in fig. 1, the method comprises at least the following steps:
s100, responding to the situation that the number of pictures in a data set needing to be predicted is larger than a set number threshold, averagely dividing the data set into a plurality of sub data sets, and predicting each sub data set through a deep learning model to obtain first prediction data;
s200, obtaining judgment data of each subdata set through artificial prediction judgment;
s300, performing Kalman filtering on the first prediction data and the judgment data of each sub data set to obtain second prediction data of each sub data set;
s400, obtaining the error rate of the deep learning model according to the first prediction data and the second prediction data;
and S500, responding to the error rate exceeding the error rate threshold value, and triggering the deep learning model to retrain.
In the concept of the invention, the retraining frequency, the equipment and calculation power used by retraining, the error of manual marking, the size of the data set predicted by the primary model and other factors are considered, and Kalman filtering is combined to realize the automatic classification and accumulation of the prediction set and the correction of the model prediction accuracy. The Kalman filtering is a combination of prediction and measurement, the prediction is from an empirical model and is obtained by calculation of modeling of a system by a person, and the other part is measurement correction and is correction of the model. Simply speaking, the prediction error filtering is adopted, so that the process measurement value information cannot be filtered, and the prediction value is continuously corrected through the measurement value, so that the dynamic optimal prediction value, namely the second prediction data, is obtained. In some embodiments of the invention, batch prediction is adopted for a big data set, a true optimal estimation value, namely second prediction data, is obtained through prediction data of a model and judgment data of artificial estimation by using Kalman filtering, and the error rate of the optimal estimation of the model is calculated; triggering corresponding retraining tuning when the error rate reaches a threshold value; and after retraining, replacing the shelf, performing version management and the like.
According to some embodiments of the tuning management method of the deep learning model of the present invention, the method further comprises: in response to the fact that the number of pictures of the data set needing to be predicted is not larger than a set number threshold, predicting the data set through a deep learning model to obtain first prediction data, judging through manual prediction to obtain judgment data of the data set, and performing accumulated counting on the data with the first prediction data inconsistent with the judgment data; triggering retraining of the deep learning model in response to an accumulated value of the accumulated count exceeding an accumulated threshold.
In some embodiments of the invention, the model retraining condition of the whole platform is divided into a large prediction set and a small prediction set by a threshold, the large prediction set is described as above, the prediction set is divided into a plurality of equal parts by combining the existing Kalman filtering algorithm, and more accurate model prediction accuracy is calculated, so that whether the existing model is trained and optimized again is determined, and the management function of the automatic optimization deep learning model of the platform is realized by using a prediction error result accumulation mode aiming at the small prediction set.
In some embodiments of the present invention, the configuration counts the number of pictures of the data set that need to be predicted, and if a value (i.e. a number threshold) is not exceeded, the number is recorded as small data set a, and in some embodiments the number threshold is 1000. If the small data set is the small data set, setting the number of pictures of which the small data set model fails to predict (namely, the pictures do not accord with the manual recognition result) as a variable a, and storing all the pictures of the small data set A in a folder as an optimized training set T1; the variable a is summed for each prediction, and retraining and tuning are performed using the training set T1 if a exceeds a threshold (e.g., 500 sheets).
According to some embodiments of the tuning management method for the deep learning model of the present invention, the first prediction data, the judgment data, and the second prediction data all use a target sample set as a judgment condition, and the target sample set includes a positive example sample set or a negative example sample set.
And using Kalman filtering to obtain second prediction data of the real target sample, and calculating the error rate of the optimal estimation of the model. The target sample may be a positive example sample set, that is, the first prediction data, the judgment data and the second prediction data all use positive examples as prediction and judgment conditions, and count, predict and judge the positive examples of data. Conversely, the target sample may be a reverse sample set, that is, the first prediction data, the judgment data, and the second prediction data all use a reverse example as a prediction and judgment condition, and count, predict, and judge the data of the reverse example. In an embodiment of the present invention, the target sample set is a positive example sample set, that is, in some embodiments of the present invention, the positive example data is counted, predicted and judged.
According to some embodiments of the tuning management method for the deep learning model of the present invention, the first prediction data includes prediction data of the number of pictures and a prediction accuracy, and the determination data includes determination data of the number of pictures and a determination accuracy.
In some embodiments of the present invention, an optimal estimation value (i.e., second prediction data) is obtained by using kalman filtering according to the prediction data of the model and the judgment data of the artificial estimation, and combining the model accuracy and the artificial identification accuracy, and an optimal estimation error rate of the model is calculated. In some embodiments of the present invention, the number of the data set pictures to be predicted is counted, and exceeds a value (i.e., a number threshold), and in some embodiments of the present invention, the number threshold is 1000, and the number is denoted as a large data set B. If the large data set is a large data set, setting pictures of the large data set B as a training set T2, averagely dividing the large data set B into a plurality of parts, 10 parts in some embodiments of the invention, j parts in each part, predicting according to batches to obtain the predicted number x of each batch of the deep learning model, wherein the accuracy in the introduction of the deep learning model is r; due to factors such as subjective consciousness of human eyes and human beings and the like, human beings have partial deviation on image identification, so that the number y of positive examples of a human inspection result is obtained, and the accuracy rate of the human identification is s; taking the batch as an axis, substituting the predicted values x and r of the model and the artificial estimated values y and s into Kalman filtering to obtain the optimal estimated value k of the batch, and substituting the Kalman gain into other subsequent data sets for continuous use; obtaining the error rate t of the deep learning model according to the first prediction data and the second prediction data; if the error rate T exceeds the threshold, retraining tuning is triggered and the training set T2 is used.
According to some embodiments of the tuning management method of the deep learning model of the present invention, the step S400 of obtaining the error rate of the deep learning model according to the first prediction data and the second prediction data further includes: and dividing the difference between the first prediction data and the second prediction data by the number of pictures of the subdata set to obtain the error rate of the deep learning model.
In some embodiments of the present invention, the difference between the first prediction data and the optimal estimation value (i.e. the second prediction data) is divided by the number of pictures in the batch to obtain the optimal estimated model error rate. For large data set B, for example: and B, 10000 pictures exist, the accuracy of the known model is 96%, the positive case number of the prediction is 1000, and the positive case number is 800 after artificial inspection. If manual identification is the correct standard without using the kalman filtering idea, the error rate is (1000-800)/10000-0.02. Using the Kalman filtering concept, manual identification can also be mistaken, assuming that the accuracy of manual identification is 80% (naked eyes are not more accurate than machines, people generally believe the judgment of experience more, but still have the possibility of medical misdiagnosis or other misjudgment), substituting into the formula: (1-0.96/(0.96+ 0.8)). times.1000 +0.96/(0.96+ 0.8). times.800. apprxeq.891. The number of positive examples for obtaining the optimal estimate is 891, and the error rate is (1000-. And as the times are increased, the Kalman gain of 0.96/(0.96+0.8) is continuously optimized.
In another aspect of the embodiments of the present invention, an embodiment of an optimization management apparatus for a deep learning model is provided. The device includes: the large data set model prediction module is configured to respond that the number of pictures in the data set needing to be predicted is larger than a set number threshold value, averagely divide the data set into a plurality of sub data sets, and predict each sub data set through a deep learning model to obtain first prediction data; the artificial prediction module is configured to obtain judgment data of each subdata set through artificial prediction judgment; a Kalman filtering module configured to perform Kalman filtering on the first prediction data and the judgment data of each sub data set to obtain second prediction data of each sub data set; the error rate calculation module is configured to obtain the error rate of the deep learning model according to the first prediction data and the second prediction data; a retraining determination module configured to trigger retraining of the deep learning model in response to the error rate exceeding an error rate threshold.
According to some embodiments of the tuning management apparatus for a deep learning model of the present invention, the apparatus further comprises a small data set retraining determination module, the small data set retraining determination module is configured to: in response to the fact that the number of pictures of the data set needing to be predicted is not larger than a set number threshold, predicting the data set through a deep learning model to obtain first prediction data, judging through manual prediction to obtain judgment data of the data set, and performing accumulated counting on the data with the first prediction data inconsistent with the judgment data; triggering retraining of the deep learning model in response to an accumulated value of the accumulated count exceeding an accumulated threshold.
According to some embodiments of the tuning management device for a deep learning model of the present invention, the first prediction data includes prediction data of the number of pictures and a prediction accuracy, and the judgment data includes judgment data of the number of pictures and a judgment accuracy.
In view of the above object, another aspect of the embodiments of the present invention further provides a computer device, including: at least one processor; and the memory stores a computer program which can run on the processor, and the processor executes the tuning management method of the deep learning model when executing the program.
In another aspect of the embodiments of the present invention, a computer-readable storage medium is further provided, where a computer program is stored, and the computer program is executed by a processor to perform the foregoing tuning management method for a deep learning model.
As such, those skilled in the art will appreciate that all of the embodiments, features and advantages set forth above with respect to the tuning management method for a deep learning model according to the present invention apply equally well to apparatus, computer devices and media according to the present invention. For the sake of brevity of the present disclosure, no repeated explanation is provided herein.
It should be particularly noted that, the steps in the foregoing methods, apparatuses, devices and media for tuning management of deep learning models may be mutually intersected, replaced, added or deleted, and therefore, these methods, apparatuses, devices and media for tuning management of deep learning models, which are transformed by reasonable permutation and combination, should also belong to the scope of the present invention, and should not limit the scope of the present invention to the embodiments.
Finally, it should be noted that, as one of ordinary skill in the art can appreciate that all or part of the processes of the methods of the above embodiments can be implemented by a computer program to instruct related hardware, and the program of the tuning management method for deep learning model can be stored in a computer readable storage medium, and when executed, the program can include the processes of the embodiments of the methods described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.
Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.
Further, it should be appreciated that the computer-readable storage media (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of example, and not limitation, nonvolatile memory can include Read Only Memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which can act as external cache memory. By way of example and not limitation, RAM is available in a variety of forms such as synchronous RAM (DRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to comprise, without being limited to, these and other suitable types of memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein may be implemented or performed with the following components designed to perform the functions herein: a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and/or any other such configuration.
The steps of a method or algorithm described in connection with the disclosure herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements and the like that may be made without departing from the spirit or scope of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (7)
1. A tuning management method for a deep learning model is characterized by comprising the following steps:
responding to the situation that the number of pictures in a data set needing to be predicted is larger than a set number threshold value, averagely dividing the data set into a plurality of sub data sets, and predicting each sub data set through a deep learning model to obtain first prediction data;
judging data of each subdata set is obtained through manual prediction and judgment;
performing Kalman filtering on the first prediction data and the judgment data of each of the sub data sets to obtain second prediction data of each of the sub data sets;
obtaining the error rate of the deep learning model according to the first prediction data and the second prediction data;
the obtaining the error rate of the deep learning model according to the first prediction data and the second prediction data further includes:
dividing the difference between the first prediction data and the second prediction data by the number of pictures of the subdata set to obtain the error rate of the deep learning model;
triggering retraining of the deep learning model in response to the error rate exceeding an error rate threshold;
in response to the fact that the number of pictures of the data set needing to be predicted is not larger than the set number threshold, predicting the data set through a deep learning model to obtain first prediction data, judging data of the data set through manual prediction judgment, and performing accumulated counting on the data of the first prediction data, which are inconsistent with the judgment data; triggering retraining of the deep learning model in response to an accumulated value of an accumulated count exceeding an accumulated threshold.
2. The tuning management method for the deep learning model according to claim 1, wherein the first prediction data, the judgment data, and the second prediction data each have a target sample set as a judgment condition, and the target sample set includes a positive example sample set or a negative example sample set.
3. The tuning management method for the deep learning model according to claim 1, wherein the first prediction data includes prediction data of the number of pictures and a prediction accuracy, and the determination data includes determination data of the number of pictures and a determination accuracy.
4. An apparatus for tuning management of a deep learning model, the apparatus comprising:
the big data set model prediction module is configured to respond that the number of pictures in a data set needing prediction is larger than a set number threshold value, averagely divide the data set into a plurality of sub data sets, and predict each sub data set through a deep learning model to obtain first prediction data;
the artificial prediction module is configured to obtain judgment data of each subdata set through artificial prediction judgment;
a Kalman filtering module configured to perform Kalman filtering on the first prediction data and the judgment data of each of the sub data sets to obtain second prediction data of each of the sub data sets;
an error rate calculation module configured to derive an error rate of the deep learning model from the first prediction data and the second prediction data;
a retraining determination module configured to trigger retraining of the deep learning model in response to the error rate exceeding an error rate threshold;
the obtaining the error rate of the deep learning model according to the first prediction data and the second prediction data further comprises:
dividing the difference between the first prediction data and the second prediction data by the number of pictures of the subdata set to obtain the error rate of the deep learning model;
triggering retraining of the deep learning model in response to the error rate exceeding an error rate threshold;
in response to the fact that the number of pictures of the data set needing to be predicted is not larger than the set number threshold, predicting the data set through a deep learning model to obtain first prediction data, judging data of the data set through manual prediction judgment, and performing accumulated counting on the data of the first prediction data, which are inconsistent with the judgment data; triggering retraining of the deep learning model in response to an accumulated value of an accumulated count exceeding an accumulated threshold.
5. The tuning management apparatus for a deep learning model according to claim 4, wherein the first prediction data includes prediction data of the number of pictures and a prediction accuracy, and the determination data includes determination data of the number of pictures and a determination accuracy.
6. A computer device, comprising:
at least one processor; and
memory storing a computer program operable on the processor, wherein the processor, when executing the program, performs the method of any of claims 1-3.
7. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010024479.8A CN111259939B (en) | 2020-01-10 | 2020-01-10 | Tuning management method, device, equipment and medium for deep learning model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010024479.8A CN111259939B (en) | 2020-01-10 | 2020-01-10 | Tuning management method, device, equipment and medium for deep learning model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111259939A CN111259939A (en) | 2020-06-09 |
| CN111259939B true CN111259939B (en) | 2022-06-07 |
Family
ID=70952742
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010024479.8A Active CN111259939B (en) | 2020-01-10 | 2020-01-10 | Tuning management method, device, equipment and medium for deep learning model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111259939B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114004224A (en) * | 2020-07-17 | 2022-02-01 | 中国电信股份有限公司 | Complex address word segmentation method and device and computer readable storage medium |
| CN111931698B (en) * | 2020-09-08 | 2021-01-26 | 平安国际智慧城市科技股份有限公司 | Image deep learning network construction method and device based on small training set |
| CN112132218B (en) * | 2020-09-23 | 2024-04-16 | 平安国际智慧城市科技股份有限公司 | Image processing method, device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108280552A (en) * | 2018-02-24 | 2018-07-13 | 合肥工业大学 | Methods of electric load forecasting and system, storage medium based on deep learning |
| US10223601B1 (en) * | 2017-10-12 | 2019-03-05 | Denso International America, Inc. | Synthetic traffic object generator |
| CN109446049A (en) * | 2018-11-01 | 2019-03-08 | 郑州云海信息技术有限公司 | A kind of server error diagnosis method and apparatus based on supervised learning |
| CN110032449A (en) * | 2019-04-16 | 2019-07-19 | 苏州浪潮智能科技有限公司 | A kind of method and device for the performance optimizing GPU server |
-
2020
- 2020-01-10 CN CN202010024479.8A patent/CN111259939B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10223601B1 (en) * | 2017-10-12 | 2019-03-05 | Denso International America, Inc. | Synthetic traffic object generator |
| CN108280552A (en) * | 2018-02-24 | 2018-07-13 | 合肥工业大学 | Methods of electric load forecasting and system, storage medium based on deep learning |
| CN109446049A (en) * | 2018-11-01 | 2019-03-08 | 郑州云海信息技术有限公司 | A kind of server error diagnosis method and apparatus based on supervised learning |
| CN110032449A (en) * | 2019-04-16 | 2019-07-19 | 苏州浪潮智能科技有限公司 | A kind of method and device for the performance optimizing GPU server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111259939A (en) | 2020-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111259939B (en) | Tuning management method, device, equipment and medium for deep learning model | |
| CN109815991B (en) | Training method and device of machine learning model, electronic equipment and storage medium | |
| CN108664920B (en) | Real-time large-scale cascading face clustering method and device | |
| JP2021533474A (en) | Node classification method, model training method, and its equipment, equipment and computer program | |
| CN110705598B (en) | Intelligent model management method, intelligent model management device, computer equipment and storage medium | |
| CN110942248B (en) | Training method and device for transaction wind control network and transaction risk detection method | |
| CN111429006A (en) | Financial risk index prediction model construction method and device and risk situation prediction method and device | |
| CN111382020A (en) | Transaction flow monitoring method and system | |
| WO2021051944A1 (en) | Automatic sleep aid music pushing method and apparatus, computer device, and storage medium | |
| CN111367773B (en) | Method, system, equipment and medium for detecting server network card | |
| CN109754854B (en) | Method and system for matching diagnosis codes and diagnosis names | |
| CN117742816A (en) | Operation control method and device of baseboard management controller | |
| CN111176932B (en) | Method and device for recording abnormal event log and readable medium | |
| CN116681967A (en) | Target detection method, device, equipment and medium | |
| CN116861480A (en) | Sensitive data identification method, device, equipment and storage medium | |
| CN111563147A (en) | Entity linking method and device in knowledge question-answering system | |
| CN116302870A (en) | Mechanical hard disk health assessment method, system and storage medium based on evolutionary diagram | |
| CN112802483B (en) | Method, device and storage medium for optimizing intention recognition confidence threshold | |
| Rezek et al. | Parametric model order estimation: a brief review | |
| CN113034284A (en) | Stock tendency analysis and early warning system based on algorithm, big data and block chain | |
| CN114489498A (en) | Method, device, equipment and medium for predicting residual life of NAND Flash medium | |
| CN114218134A (en) | Method and device for caching users | |
| CN112308294A (en) | Default probability prediction method and device | |
| CN112508130A (en) | Clustering method and device, electronic equipment and storage medium | |
| CN111507578A (en) | Risk assessment method and related device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |