CN112215357A - Model optimization method, device, equipment and computer readable storage medium - Google Patents
Model optimization method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN112215357A CN112215357A CN202011057563.6A CN202011057563A CN112215357A CN 112215357 A CN112215357 A CN 112215357A CN 202011057563 A CN202011057563 A CN 202011057563A CN 112215357 A CN112215357 A CN 112215357A
- Authority
- CN
- China
- Prior art keywords
- model
- data
- reasoning
- test data
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a model optimization method, a model optimization device, model optimization equipment and a computer-readable storage medium. The model optimization device includes: the system comprises a user scene data unit, a training test data set unit, a model and evaluation unit, a model training and updating unit and a user scene reasoning unit. In the invention, the user scene data unit firstly acquires the user scene data and then acquires the test data according to the user scene data, thereby solving the problems that the provider of the current model optimization device can not cover all scenes of the user and the training data is deficient. The method adopts the second test data to train the first reasoning model, adopts the first test data to carry out online evaluation on the second reasoning model, and trains and reasons the reasoning model at the same time, thereby not only improving the performance of the algorithm, but also occupying no resources used by the reasoning model and realizing the separation of online training, reasoning and user use.
Description
Technical Field
The invention relates to the technical field of model optimization, in particular to a model optimization method, a model optimization device, model optimization equipment and a computer-readable storage medium.
Background
The performance of the deep learning target detection algorithm is often greatly related to the actual scenes, data sets and category distribution, but the provider of the algorithm cannot cover all the scenes or has a relatively complete and full-category data set when training the algorithm model, and the trained model can cause that the training test result cannot be achieved in the process of actual use of a user.
Disclosure of Invention
The present invention is directed to solving at least one of the above problems.
To this end, a first object of the invention is to provide a model optimization method.
A second object of the present invention is to provide a model optimization apparatus.
A third object of the present invention is to provide a model optimization apparatus.
A fourth object of the present invention is to provide a computer-readable storage medium.
In order to achieve the first object of the present invention, the technical solution of the present invention provides a model optimization method, including: acquiring user scene data; acquiring first test data according to user scene data; selecting a first inference model from an initialization model library according to a user scene; training and evaluating the first reasoning model by adopting the first test data to obtain a second reasoning model; judging whether an expansion threshold condition is reached, and amplifying the first test data based on the reaching of the expansion threshold condition to obtain second test data; training and evaluating the second reasoning model by adopting second test data, and updating evaluation conditions; judging whether the second reasoning model is replaced by the first reasoning model or not according to the evaluation condition and the model threshold, updating the initialization model base by adopting the second reasoning model based on the replacement of the first reasoning model by adopting the second reasoning model, and setting the second reasoning model as a target reasoning model, otherwise, setting the first reasoning model as the target reasoning model; judging whether the end condition is met, adopting a target reasoning model to carry out reasoning based on the end condition being met, and continuing to amplify the test data based on the condition not being met to carry out model optimization.
According to the technical scheme, the user scene data are firstly acquired, then the test data are acquired according to the user scene data, and the problems that all scenes of a user cannot be covered by an algorithm provider at present and training data are deficient can be solved. In the technical scheme, after the second inference model is evaluated, the evaluation condition is updated, the model threshold value is set, whether the first inference model is replaced by the second inference model is judged according to the model threshold value, and the performance of the inference model is better improved by increasing the judgment on whether the inference model is updated and the automatic updating of the evaluation condition.
In addition, the technical scheme provided by the invention can also have the following additional technical characteristics:
in the above technical solution, before the obtaining of the user scene data is executed, the method further includes: and constructing an initialization model library, wherein the initialization model comprises a light-weight initialization model and/or a medium initialization model and/or a heavy initialization model.
By setting three initialization models, the model selection is more flexible and closer to the user scene.
In the above technical solution, training and evaluating the first inference model by using the first test data includes: the first test data comprise first evaluation data and first training data, the first reasoning model is trained by adopting the first training data, the first reasoning model is evaluated by adopting the first evaluation data, and the index value of the first reasoning model is obtained.
The first inference model is trained and evaluated through the first test data, the first test data are based on user scene data, model training is conducted based on the user scene data, the inference model can cover the user scene and the training data, and then inference is conducted through the inference model, and a result which accords with the user scene is obtained.
In the above technical solution, the expanding threshold condition includes: the number of target categories in the test data is less than the average of the number of categories; and/or the target class distribution proportion is smaller than the average value of the class number; and/or after the inference model tests and evaluates the user scene data, the confidence coefficient of a single target class is higher than the first confidence coefficient, wherein the target class number, the average value of the class number and the target class distribution proportion are normalized to an interval (0, 1).
By setting the expansion threshold condition, the target class data with small quantity of expansion classes or small distribution proportion can be pertinently expanded, so that samples in subsequently used training data are distributed more reasonably, and meanwhile, the expanded data is screened, so that the data to be expanded is closer to the effect of manual supplementation.
In the technical scheme, the second test data is adopted to train and evaluate the second inference model, the evaluation condition is updated, the second test data comprises second evaluation data and second training data, the second inference model is trained by adopting the second training data, the second inference model is evaluated by adopting the second evaluation data, and the index value of the second inference model is obtained; and updating the evaluation condition according to the index value of the second reasoning model.
The evaluation condition refers to that the index value of the inference model is increased step by step according to the proportion after each model training, and the index value is used for evaluating the obtained updated inference model. By updating the evaluation conditions, the subsequently updated inference model can be effectively evaluated, so that the model optimization process is simpler and more convenient.
In the above technical solution, determining whether the second inference model is replaced by the first inference model according to the evaluation condition and the model threshold includes: and performing online evaluation on the second reasoning model by adopting the first test data, judging whether the second reasoning model meets a model threshold value or not based on the online evaluation result meeting the evaluation condition, and replacing the first reasoning model by adopting the second reasoning model based on the second reasoning model meeting the model threshold value.
In the technical scheme, the first reasoning model is trained by adopting the second test data, the second reasoning model is evaluated on line by adopting the first test data, and the training and reasoning of the reasoning model are carried out simultaneously, so that the performance of the algorithm is improved, resources used by the reasoning model are not occupied, and the separation of on-line training, reasoning and user use is realized.
To achieve the second object of the present invention, the technical solution of the present invention provides a model optimization apparatus, including: a user scene data unit; training a test data set unit; a model and evaluation unit; a model training and updating unit; a user scenario inference unit; the user scene data unit acquires user scene data; the training test data set unit acquires first test data according to user scene data; the model and evaluation unit selects a first inference model from the initialized model library according to the user scene; the model training and updating unit trains the first reasoning model by adopting first test data to obtain a second reasoning model, and the model and evaluation unit evaluates the first reasoning model by adopting the first test data; judging whether an expansion threshold condition is reached, and training a test data set unit to amplify the first test data based on the reaching of the expansion threshold condition to obtain second test data; the model training and updating unit trains the second reasoning model by adopting second test data, and the model and evaluation unit evaluates the second reasoning model by adopting the second test data and updates evaluation conditions; the model training and updating unit judges whether the second reasoning model is replaced by the first reasoning model or not according to the evaluation condition and the model threshold value, based on the fact that the first reasoning model is replaced by the second reasoning model, the initialization model base is updated by the second reasoning model, the second reasoning model is set as a target reasoning model, and otherwise, the first reasoning model is set as the target reasoning model; and judging whether the end condition is met, adopting a target reasoning model to carry out reasoning by the user scene reasoning unit based on the end condition being met, and training the test data set unit to continue to amplify the test data and carry out model optimization based on the condition that the end condition is not met.
The user scene data unit firstly acquires user scene data, and then acquires test data and training data according to the user scene data, so that the problem that the provider of the existing model optimization device cannot cover all scenes of a user and the training data is deficient can be solved. The first reasoning model is trained by adopting the second test data, the second reasoning model is evaluated on line by adopting the first test data, and the training and reasoning of the reasoning model are carried out simultaneously, so that the performance of the algorithm is improved, resources used by the reasoning model are not occupied, and the separation of the on-line training, the reasoning and the use of a user is realized.
In addition, the technical scheme provided by the invention can also have the following additional technical characteristics:
in the above technical solution, the user scene data unit includes: at least one camera; the camera carries out data snapshot on a user scene at least comprising one target to be detected to obtain user scene data.
The method comprises the steps of collecting images in a real scene of a user by arranging a camera, then, after image data in the real scene are evaluated on line by an inference model, carrying out data amplification by adopting an automatically generated image with a real label, and amplifying the data by using the real scene image, so that all scenes of the user can be covered, and the trained model can be more adaptive to the scene of the user in the use process of the real user, and the test effect is optimized on line.
To achieve the third object of the present invention, the technical solution of the present invention provides a model optimization apparatus, including: the device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program; wherein the processor, when executing the computer program, implements the steps of the model optimization method according to any of the aspects of the present invention.
The model optimization device provided in the technical solution of the present invention implements the steps of the model optimization method according to any one of the technical solutions of the present invention, and thus has all the beneficial effects of the model optimization method according to any one of the technical solutions of the present invention, which are not described herein again.
To achieve the fourth object of the present invention, the technical solution of the present invention provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed, the steps of the model optimization method of any one of the technical solutions are implemented.
The computer-readable storage medium provided in the technical solution of the present invention implements the steps of the model optimization method according to any one of the technical solutions of the present invention, so that the computer-readable storage medium has all the beneficial effects of the model optimization method according to any one of the technical solutions of the present invention, and details thereof are not repeated herein.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flow chart of a model optimization method according to an embodiment of the present invention;
FIG. 2 is a second flowchart of a model optimization method according to an embodiment of the present invention;
FIG. 3 is a third flowchart of a model optimization method according to an embodiment of the present invention;
FIG. 4 is a fourth flowchart of a model optimization method according to an embodiment of the present invention;
FIG. 5 is a fifth flowchart of a model optimization method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a model optimization apparatus according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating a user context data unit according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a model optimization apparatus according to an embodiment of the present invention;
FIG. 9 is a second schematic diagram of the model optimization apparatus according to an embodiment of the present invention;
FIG. 10 is a sixth flowchart of a model optimization method according to an embodiment of the present invention;
FIG. 11 is a schematic diagram of a model optimization system according to an embodiment of the present invention;
FIG. 12 is a second schematic diagram of the model optimization system according to an embodiment of the present invention.
Wherein, the correspondence between the reference numbers and the part names in fig. 6 to 12 is:
100: a model optimization device; 110: a user scene data unit; 112: a camera; 120: training a test data set unit; 122: first test data; 124: second test data; 130: a model and evaluation unit; 140: a model training and updating unit; 150: a user scenario inference unit; 200: a model optimization device; 210: a memory; 220: a processor; 300: a model optimization system; 310: a computing platform; 312: a host computing platform; 3122: a first computing platform; 314: a secondary computing platform; 3142: a second computing platform; 3144: a third computing platform; 3146: a fourth computing platform; 3148: an Nth computing platform; 320: a local bin; 322: a storage bin; 324: a data bin; 326: a model bin; 328: and (5) post-processing screening bins.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
A model optimization method, a model optimization apparatus 100, a model optimization device 200, and a computer-readable storage medium according to some embodiments of the present invention are described below with reference to fig. 1 to 12.
Example 1:
as shown in fig. 1, the present embodiment provides a model optimization method, including:
step S102, user scene data is obtained;
step S104, acquiring first test data according to user scene data;
step S106, selecting a first inference model from the initialization model library according to the user scene;
step S108, training and evaluating the first reasoning model by adopting the first test data to obtain a second reasoning model;
step S110, judging whether an expansion threshold condition is reached;
step S112, based on reaching the expansion threshold condition, the first test data is amplified to obtain second test data;
step S114, training and evaluating the second reasoning model by adopting second test data, and updating evaluation conditions;
step S116, judging whether the second inference model is replaced by the first inference model or not according to the evaluation condition and the model threshold value;
step S118, replacing the first reasoning model by the second reasoning model, updating the initialization model base by the second reasoning model, and setting the second reasoning model as a target reasoning model;
otherwise, go to step S120;
step S120, setting a first reasoning model as a target reasoning model;
step S122, judging whether an end condition is met;
step S124, based on the end condition, adopting a target reasoning model to carry out reasoning;
and step S126, based on the condition that the end condition is not met, continuing to amplify the test data, and performing model optimization.
For example, in step S114, the trained inference model is trained and evaluated by using the augmented test data to obtain an updated inference model, in step S126, based on the condition that the updated inference model does not meet the end condition, the method returns to step S110 to augment the test data, and then the augmented data is used to train and evaluate the updated inference model to determine whether the updated inference model is replaced by the inference model before updating until the end condition is met.
The performance of the deep learning target detection algorithm is often in great relation with the actual user scenes, data sets and target category distribution, and a provider of the deep learning target detection algorithm cannot cover all the user scenes or has a relatively complete data set with sufficient target categories when training an algorithm model, so that the trained inference model cannot achieve the trained test result in the use process of an actual user.
Many methods for improving the target detection algorithm are provided, but the coverage of the trained model to the actual user scene is mostly not considered, and the data set considers more local extensions (such as the target detection method and device disclosed by the related art and the corresponding model training method and device), but still cannot be well adapted to the actual data of the user.
According to the embodiment, the user scene data is firstly acquired, and then the test data is acquired according to the user scene data, so that the problems that all scenes of a user cannot be covered by a deep learning target detection algorithm provider at present and training data is deficient can be solved.
In this embodiment, after the index of the second inference model is obtained, the evaluation condition is updated, where the evaluation condition is that the index of the inference model is gradually increased in proportion after each model training, and the index value is used for evaluation of the obtained updated inference model. By updating the evaluation conditions, the subsequently updated inference model can be effectively evaluated, so that the model optimization process is simpler and more convenient. And a model threshold value is also set, whether the second inference model is replaced by the first inference model is judged according to the model threshold value, and the performance of the inference model is better improved by increasing the judgment of the model threshold value whether the inference model is updated and the automatic updating of the evaluation condition. For example, the model threshold is divided into an accuracy evaluation part and a speed evaluation part, wherein the accuracy aspect adopts P (accuracy), R (recall), AP (single-class evaluation accuracy), mAP (average value of multiple classes of APs) and F-Measure (comprehensive reflection of P and R), the speed aspect adopts time evaluation for detecting single piece, the setting mode is that the priority of the speed and the accuracy is firstly set, and whether the model is replaced or not is evaluated according to the priority and further according to the index value of each aspect (for example, if a certain index value is higher than the last index value, the model can be replaced).
For example, the ending condition of the present embodiment may be optimized for the user interaction ending model. The method specifically comprises the following steps: after the model optimization is completed, the user is waited for instructions of the user, the user is asked to indicate whether to use the optimized model or not, the model optimization is ended, or the optimization is continued, and the requirements of the user can be timely acquired through interaction with the user, so that the use experience of the user is improved. And the user interaction is adopted as an end condition, and the user is taken as a main body, so that the initiative of algorithm optimization is enhanced, and the algorithm is more suitable for user scenes.
Example 2:
as shown in fig. 2, in addition to the technical features of the above embodiment, the present embodiment further includes the following technical features:
before executing acquiring the user scene data, the method further comprises:
step S202, an initialization model library is constructed, wherein the initialization model comprises a light-weight initialization model and/or a medium initialization model and/or a heavy initialization model.
By way of example, the initialization model may include three types, a lightweight initialization model, a medium initialization model, and a lightweight initialization model.
The lightweight initialization model is a model detection network which has less parameter in a model structure, is precise and simplified in network and can more highlight the detection rate on the basis of certain accuracy.
The medium initial model refers to a model detection network with medium parameters in a model structure and balanced detection rate and accuracy on the basis of certain cutting of the network structure.
The weight-type initialization model is a model detection network which has a complex network in a model structure, more parameter quantity, more occupied memory resources and higher attention on accuracy improvement in the aspect of basically meeting the speed requirement.
In this embodiment, in model selection, the types of models are selected according to the abundance of the target to be detected in the user data and the configuration condition of the computing resources, in combination with the requirements of speed and accuracy. By setting three initialization models, selection is provided for the user on whether the computing resources and the scene data target are rich or not and iteration, and a flexible bias selection can be made on the aspects of accuracy and speed.
Example 3:
as shown in fig. 3, in addition to the technical features of the above embodiment, the present embodiment further includes the following technical features:
training and evaluating the first inference model using the first test data, comprising:
step S302, the first test data comprise first evaluation data and first training data, the first reasoning model is trained by the first training data, the first reasoning model is evaluated by the first evaluation data, and the index value of the first reasoning model is obtained.
And the index value of the first inference model is used for quantitatively evaluating the quality of the model and is a reference basis for model comparison and updating.
The first inference model is trained and evaluated through the first test data, the first training data are based on user scene data, model training is conducted based on the user scene data, the inference model can cover the user scene and the training data, and then inference is conducted through the inference model, and a result which accords with the user scene is obtained.
Example 4:
in addition to the technical features of the above embodiment, the present embodiment further includes the following technical features:
the extended threshold conditions include:
the number of target categories in the test data is less than the average of the number of categories; and/or the target class distribution proportion is smaller than the average value of the class number; and/or after the inference model tests and evaluates the user scene data, the confidence coefficient of a single target class is higher than the first confidence coefficient, wherein the target class number, the average value of the class number and the target class distribution proportion are normalized to an interval (0, 1).
For example, the target category number in the test data is normalized and unified to the interval (0, 1); the average value of the category number is also unified to the interval (0, 1) in a normalization mode, and the target category distribution proportion is also unified to the interval (0, 1) in a normalization mode.
The category distribution is a visual presentation of the number of categories.
The single object category is determined according to actual requirements, and refers to any one of the object categories.
By setting the expansion threshold condition, the target class data with small quantity of expansion classes or small distribution proportion can be pertinently expanded, so that samples in subsequently used training data are distributed more reasonably, and meanwhile, the expanded data is screened, so that the data to be expanded is closer to the effect of manual supplementation.
In this embodiment, the confidence level of the single target class is higher than the first confidence level, which means that after the inference model tests and evaluates the user scenario data, the confidence level of the single target class is higher than one time or 0.9 times of the confidence level before the class (i.e., before the test and evaluation), and is specifically adjusted according to the evaluation result of the initialization model.
Example 5:
as shown in fig. 4, in addition to the technical features of the above embodiment, the present embodiment further includes the following technical features:
training and evaluating the second reasoning model by adopting the second test data, and updating evaluation conditions, wherein the evaluation conditions comprise:
step S402, the second test data comprise second evaluation data and second training data, the second reasoning model is trained by the second training data, the second reasoning model is evaluated by the second evaluation data, and an index value of the second reasoning model is obtained;
and step S404, updating the evaluation condition according to the index value of the second inference model.
And training and evaluating the second inference model through the second test data, wherein the second training data are expanded user scene data, model training is carried out based on the user scene data, so that the inference model can cover the user scene and the training data, and then inference is carried out through the inference model to obtain a result conforming to the user scene.
The evaluation condition refers to that the index value of the inference model is increased step by step according to the proportion after each model training, and the index value is used for evaluating the obtained updated inference model. By updating the evaluation conditions, the subsequently updated inference model can be effectively evaluated, so that the model optimization process is simpler and more convenient.
Example 6:
in addition to the technical features of the above embodiment, the present embodiment further includes the following technical features:
as shown in fig. 5, the determining whether the second inference model replaces the first inference model according to the evaluation condition and the model threshold includes:
and S502, carrying out online evaluation on the second inference model by adopting the first test data, judging whether the second inference model meets a model threshold value or not based on the condition that an online evaluation result meets an evaluation condition, and replacing the first inference model by adopting the second inference model based on the condition that the second inference model meets the model threshold value.
In this embodiment, the first inference model is trained by using the second test data, the second inference model is evaluated online by using the first test data, and the training and the inference of the inference model are performed simultaneously, so that the performance of the algorithm is improved, resources used by the inference model are not occupied, and the separation of online training, inference and user use is realized.
The related art discloses a remote online universal target detection system based on SSD-T, and the related art comprises: the client submits an image of a target to be detected and sends an image augmentation and training instruction to the server; after the model is trained, a user submits a standard image to be recognized waiting for target detection on line for detection; the server automatically stores the target image to be detected and the standard image to be identified submitted by the client; when receiving image augmentation and training requests sent by a client, the method automatically utilizes submitted target images to be detected to perform deep learning training set augmentation and model training. And after the model is trained, the server performs target detection on the standard image to be recognized submitted by the client and feeds back a detection result.
The training of the model in the related technology is on-line promotion, the emphasis is on the amplification of a data set to improve the model, the defects that the model detection and the model training are separately carried out, the occupied resources can only be unidirectional, and the training and reasoning cannot be simultaneously completed exist.
In the related art, the trained model needs to be exported and then imported for use next time, and the operation is complex. In addition, the present embodiment further adds a function of determining whether the model is updated and an automatic update of the determination condition so as to continue to improve the model performance.
Example 7:
as shown in fig. 6, the present embodiment provides a model optimization apparatus 100, including: a user scenario data unit 110, a training test data set unit 120, a model and evaluation unit 130, a model training and updating unit 140, and a user scenario inference unit 150. Wherein, the user scene data unit 110 obtains user scene data; the training test data set unit 120 obtains first test data according to the user scene data; the model and evaluation unit 130 selects a first inference model from the initialized model library according to the user scene; the model training and updating unit 140 trains the first inference model by using the first test data to obtain a second inference model, and the model and evaluation unit 130 evaluates the first inference model by using the first test data; judging whether an expansion threshold condition is reached, and training the test data set unit 120 to amplify the first test data based on the reaching of the expansion threshold condition to obtain second test data; the model training and updating unit 140 trains the second inference model by using the second test data, and the model and evaluation unit 130 evaluates the second inference model by using the second test data and updates the evaluation condition; the model training and updating unit 140 judges whether the second inference model is replaced with the first inference model or not according to the evaluation condition and the model threshold value, updates the initialization model base by using the second inference model based on the replacement of the first inference model by using the second inference model, and sets the second inference model as a target inference model, otherwise, sets the first inference model as the target inference model; and judging whether the ending condition is met, wherein the user scene reasoning unit 150 adopts a target reasoning model to carry out reasoning based on the ending condition being met, and the training test data set unit 120 continues to amplify the test data to carry out model optimization based on the condition that the ending condition is not met.
In this embodiment, a user scenario data unit 110, a training test data set unit 120, a model and evaluation unit 130, a model training and updating unit 140, and a user scenario inference unit 150 are added on the basis of deep learning target detection inference.
The user context data unit 110 first obtains user context data, and then obtains test data and training data according to the user context data, so as to solve the problem that the provider of the current model optimization device 100 cannot cover all the user contexts and the lack of training data.
In this embodiment, the first inference model is trained by using the second test data, the second inference model is evaluated online by using the first test data, and the training and the inference of the inference model are performed simultaneously, so that the performance of the algorithm is improved, resources used by the inference model are not occupied, and the separation of online training, inference and user use is realized.
In this embodiment, after the evaluation index of the second inference model is obtained, the evaluation condition is updated, and the threshold is set, and whether the first inference model is replaced by the second inference model is determined according to the threshold, and the performance of the inference model is better improved by increasing the determination of whether the inference model is updated and the automatic update of the evaluation condition.
For example, the training test data set unit 120 performs format uniform conversion on the test data. By converting the user data and the test data into a uniform format, the training, the online evaluation and the like of the subsequent model can be rapidly, stably and smoothly carried out. The training test data set unit 120 performs format uniform conversion on the test data. The format of the evaluation data is pictures, and the format of the training data is pictures and files actually labeled with xml and the like. The training test data set unit 120 performs format unified conversion on the evaluation data and the training data, and converts the evaluation data and the training data into a picture and a txt file, wherein the txt file of the training data is: the data type number, the picture path/picture name, the class number and the normalized target frame coordinate, and the evaluation data txt file is as follows: data type number, picture path/picture name.
Example 8:
as shown in fig. 7, in addition to the technical features of the above embodiment, the present embodiment further includes the following technical features:
the user scene data unit 110 includes: at least one camera 112; the camera 112 captures data of a user scene including at least one target to be detected, so as to obtain user scene data.
In this embodiment, the camera 112 is arranged to collect images in the real scene of the user, then, after the inference model online evaluates the image data in the real scene, the automatically generated image with the real label is used to amplify the data, and the data is amplified by applying the real scene image, so that all scenes of the user can be covered, and the trained model can better adapt to the scene of the user in the use process of the real user, so that the test effect is optimized online.
The related technology discloses a remote end on-line general target detection system based on SSD-T, which adopts a transformed target image to be detected to be randomly placed in an image of a database, and the recorded randomly placed position is used as a real frame label. The data amplification mode of the embodiment is to automatically generate an image with a real label to amplify data after the image data in the actual scene is evaluated online by an initialization model.
Example 9:
as shown in fig. 8, the present embodiment provides a model optimization apparatus 200, which includes a memory 210 storing a computer program. And a processor 220 executing the computer program. Wherein the processor 220, when executing the computer program, performs the steps of the model optimization method according to any of the embodiments of the present invention.
Example 10:
the present embodiments provide a computer-readable storage medium, comprising: the computer readable storage medium stores a computer program which, when executed, performs the steps of the model optimization method according to any one of the embodiments of the present invention.
The specific embodiment is as follows:
the embodiment utilizes the technologies of deep learning, image processing, network communication and the like, and realizes the autonomous improvement of the performance of the target detection algorithm under the user scene by means of the current computational platform, and solves the problems that the current algorithm provider cannot cover all scenes of the user and the training data is deficient.
As shown in fig. 9, the present embodiment provides a model optimization apparatus 100, which adds a user scenario data unit 110, a training test data set unit 120, a model and evaluation unit 130, a model training and updating unit 140, and a user scenario inference unit 150 on the basis of deep learning target detection inference.
The camera 112 captures data, and the camera 112 captures a user scene including at least one target to be detected, so as to obtain user scene data.
In fig. 9, the first test data 122 includes first evaluation data and first training data, and the second test data 124 includes second evaluation data and second training data, in order to reasonably compare the performance of the current model and the updated model, the second test data 124 is data that is newly updated to the first test data 122 after the model and evaluation unit 130 performs data set amplification, and is used for next model evaluation, and the model evaluation test data set in the model training and updating unit 140 still uses the first test data 122, that is, the data set of the second test data 124 is not updated. The second initialized model library output by the model training and updating unit 140 is the model output after the third model evaluation judgment is updated, and is updated into the model bin 326 of the local bin 320 unit, and simultaneously replaces the model in the first initialized model library corresponding to the selected model, so as to be used for the next evaluation update.
As shown in fig. 10, the model optimization method includes:
step S602, initializing model construction.
There are at least three models A, B, C, light weight, medium, and heavy weight.
And step S604, selecting a reasoning model in advance according to the scene and the user data, carrying out model evaluation, and acquiring a corresponding index value.
Index values such as AP, mAP, FPS, wait for user instructions.
Step S606, setting the expansion threshold condition of the user data set, expanding the data set after the condition is met, and updating the whole data set.
Step S608, train the selected model with the newly updated training data, and set a new model evaluation index, which automatically increases the conditions after each model training update.
And step S610, index evaluation and management optimization of the model.
And (4) carrying out on-line evaluation on the model according to the conditions set in the step (S208), wherein the adopted test set is the last test set, and comparison is convenient.
Step S612, obtaining the updated model, comparing the updated model with the current model according to a set threshold value, and selecting whether to replace the current model, if so, updating the initialized model library.
And step S614, if the model is updated, adopting the new model for reasoning, and waiting for an instruction, otherwise, continuously using the original model.
If the model is updated, a new model is adopted for reasoning, an instruction is waited, the step S616 is entered, otherwise, the original model is continuously used, and the step S604 is returned.
Step S616, the user interacts whether the model needs to be optimized.
And (4) whether the user interaction needs to optimize the model, if so, returning to the step S604, and if not, ending and waiting for instructions.
According to the embodiment, the performance of the target detection algorithm can be improved on line in a user scene, the problem that an algorithm provider cannot cover all scenes of a user and the training data are deficient is effectively solved, and meanwhile, the user experience is improved through automation and user interaction means.
The improvement of hardware and calculation power of the existing computing platform is beneficial to the implementation of a method for improving the performance of a detection algorithm on line in an actual scene, so that the method has better adaptability.
As shown in fig. 11, the embodiment further provides a model optimization system 300 based on the method, which includes a computing platform 310 and a local bin 320, as shown in fig. 12, the computing platform 310 is used for training and reasoning testing, and is further divided into a primary computing platform 312 and a secondary computing platform 314, and the local bin 320 is used for data processing, and includes a storage bin 322, a data bin 324 (including a training data set and a testing data set), a model bin 326 (including an initialization model, a current reasoning model and an update model), and a post-processing screening bin 328 (cleaning and updating model output data).
Among other things, the host computing platform 312: including a first computing platform 3122, which may employ hisi3559A, for primarily model inference calculations, where the selected current detection model infers data from the user's context and interacts the detection results with the secondary computing platform 314 and the local bin 320.
Secondary computing platform 314: including second computing platform 3142, third computing platform 3144, fourth computing platform 3146, …, and nth computing platform 3148, where second computing platform 3142, third computing platform 3144, fourth computing platform 3146, …, and nth computing platform 3148 may each employ hisi3559A for online training, model evaluation of selected models, including training data sets augmented with local training sets and user context data, and interacting with local bin 320 and host computing platform 312.
Storage bin 322: the image data, the model parameter file and the buffer file generated in the middle are stored, updated and cleared periodically, and the data bin 324, the model bin 326 and the post-processing screening bin 328 are linked through nodes.
Data bin 324: the strategy is to divide the data into an evaluation data set and a training data set according to a certain proportion, store the data in a storage bin 322, and access, update and modify the data through nodes.
Model bin 326: for updating models, including initialization models (A, B, C, etc.), current inference models, and update models, model files are stored in the storage 322, and files are accessed, modified, and updated through the nodes.
Post-treatment screening bin 328: the method is used for labeling, screening and evaluating the data output during model reasoning, the evaluation strategy is to minimize the cost ratio of the label data generated by model output, and the threshold value is set in advance.
In summary, the embodiment of the invention has the following beneficial effects:
1. according to the embodiment, the user scene data is firstly acquired, and then the test data and the training data are acquired according to the user scene data, so that the problem that an algorithm provider cannot cover all scenes of a user and the training data are deficient can be solved.
2. In this embodiment, the first inference model is trained by using the second test data, the second inference model is evaluated online by using the first test data, and the training and the inference of the inference model are performed simultaneously, so that the performance of the algorithm is improved, resources used by the inference model are not occupied, and the separation of online training, inference and user use is realized.
In the present invention, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance; the term "plurality" means two or more unless expressly limited otherwise. The terms "mounted," "connected," "fixed," and the like are to be construed broadly, and for example, "connected" may be a fixed connection, a removable connection, or an integral connection; "coupled" may be direct or indirect through an intermediary. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the description of the present invention, it is to be understood that the terms "upper", "lower", "left", "right", "front", "rear", and the like indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of description and simplification of description, but do not indicate or imply that the referred device or unit must have a specific direction, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention.
In the description herein, the description of the terms "one embodiment," "some embodiments," "specific embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of model optimization, comprising:
acquiring user scene data;
acquiring first test data according to the user scene data;
selecting a first inference model from an initialization model library according to a user scene;
training and evaluating the first reasoning model by adopting the first test data to obtain a second reasoning model;
judging whether an expansion threshold condition is reached, and amplifying the first test data based on reaching the expansion threshold condition to obtain second test data;
training and evaluating the second reasoning model by adopting the second test data, and updating an evaluation condition;
judging whether the second inference model is replaced by the first inference model or not according to the evaluation condition and a model threshold value, updating the initialization model base by adopting the second inference model based on replacing the first inference model by adopting the second inference model, and setting the second inference model as a target inference model, otherwise, setting the first inference model as the target inference model;
judging whether an ending condition is met, adopting the target reasoning model to carry out reasoning based on the condition that the ending condition is met, and continuing to amplify the test data based on the condition that the ending condition is not met so as to carry out model optimization.
2. The model optimization method of claim 1, wherein prior to performing the obtaining user context data, further comprising:
and constructing the initialization model library, wherein the initialization model comprises a light-weight initialization model and/or a medium initialization model and/or a heavy initialization model.
3. The model optimization method of claim 1, wherein said training and evaluating said first inference model using said first test data comprises:
the first test data comprise first evaluation data and first training data, the first reasoning model is trained by adopting the first training data, the first reasoning model is evaluated by adopting the first evaluation data, and the index value of the first reasoning model is obtained.
4. The model optimization method of claim 3, wherein the expansion threshold condition comprises:
the number of target categories in the test data is less than the average of the number of categories; and/or
The target category distribution proportion is smaller than the average value of the category number; and/or
After the inference model tests and evaluates the user scene data, the confidence coefficient of a single target class is higher than a first confidence coefficient;
wherein the target class number, the average of the class numbers, and the target class distribution ratio are all normalized to an interval (0, 1).
5. The model optimization method of claim 3, wherein said training and evaluating said second inference model using said second test data, updating evaluation conditions, comprises:
the second test data comprise second evaluation data and second training data, the second reasoning model is trained by adopting the second training data, the second reasoning model is evaluated by adopting the second evaluation data, and an index value of the second reasoning model is obtained;
and updating the evaluation condition according to the index value of the second reasoning model.
6. The model optimization method of claim 5, wherein said determining whether said second inference model replaces said first inference model based on said evaluation conditions and model thresholds comprises:
and performing online evaluation on the second reasoning model by adopting the first test data, judging whether the second reasoning model meets the model threshold value or not based on the online evaluation result meeting the evaluation condition, and replacing the first reasoning model by adopting the second reasoning model based on the second reasoning model meeting the model threshold value.
7. A model optimization apparatus (100), comprising:
a user scene data unit (110);
training a test data set unit (120);
a model and evaluation unit (130);
a model training and updating unit (140);
a user context inference unit (150);
wherein the user scene data unit (110) acquires user scene data; a training test data set unit (120) acquires first test data according to the user scene data; the model and evaluation unit (130) selects a first inference model from the initialized model library according to the user scene; the model training and updating unit (140) trains the first reasoning model by adopting the first test data to obtain a second reasoning model, and the model and evaluation unit (130) evaluates the first reasoning model by adopting the first test data; judging whether an expansion threshold condition is reached, and training a test data set unit (120) to amplify the first test data to obtain second test data based on the reaching of the expansion threshold condition; the model training and updating unit (140) trains the second reasoning model by adopting the second test data, and the model and evaluation unit (130) evaluates the second reasoning model by adopting the second test data and updates evaluation conditions; a model training and updating unit (140) judges whether the second inference model changes the first inference model according to the evaluation condition and the model threshold value, based on changing the first inference model by the second inference model, the initialization model base is updated by the second inference model, the second inference model is set as a target inference model, otherwise, the first inference model is set as the target inference model; and judging whether an end condition is met, adopting the target reasoning model to carry out reasoning by the user scene reasoning unit (150) based on the end condition being met, and continuing to amplify the test data by the training test data set unit (120) based on the condition that the end condition is not met so as to carry out model optimization.
8. The model optimization device (100) of claim 7, wherein the user context data unit (110) comprises:
at least one camera (112);
the camera (112) carries out data snapshot on a user scene at least comprising one target to be detected, and user scene data are obtained.
9. A model optimization device (200), comprising:
a memory (210) in which a computer program is stored;
a processor (220) executing the computer program;
wherein the processor (220), when executing the computer program, realizes the steps of the model optimization method according to any one of claims 1 to 6.
10. A computer-readable storage medium, comprising:
the computer-readable storage medium stores a computer program which, when executed, implements the steps of the model optimization method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011057563.6A CN112215357A (en) | 2020-09-29 | 2020-09-29 | Model optimization method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011057563.6A CN112215357A (en) | 2020-09-29 | 2020-09-29 | Model optimization method, device, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112215357A true CN112215357A (en) | 2021-01-12 |
Family
ID=74051006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011057563.6A Pending CN112215357A (en) | 2020-09-29 | 2020-09-29 | Model optimization method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112215357A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113297169A (en) * | 2021-02-26 | 2021-08-24 | 阿里云计算有限公司 | Database instance processing method, system, device and storage medium |
| CN114741269A (en) * | 2022-04-14 | 2022-07-12 | 网思科技股份有限公司 | Method for estimating service performance of reasoning system |
| WO2022237484A1 (en) * | 2021-05-12 | 2022-11-17 | 华为云计算技术有限公司 | Inference system and method, apparatus, and related device |
| WO2023016537A1 (en) * | 2021-08-13 | 2023-02-16 | 华为技术有限公司 | Database management system, data processing method, and device |
| CN116819964A (en) * | 2023-06-20 | 2023-09-29 | 小米汽车科技有限公司 | Model optimization method, model optimization device, electronic device, vehicle and medium |
| CN116976443A (en) * | 2023-09-22 | 2023-10-31 | 浙江大华技术股份有限公司 | Performance index debugging method, terminal equipment and computer storage medium |
| CN117315380A (en) * | 2023-11-30 | 2023-12-29 | 深圳市健怡康医疗器械科技有限公司 | Deep learning-based pneumonia CT image classification method and system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130129143A1 (en) * | 2011-11-21 | 2013-05-23 | Seiko Epson Corporation | Global Classifier with Local Adaption for Objection Detection |
| CN109214360A (en) * | 2018-10-15 | 2019-01-15 | 北京亮亮视野科技有限公司 | A kind of construction method of the human face recognition model based on ParaSoftMax loss function and application |
| CN109978062A (en) * | 2019-03-28 | 2019-07-05 | 北京九章云极科技有限公司 | A kind of model on-line monitoring method and system |
| CN110322005A (en) * | 2019-06-25 | 2019-10-11 | 深圳爱莫科技有限公司 | Neural network model training method and device, face identification method |
| CN110503206A (en) * | 2019-08-09 | 2019-11-26 | 阿里巴巴集团控股有限公司 | A kind of prediction model update method, device, equipment and readable medium |
| CN110705598A (en) * | 2019-09-06 | 2020-01-17 | 中国平安财产保险股份有限公司 | Intelligent model management method and device, computer equipment and storage medium |
| CN111191797A (en) * | 2020-01-03 | 2020-05-22 | 深圳追一科技有限公司 | Information processing method, information processing device, electronic equipment and storage medium |
| CN111445029A (en) * | 2020-03-30 | 2020-07-24 | 北京嘉楠捷思信息技术有限公司 | Model evaluation method, model evaluation device and computer-readable storage medium |
-
2020
- 2020-09-29 CN CN202011057563.6A patent/CN112215357A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130129143A1 (en) * | 2011-11-21 | 2013-05-23 | Seiko Epson Corporation | Global Classifier with Local Adaption for Objection Detection |
| CN109214360A (en) * | 2018-10-15 | 2019-01-15 | 北京亮亮视野科技有限公司 | A kind of construction method of the human face recognition model based on ParaSoftMax loss function and application |
| CN109978062A (en) * | 2019-03-28 | 2019-07-05 | 北京九章云极科技有限公司 | A kind of model on-line monitoring method and system |
| CN110322005A (en) * | 2019-06-25 | 2019-10-11 | 深圳爱莫科技有限公司 | Neural network model training method and device, face identification method |
| CN110503206A (en) * | 2019-08-09 | 2019-11-26 | 阿里巴巴集团控股有限公司 | A kind of prediction model update method, device, equipment and readable medium |
| CN110705598A (en) * | 2019-09-06 | 2020-01-17 | 中国平安财产保险股份有限公司 | Intelligent model management method and device, computer equipment and storage medium |
| CN111191797A (en) * | 2020-01-03 | 2020-05-22 | 深圳追一科技有限公司 | Information processing method, information processing device, electronic equipment and storage medium |
| CN111445029A (en) * | 2020-03-30 | 2020-07-24 | 北京嘉楠捷思信息技术有限公司 | Model evaluation method, model evaluation device and computer-readable storage medium |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113297169A (en) * | 2021-02-26 | 2021-08-24 | 阿里云计算有限公司 | Database instance processing method, system, device and storage medium |
| CN113297169B (en) * | 2021-02-26 | 2022-05-31 | 阿里云计算有限公司 | Database instance processing method, system, device and storage medium |
| WO2022237484A1 (en) * | 2021-05-12 | 2022-11-17 | 华为云计算技术有限公司 | Inference system and method, apparatus, and related device |
| WO2023016537A1 (en) * | 2021-08-13 | 2023-02-16 | 华为技术有限公司 | Database management system, data processing method, and device |
| CN114741269A (en) * | 2022-04-14 | 2022-07-12 | 网思科技股份有限公司 | Method for estimating service performance of reasoning system |
| CN114741269B (en) * | 2022-04-14 | 2022-09-23 | 网思科技股份有限公司 | Method for estimating service performance of reasoning system |
| CN116819964A (en) * | 2023-06-20 | 2023-09-29 | 小米汽车科技有限公司 | Model optimization method, model optimization device, electronic device, vehicle and medium |
| CN116819964B (en) * | 2023-06-20 | 2024-02-06 | 小米汽车科技有限公司 | Model optimization method, model optimization device, electronic device, vehicle and medium |
| CN116976443A (en) * | 2023-09-22 | 2023-10-31 | 浙江大华技术股份有限公司 | Performance index debugging method, terminal equipment and computer storage medium |
| CN116976443B (en) * | 2023-09-22 | 2023-12-29 | 浙江大华技术股份有限公司 | Performance index debugging method, terminal equipment and computer storage medium |
| CN117315380A (en) * | 2023-11-30 | 2023-12-29 | 深圳市健怡康医疗器械科技有限公司 | Deep learning-based pneumonia CT image classification method and system |
| CN117315380B (en) * | 2023-11-30 | 2024-02-02 | 深圳市健怡康医疗器械科技有限公司 | Deep learning-based pneumonia CT image classification method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112215357A (en) | Model optimization method, device, equipment and computer readable storage medium | |
| CN106503236B (en) | Artificial intelligence based problem classification method and device | |
| CN109344884B (en) | Media information classification method, method and device for training picture classification model | |
| CN115408151A (en) | Method for accelerating learning training of bang | |
| CN113516255A (en) | Federal learning modeling optimization method, apparatus, readable storage medium, and program product | |
| CN111160469B (en) | Active learning method of target detection system | |
| CN110428114B (en) | Fruit tree yield prediction method, device, equipment and computer readable storage medium | |
| JP2007317068A (en) | Recommending device and recommending system | |
| CN113037783B (en) | Abnormal behavior detection method and system | |
| CN112596543B (en) | Multi-machine collaborative inspection system | |
| CN112540845A (en) | Mobile edge calculation-based collaboration system and method | |
| CN111291618A (en) | Labeling method, device, server and storage medium | |
| CN113301376A (en) | Live broadcast interaction mode and system based on virtual reality technology | |
| CN111199540A (en) | Image quality evaluation method, image quality evaluation device, electronic device, and storage medium | |
| CN108924196B (en) | Industrial Internet green energy management system | |
| CN113793298A (en) | Pulmonary nodule detection model construction optimization method, equipment, storage medium and product | |
| CN110705627A (en) | Target detection method, target detection system, target detection device and readable storage medium | |
| CN112070162A (en) | Multi-class processing task training sample construction method, device and medium | |
| CN112070062A (en) | Hadoop-based crop waterlogging image classification detection and implementation method | |
| CN117834724B (en) | Video learning resource management system based on big data analysis | |
| CN111209428A (en) | Image retrieval method, device, equipment and computer readable storage medium | |
| CN113849101B (en) | Information processing method, information processing device, electronic equipment and computer readable storage medium | |
| US11722551B2 (en) | Optimization of files compression | |
| CN111813528B (en) | Video big data standardization convergence gateway system and method based on task statistical characteristics | |
| CN110830583B (en) | Mine resource detection method, server and user side equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |