CN112115975A - Deep learning network model fast iterative training method and equipment suitable for monitoring device - Google Patents
Deep learning network model fast iterative training method and equipment suitable for monitoring device Download PDFInfo
- Publication number
- CN112115975A CN112115975A CN202010832070.9A CN202010832070A CN112115975A CN 112115975 A CN112115975 A CN 112115975A CN 202010832070 A CN202010832070 A CN 202010832070A CN 112115975 A CN112115975 A CN 112115975A
- Authority
- CN
- China
- Prior art keywords
- model
- network
- layer
- deployment model
- monitoring device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012806 monitoring device Methods 0.000 title claims abstract description 105
- 238000012549 training Methods 0.000 title claims abstract description 82
- 238000013135 deep learning Methods 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000001514 detection method Methods 0.000 claims abstract description 60
- 238000000605 extraction Methods 0.000 claims abstract description 60
- 238000003062 neural network model Methods 0.000 claims description 18
- 238000012360 testing method Methods 0.000 claims description 18
- 238000002372 labelling Methods 0.000 claims description 17
- 238000013524 data verification Methods 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 claims description 6
- 230000003213 activating effect Effects 0.000 claims description 3
- 238000013508 migration Methods 0.000 claims description 3
- 230000005012 migration Effects 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The application provides a deep learning network model fast iterative training method and equipment suitable for a monitoring device, wherein the method comprises the following steps: respectively training an offline complex model and an online deployment model; based on preset time, identifying image data which is from a monitoring device and identified by the on-line deployment model through the off-line complex model; determining an on-line deployment model needing to be upgraded in the on-line deployment models respectively corresponding to the plurality of monitoring devices; based on triggering of a preset condition, locking a preset layer and all layers in front of the preset layer in a network feature extraction layer of an on-line deployment model to be upgraded, and training a plurality of layers behind the preset layer in the network feature extraction layer and a detection head network layer of the on-line deployment model. According to the method, the offline complex multi-model cascade hybrid integrated detection mode is adopted, and the online deployment model is combined for joint training, so that the differentiated customized detection and optimized upgrading capability aiming at different scenes is realized.
Description
Technical Field
The application relates to the technical field of deep learning, in particular to a deep learning network model fast iterative training method and equipment suitable for a monitoring device
Background
In recent years, the development of artificial intelligence technology is rapid, and especially, intelligent monitoring equipment terminals with deep learning technology as a core are widely applied. The existing monitoring equipment terminal generates a large amount of image data to be transmitted back to the cloud server, and both the network bandwidth and the computing throughput become the performance bottleneck of the cloud computing service. Therefore, the intelligent analysis model is migrated from the server side to the device front end, and the intelligent analysis mode based on the device front end is becoming more and more popular. However, as the network structure of deep learning becomes more and more complex, the network parameters become more and more, the resource demand becomes larger and larger, and the deployability of the deep learning network model on the mobile device is seriously hindered. The current intelligent analysis mode based on the equipment terminal is limited by hardware resources of the equipment, so that a large complex model of a server end cannot be effectively utilized. Therefore, the current deployment at the equipment terminal is a lightweight network model, the calculation power and the recognition accuracy are far lower than those of a server-side model, and the detection effect on the target object is greatly different from that of the server-side model.
And the detection scenes faced by large-scale equipment terminals are various, the environment is complicated, the models deployed at present are mostly based on a large amount of labeled data of different scenes for basic training, the generalization capability of the models is enhanced, and the detection effect is poor for each scene deployed to an actual detection line. If label training is carried out on each piece of equipment, labor cost is high, and time and labor are wasted. With the increasing number of the equipment terminals, the workload of customizing the model only by artificial secondary labeling training for application scenes with diversified scenes also shows exponential growth, and how to quickly iterate the equipment terminal model to adapt to a single scene is a great obstacle to realizing batch customization service of the model at present.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a method and equipment for fast iterative training of a deep learning network model, which are suitable for a monitoring device, and solves the problems of high labor cost and long model training period in the process of carrying out individual labeling training on deployed models on lines.
On one hand, the embodiment of the invention provides a deep learning network model fast iterative training method suitable for a monitoring device, which comprises the following steps: respectively training an offline complex model and an online deployment model; the offline complex model is a deep neural network model or a neural network model formed by cascading a plurality of deep neural network models, and is deployed on a server, and the online deployment model is a lightweight neural network model and is deployed on a plurality of monitoring devices corresponding to the server; based on preset time, identifying image data which is from a monitoring device and identified by the on-line deployment model through the off-line complex model; determining an on-line deployment model needing to be upgraded in the on-line deployment models respectively corresponding to the plurality of monitoring devices; based on triggering of a preset condition, locking a preset layer and all layers in front of the preset layer in a network feature extraction layer of an on-line deployment model to be upgraded, and training a plurality of layers behind the preset layer in the network feature extraction layer and a detection head network layer of the on-line deployment model.
According to the method and the device, the online deployment model needing to be upgraded is determined through the image identified by the online deployment model of the offline complex multi-model cascade detection, the online deployment model needing to be upgraded is trained in a targeted manner, and manual marking is replaced by high-precision deep neural network model marking, so that a large amount of manpower is not needed for customized upgrading of the online deployment model; in addition, when the online deployment model is upgraded, the network feature extraction layers with the specified number of layers are locked, only the unlocked network feature extraction layers and the detection head network layers are trained, and the training period of the online deployment model is greatly shortened.
In one embodiment, after training a plurality of layers of the network feature extraction layer after the number of layers where the network feature extraction layer is currently located and the detection head network layer of the on-line deployment model, the method further includes writing the number of layers of the locked network feature extraction layer, the network weight of each layer of the trained network feature extraction layer and the network weight of each layer of the trained detection head network layer into an update file; and informing the monitoring device where the on-line deployment model needing to be upgraded is located so that the monitoring device can acquire the update file to lock a preset layer in the on-line deployment model network feature extraction layer and all layers before the preset layer, and initializing all network feature extraction layers after the preset layer and a detection head network layer by layer according to corresponding network weights to complete model upgrading.
According to the method and the device, the number of the locked network feature extraction layers of the upgraded online deployment model, the network weight of each trained unlocked network feature extraction layer and the network weight of each detection head network layer are written into the update file for storage, the online deployment model only needs to read data in the update file, the network layers unlocked by the deployment model on the original line are initialized layer by layer according to the corresponding network weights, the model upgrading can be completed, the data transmission quantity is greatly reduced, the whole model does not need to be transmitted, and only one update file needs to be transmitted for each model upgrading.
In one embodiment, the locking of a preset layer and all layers in front of the preset layer in a network feature extraction layer of an on-line deployment model to be upgraded, and the training of a plurality of layers behind the preset layer in the network feature extraction layer and a detection head network layer of the on-line deployment model specifically comprise the steps of locking all network feature extraction layers of the on-line deployment model based on triggering of a preset condition, and only training the detection head network layer; and detecting the accuracy of the trained model, if the accuracy does not reach a first preset value, reducing the numerical value of the preset layer by layer, continuing to train the network feature extraction layer and all the detection head network layers after the preset layer, and stopping training until the accuracy of the trained model reaches or exceeds the first preset value.
According to the embodiment of the application, the number of layers of the locked network feature extraction layer is dynamically adjusted, when the model accuracy rate obtained by only training the detection head network layer does not meet the standard, the number of layers of the locked network feature extraction layer is reduced layer by layer, the rest network feature extraction layer and the detection head network layer are trained together, and the training is stopped until the accuracy rate of the obtained on-line deployment model reaches the preset value, so that the accuracy of the trained on-line deployment model is close to the accuracy of the deep neural network, a target object can be detected more accurately during target detection, and the model is more integrated into the scene.
In one embodiment, the method further comprises: when the on-line deployment model is trained for the first time, storing network layer data of the on-line deployment model into a network configuration file and an original weight file; the network configuration file comprises a deep learning algorithm currently used by the online deployment model and a network configuration file of the deep learning algorithm; the original weight file comprises the network weight of each layer of network layer in the initial state of the on-line deployment model; initializing the network structure of the initial on-line deployment model layer by layer according to the network configuration file and the original weight file, and installing the on-line deployment model in each monitoring device.
In one embodiment, after the image data from the monitoring device and identified via the online deployment model is identified by the offline complex model, the method further comprises: labeling a target object on the image data identified by the complex offline model; comparing the marked image data with corresponding image data identified by the on-line deployment model on the monitoring device; if the type and the position of the target object identified by the complex offline model are different from the type and the position identified by the online deployment model for the same image, the image is a false alarm image; and if the same image is subjected to target object identification by the complex offline model, and the target object is not identified by the online deployment model, the image is a false negative image.
In one embodiment, the triggering of the preset condition specifically includes: determining the identification accuracy rate of the hidden danger of each online deployment model according to the missing report rate and the false report rate of the hidden danger identification of each online deployment model in a preset period; and in a preset period, activating an upgrading task of the on-line deployment model when the accuracy of any on-line deployment model is lower than a corresponding preset value.
In an embodiment, the accuracy of identifying the hidden danger of the deployment model on each line specifically includes: when the on-line deployment model is upgraded, one part of the off-line complex model labeled data is extracted as a training sample, the other part of the off-line complex model labeled data is used as a first test sample, and one part of the training sample is taken out as a second test sample; after the on-line deployment model is trained each time, the first test sample and the second test sample are used for carrying out cross data verification on the trained model, so that the identification accuracy of the on-line deployment model is obtained, and whether the on-line deployment model is updated or not is determined.
In one embodiment, the training of the offline complex model and the online deployment model respectively specifically includes: respectively training an offline complex model and an online deployment model based on the same large-scale manual labeling data set; the offline complex model traverses the image data in the large-scale manual labeling dataset; extracting difficult example samples with identification precision lower than a second preset value in the identification result; and adding a new deep learning algorithm for the difficult example sample, and integrating the new deep learning algorithm into the offline complex model.
According to the embodiment of the application, the offline complex model and the online deployment model are trained through the same labeled large-scale data set, and the offline complex model is trained through a difficult sample, so that the accuracy of the offline complex model reaches a higher standard, manual labeling can be replaced approximately, a large amount of manpower is saved, and the working pressure of workers is reduced.
In one embodiment, the method further comprises: and in the pictures of the large-scale labeling data set, target objects of different types are scratched, and the scratched target objects of different types are fused into a non-target image in an image collected by the monitoring device through data enhancement and style migration technologies to obtain a new target image so as to expand a training sample.
On the other hand, the embodiment of the invention provides a monitoring device for fast iterative training based on a deep learning network model, which comprises: a processor; and a memory having stored thereon computer-executable code that, when executed, causes the processor to perform a method of fast iterative training of a deep learning network model suitable for use in a surveillance device according to any of claims 1-9.
The embodiment of the invention combines an offline complex multi-model cascade detection mode with an online deployment model to carry out joint detection on the scene. The online deployment model is adapted to a single scene of the terminal equipment by fast iteration, the detection index of the online deployment model can be improved, and the capability of differential customized detection, optimization and upgrade of different scenes is realized. With the increasing number of the equipment terminals, for application scenes with diversified scenes, the labor cost can be effectively reduced, the training period is shortened, model tuning and upgrading iteration are fast, and the batch customized service of the models is realized.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and not to limit the invention. In the drawings:
fig. 1 is a schematic diagram of a deep learning network model fast iterative training system suitable for a monitoring device according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a deep learning network model fast iterative training method suitable for a monitoring device according to an embodiment of the present application;
fig. 3 is a schematic diagram of a deep learning network model fast iterative training device suitable for a monitoring device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear, the following description of the present application will be made in detail and completely with reference to the embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The existing intelligent monitoring technology taking a deep learning technology as a core is a lightweight network model deployed at an equipment terminal, and the detection effect is greatly different from that of a server-side model. And most models deployed to equipment terminals are subjected to basic training based on a large number of different scenes, so that the generalization capability is strong, the pertinence is poor, the detection effect on scene targets under actual single equipment is poor, the false alarm is high, and the targeted model differentiation training is required.
And the training of the current customized differentiated model needs a large amount of manual secondary labeling confirmation, so that the workload is very large for large-scale equipment application scenes, and the rapid upgrading and iterative optimization of the model are not facilitated.
In order to solve the above problems, embodiments of the present invention provide a method and an apparatus for fast iterative training of a deep learning network model, which are suitable for a monitoring device.
The technical solutions proposed in the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of a deep learning network model fast iterative training system suitable for a monitoring device according to an embodiment of the present application.
As shown in fig. 1, the monitoring system includes a cloud server 100 and a plurality of monitoring devices, where the cloud server 100 includes a GPU cluster 140 and a storage server 150; the plurality of monitoring devices include a monitoring device 110, a monitoring device 120, a monitoring device 130, and the like.
It should be noted that, in the monitoring system of the embodiment of the present application, a plurality of monitoring devices are provided, each of the plurality of monitoring devices is in communication with the cloud server 100, the number of the monitoring devices may be one, or may be a plurality of monitoring devices, as shown in fig. 1, each of the plurality of monitoring devices is provided with a monitoring device 110, a monitoring device 120, and a monitoring device 130. In the embodiment of the present application, the functions, structures and connections of the monitoring devices are the same, and for convenience of description, the monitoring device 110 is taken as an example for explanation.
In one embodiment, GPU cluster 140 first trains the offline complex model and the online deployment model, respectively, based on the same large-scale manually labeled dataset. The online deployment model is a lightweight neural network model, and is installed in the monitoring device 110, and is used for performing target detection on a scene image acquired by the monitoring device. The offline complex model is a deep neural network model or a neural network model formed by cascading a plurality of deep neural network models, is deployed on the cloud server 100, and is used for performing secondary detection on an image after the detection of the deployed model on the line and labeling a target object.
Specifically, the monitoring device 110 acquires scene images according to a preset time interval, and an online deployment model installed on the monitoring device 110 performs target detection on the scene images acquired by the monitoring device 110 and marks out target objects. The monitoring device 110 sends all the image data identified by the on-line deployment model in a preset period to the GPU cluster 140 of the cloud server 100. Based on a preset time interval, or based on a trigger, or based on a preset time, GPU cluster 140 detects the image identified by the deployment model on the line again through the offline complex model, and marks out the target object. GPU cluster 140 determines whether the on-line deployment model corresponding to monitoring device 110 needs to be upgraded, and if the on-line deployment model needs to be upgraded, activates the upgrade task of the on-line deployment model. The secondary detection time interval of the GPU cluster 140 deploying the model identification image on the line is longer than the time interval of the monitoring device 110 performing the target detection on the acquired scene image.
After determining that the on-line deployment model corresponding to the monitoring device 110 needs to be upgraded, the GPU cluster 140 first obtains the on-line deployment model, locks all network feature extraction layers of the on-line deployment model, and only iteratively trains the detection head network layer of the on-line deployment model. After training is finished, the GPU cluster performs cross data verification on the trained on-line deployment model to obtain the accuracy rate of the on-line deployment model. And if the accuracy rate of the trained on-line deployment model does not reach the preset value, subtracting one from the number of the locked network feature extraction layers, training the unlocked network feature extraction layers and all the detection head network layers, and circularly executing the subtraction of one from the locked network feature extraction layers and the training of the unlocked network layers until the accuracy rate of the on-line deployment model reaches the preset value.
The GPU cluster 140 writes the number of layers of the network feature extraction layer finally locked by the updated online deployment model, the network weight of each layer of the trained network feature extraction layer, and the weight data of each layer of the trained detection head network layer into an update file, and stores the update file in the storage server 150. The monitoring device 110 obtains a corresponding update file from the storage server 150, locks a specified number of network feature extraction layers according to the content recorded in the update file, initializes the trained network feature extraction layers and the detection head network layers layer by layer according to corresponding network weights, and completes updating of the online deployment model after initialization is completed.
Therefore, the GPU cluster 140 in the embodiment of the present application determines the accuracy of the on-line deployment model by comparing the identification results of the off-line complex model and the on-line deployment model, and performs customized upgrade on the on-line deployment model, so that the neural network model can be quickly and iteratively tuned and optimized, thereby implementing a batch customized service of the model.
Fig. 2 is a flowchart of a deep learning network model fast iterative training method suitable for a monitoring device according to an embodiment of the present application.
S201, the monitoring device collects scene images at regular time and carries out target detection through an on-line deployment model.
Specifically, the monitoring device 110 collects scene images at preset time intervals, and performs target detection on the collected scene images through an installed on-line deployment model. The online deployment model frames the detected target object by using a labeling frame, labels the target type, and stores the detected image in the monitoring device 110. The monitoring device 110 periodically sends the image detected by the deploying model in one period to the cloud server 100.
For example, the monitoring device 110 acquires scene images every hour, and the scene images are temporarily stored in the monitoring device 110 after being identified by the on-line deployment model. The monitoring device 110 uploads all images acquired within one week and detected by the online deployment model, and target positions and target types corresponding to the images to the cloud server 100 once a week.
S202, detecting the scene image by the offline complex model based on a preset time interval, a preset moment or triggering, and generating annotation data.
Specifically, the offline complex model is designed mainly for high detection accuracy, and the GPU cluster 140 trains the offline complex model by using a target detection algorithm with high current detection accuracy based on a large-scale labeled data set, and combines several complex models into a complex network cascade detection model. And traversing the samples in the large-scale labeled data set by using the complex network cascade detection model, selecting difficult samples with low identification precision in the large-scale labeled data set, such as the difficult samples lower than a preset value, training a difficult sample detection model by using the difficult samples, and integrating the difficult samples into the complex network cascade detection model to obtain a final offline complex model.
It can be seen that the offline complex model mentioned in the embodiment of the present application is not limited to refer to one neural network model, and may also be a neural network model formed by cascading a plurality of deep neural network models.
In an embodiment, the cloud server 100 starts an offline complex model to perform target detection again on the image uploaded by the monitoring device 110 based on the period of uploading the image by the monitoring device 110 or triggered by the image transmission behavior of the monitoring device 110, frames the detected target object with a labeling frame, and labels the target type. And taking the image labeled by the offline complex model as labeled data and storing the labeled data into a labeled data set.
S203, the GPU cluster 140 determines an on-line deployment model needing to be upgraded.
Specifically, the GPU cluster 140 uses the annotation data detected by the offline complex model as standard data, and compares the standard data with the image data uploaded by the monitoring device 110 one by one. If the position and type of the target object identified by the offline complex model are different from the position and type of the target object identified by the online deployment model of the monitoring device 110 for the same image, the image is a false alarm image. If the target object is identified by the offline complex model and the target object is not identified by the online deployment model of the monitoring device 110 for the same image, the image is a false negative image. The images other than the missing report image and the false report image are accurate images, and the ratio of the number of the accurate images to the number of the images uploaded by the monitoring device 110 in the preset period is the accuracy of the on-line deployment model. And if the accuracy is lower than a preset value, activating an upgrading task of the online deployment model.
For example, the accuracy preset value is 90%. If there are 168 images collected and identified in one week uploaded by the monitoring device 110, wherein there are 12 missed images and 8 false images, the accurate images are 148, and the identification accuracy of the on-line deployment model corresponding to the monitoring device 110 is obtained
Less than 90%, the upgrade task for the online deployment model is activated. If there are 7 missed images and 8 false images, the accurate images are 153, and the identification accuracy of the on-line deployment model corresponding to the monitoring device 110 is obtained as
If the sum of the data values is more than 90%, the monitoring device 110 continues to use the online deployment model, and the GPU cluster stores the labeled data in one week after the labeling by the offline complex model into the labeled data set.
S204, the GPU cluster 140 extracts the marked data in the marked data set under the scene of the on-line deployment model.
Specifically, GPU cluster 140 extracts the tagged data corresponding to monitoring device 110 from the tagged data set. Note that the annotation data herein includes all annotation data corresponding to the monitoring device 110 in a plurality of previous cycles.
For example, the time taken for the monitoring device 110 to be used is 10 weeks, and accordingly, the labeled data corresponding to the monitoring device 110 in 10 weeks is stored in the labeled data set. If the deployment model on the 11 th time cycle needs to be trained, the GPU cluster needs to take out all the annotation data corresponding to the monitoring device 110 in the first 11 th week.
S205, the GPU cluster performs data enhancement and sample augmentation on the training samples.
Because a large number of samples are needed for training the neural network model, the number of samples is far from enough only by using the labeled data corresponding to the monitoring device 110, and therefore data enhancement and expansion need to be performed on the training samples. Specifically, the GPU cluster scratches various target objects in the large-scale manual labeling dataset, and fuses the target objects one by one into the non-target images corresponding to the monitoring device 110 by using the techniques of data enhancement, style migration, and the like.
It should be noted that the number of each type of object to be extracted should be kept the same or similar, for example, the difference between the numbers of each type of object is smaller than a certain preset value, so as to expand each type of training sample in a balanced manner.
And S206, the GPU cluster locks the designated network layer based on the current online deployment model, and conducts tuning training on the unlocked network layer.
Specifically, the online deployment model file includes three files, namely, a network configuration file, a deployment file, a capacity file, and an update file, namely, a deployment model file. The network configuration files comprise a deep learning algorithm used for training an initial on-line deployment model and network configuration files corresponding to the deep learning algorithm; the original weight file comprises the network weight of each layer of the trained on-line deployment model; the update file is a file used in later stage when upgrading is needed, and comprises the network weight of each layer of the network after the online deployment model is updated. And adding a super parameter lock _ layer into the network configuration file and the update file, and setting an initial value to be 0, wherein the super parameter refers to the number of the locked network feature extraction layers.
In one embodiment, when the model is deployed on the training line for the first time, the GPU cluster trains all network feature extraction layers and all detection head network layers of the on-line deployment model using a large-scale manual annotation data set. And storing the information of the corresponding on-line deployment model after training into a network configuration file and an original weight file, wherein the hyper-parameter is 0 because any network layer is not locked. Each monitoring device obtains a network configuration file and an original weight file in the cloud server 100, and loads network weights of all layers in the original weight file according to a network structure to obtain an initial online deployment model. It should be noted that the initial on-line deployment model on each monitoring device is the same.
After it is determined that the on-line deployment model of the monitoring device 110 needs to be upgraded, the GPU cluster pulls the on-line deployment model currently used by the monitoring device 110, locks all network feature extraction layers of the current on-line deployment model, and trains only the detection head network layer of the current on-line deployment model. At this time, the hyper-parameter is equal to the number of layers of all network feature extraction of the current online deployment model.
For example, assuming that the online deployment model structure layers have 135 layers, wherein the network feature extraction layer has 114 layers, the GPU cluster locks the first 114 layer network layer of the online deployment model, and only trains the next 21 detection head network layers, where the hyper-parameter equals to 114.
And S207, performing cross data verification on the trained on-line deployment model by the GPU cluster.
Specifically, when the online deployment model is upgraded and trained, a part of the annotation data corresponding to the monitoring device 110 extracted in S204 and various types of samples expanded in S205 are selected as training samples, another part is test samples, and a part of the annotation data is taken out from the training samples and also used as test samples, and the test samples are used to perform cross data verification on the trained online deployment model. The accuracy rate verification is carried out by cross mixing the training samples and the test samples, so that overfitting can be reduced to a certain extent.
For example, of the extracted annotation data corresponding to the monitoring device 110 and the various samples expanded in S205, 75% is extracted as a training sample, the remaining 25% is extracted as a test sample, 10% is extracted as a test sample from the training sample, and the two test samples are mixed together to test the trained on-line deployment model.
And S208, judging whether the identification accuracy of the trained on-line deployment model reaches a preset value by the GPU cluster.
Specifically, the GPU cluster calculates the identification accuracy of the trained on-line deployment model according to the cross data verification result, wherein the accuracy is the percentage of the number of the accurately identified test samples in the total number of the test samples. The accuracy is compared with a preset value, if the accuracy is equal to or greater than the preset value, S210 is performed, and if the accuracy is less than the preset value, S209 is performed.
For example, if the preset value is 90%, and the calculated accuracy is 85% and less than 90%, S209 is executed. If the calculated accuracy is 95% or more than 90%, S210 is performed.
S209, the GPU cluster dynamically adjusts the number of layers of the network feature extraction locked by the on-line deployment model.
Specifically, if the accuracy of the trained on-line deployment model is smaller than the preset value, the GPU cluster reduces the number of the locked network feature extraction layers by one on the basis of the number of the currently locked network feature extraction layers, and correspondingly, the superparameter is also reduced by one. And circularly executing S206-S209 after the locked network feature extraction layer is adjusted until the accuracy of the trained online deployment model reaches a preset value, namely executing S210.
For example, the number of currently locked network feature extraction layers is 114, the GPU cluster adjusts the number of locked network feature extraction layers to 113, and the hyper-parameter becomes 113. And (3) locking a front 113-layer network feature extraction layer by the GPU cluster, training a last layer of network feature extraction layer and all detection head network layers to obtain a new on-line deployment model, detecting the accuracy of the on-line deployment model, changing the number of the locked network feature extraction layers into 112 if the accuracy still does not reach the standard, training the back two-layer network feature extraction layers and all detection head network layers, and repeating the steps until the accuracy of the on-line deployment model reaches a preset value, and stopping the circulation.
S210, the GPU cluster stores the on-line deployment model with the accuracy reaching the preset value, and outputs a model updating file.
Specifically, after the on-line deployment model is trained, the GPU cluster writes the final hyper-parameters into an update file, stores the network weights after the unlocked network feature extraction layer is trained into the update file, and stores the network layer weight data of the trained detection head network layer into the update file. The update file is stored in the storage server 150.
S211, the monitoring device 110 acquires the update file to complete upgrading.
The cloud server 100 sends an upgrade instruction and an update file storage address to the monitoring device 110, and the monitoring device 110 obtains the update file according to the storage address. The on-line deployment model corresponding to the monitoring device 110 keeps the network feature extraction layer indicated by the hyper-parameter unchanged, and initializes all layers after the hyper-parameter layer by layer according to the corresponding network weight to complete the upgrade of the on-line deployment model.
For example, the hyper-parameter lock _ layer in the update file is equal to 100, after the monitoring device 110 reads the content of the update file, the corresponding online deployment model locks the front 100 layers of network feature extraction layers, and the network feature extraction layers after 100 layers are initialized layer by layer according to the corresponding network weights.
It should be noted that the method and the device for fast iterative training of the deep learning network model, which are applicable to the monitoring device, are not limited to what kind of scene, and the detected target object is not limited to what kind of object. In another embodiment, the application scenario may also be road monitoring, and the target object may be a pedestrian, a vehicle, or the like.
Fig. 3 is a schematic diagram of a deep learning network model fast iterative training device suitable for a monitoring device according to an embodiment of the present application.
As shown in fig. 3, the apparatus includes a processor 320, a memory 310.
The processor 320 is configured to train the offline complex model and the online deployment model, respectively, and further configured to identify, through the offline complex model, image data from the monitoring apparatus 110 and identified by the online deployment model, and determine which of the online deployment models respectively corresponding to the plurality of monitoring apparatuses need to be upgraded.
The processor 320 is further configured to lock a preset layer in the network feature extraction layer of the online deployment model to be upgraded and all layers before the preset layer, and train a plurality of layers after the preset layer in the network feature extraction layer and the detection head network layer.
A memory 310 having stored thereon computer executable code, which when executed, causes the processor 310 to execute the above-described method for fast iterative training of a deep learning network model for a monitoring device.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application. It should be noted that various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement and the like made without departing from the principle of the present invention shall be included in the scope of the claims of the present application.
Claims (10)
1. A deep learning network model fast iterative training method suitable for a monitoring device is characterized by comprising the following steps:
respectively training an offline complex model and an online deployment model; the offline complex model is a deep neural network model or a neural network model formed by cascading a plurality of deep neural network models, and is deployed on a server, and the online deployment model is a lightweight neural network model and is deployed on a plurality of monitoring devices corresponding to the server;
based on preset time or trigger, identifying image data which comes from a monitoring device and is identified by the online deployment model through the offline complex model;
determining an on-line deployment model needing to be upgraded in the on-line deployment models respectively corresponding to the plurality of monitoring devices;
based on triggering of a preset condition, locking a preset layer and all layers in front of the preset layer in a network feature extraction layer of an on-line deployment model to be upgraded, and training a plurality of layers behind the preset layer in the network feature extraction layer and a detection head network layer of the on-line deployment model.
2. The method for fast iterative training of the deep learning network model suitable for the monitoring device according to claim 1, wherein after training a plurality of layers after the number of layers where the network feature extraction layer is currently located and a network layer of a detection head of the on-line deployment model, the method further comprises:
writing the number of layers of the preset layer, the network weight of each layer of the trained network feature extraction layer and the network weight of each layer of the trained network layer of the detection head into an update file;
informing a monitoring device where the on-line deployment model needing to be upgraded is located so that the monitoring device can acquire the update file to lock a preset layer in the on-line deployment model network feature extraction layer and all layers before the preset layer; and initializing all network feature extraction layers and detection head network layers after the preset layer by layer according to corresponding network weights to finish model upgrading.
3. The method according to claim 1, wherein the locking of all layers in and before a preset layer in a network feature extraction layer of an online deployment model to be upgraded, and the training of a plurality of layers in and after the preset layer in the network feature extraction layer and a detection head network layer of the online deployment model specifically include:
based on the triggering of a preset condition, locking all network feature extraction layers of the on-line deployment model, and only training a detection head network layer;
and detecting the accuracy of the trained model, if the accuracy does not reach a first preset value, reducing the numerical value of the preset layer by layer, continuing to train the network feature extraction layer and all the detection head network layers behind the preset layer, and stopping training until the accuracy of the trained model reaches or exceeds the first preset value.
4. The method for fast iterative training of the deep learning network model suitable for the monitoring device according to claim 2, further comprising:
when the on-line deployment model is trained for the first time, storing network layer data of the on-line deployment model into a network configuration file and an original weight file;
the network configuration file comprises a deep learning algorithm currently used by the online deployment model and a network configuration file of the deep learning algorithm;
the original weight file comprises the network weight of each layer of network layer in the initial state of the on-line deployment model;
initializing the network structure of the initial on-line deployment model layer by layer according to the network configuration file and the original weight file, and installing the on-line deployment model in each monitoring device.
5. The method according to claim 1, wherein after the image data from the monitoring device and identified by the online deployment model is identified by the offline complex model, the method further comprises:
labeling a target object on the image data identified by the complex offline model;
comparing the marked image data with corresponding image data identified by the on-line deployment model on the monitoring device;
if the type and the position of the target object identified by the complex offline model are different from the type and the position identified by the online deployment model for the same image, the image is a false alarm image;
and if the same image is subjected to target object identification by the complex offline model, and the target object is not identified by the online deployment model, the image is a false negative image.
6. The method for fast iterative training of the deep learning network model applicable to the monitoring device according to claim 1, wherein the triggering of the preset condition specifically comprises:
determining the identification accuracy rate of the hidden danger of each online deployment model according to the missing report rate and the false report rate of the hidden danger identification of each online deployment model in a preset period; and in a preset period, activating an upgrading task of the on-line deployment model when the accuracy of any on-line deployment model is lower than a corresponding preset value.
7. The method for rapid iterative training of the deep learning network model applicable to the monitoring device according to claim 6, wherein the accuracy of identification of the hidden danger of each online deployment model is specifically as follows:
when the on-line deployment model is upgraded, one part of the off-line complex model labeled data is extracted as a training sample, the other part of the off-line complex model labeled data is used as a first test sample, and one part of the training sample is taken out as a second test sample;
after the on-line deployment model is trained each time, the first test sample and the second test sample are used for carrying out cross data verification on the trained model, so that the identification accuracy of the on-line deployment model is obtained, and whether the on-line deployment model is updated or not is determined.
8. The method for fast iterative training of the deep learning network model suitable for the monitoring device according to claim 1, wherein the separately training of the offline complex model and the online deployment model specifically comprises:
respectively training an offline complex model and an online deployment model based on the same large-scale labeling data set;
the offline complex model traverses the image data in the large-scale labeling dataset; extracting difficult example samples with identification precision lower than a second preset value in the identification result;
and adding a new deep learning algorithm for the difficult example sample, and integrating the new deep learning algorithm into the offline complex model.
9. The method for fast iterative training of the deep learning network model suitable for the monitoring device according to claim 1, further comprising:
and in the pictures of the large-scale labeling data set, target objects of different types are scratched, and the scratched target objects of different types are fused into a non-target image in an image collected by the monitoring device through data enhancement and style migration technologies to obtain a new target image so as to expand a training sample.
10. A deep learning network model fast iterative training device suitable for a monitoring device comprises:
a processor;
and a memory having stored thereon computer-executable code that, when executed, causes the processor to perform a method of fast iterative training of a deep learning network model suitable for use in a surveillance device according to any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010832070.9A CN112115975B (en) | 2020-08-18 | 2020-08-18 | Deep learning network model rapid iterative training method and equipment suitable for monitoring device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010832070.9A CN112115975B (en) | 2020-08-18 | 2020-08-18 | Deep learning network model rapid iterative training method and equipment suitable for monitoring device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112115975A true CN112115975A (en) | 2020-12-22 |
| CN112115975B CN112115975B (en) | 2024-04-12 |
Family
ID=73805035
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010832070.9A Active CN112115975B (en) | 2020-08-18 | 2020-08-18 | Deep learning network model rapid iterative training method and equipment suitable for monitoring device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112115975B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113626052A (en) * | 2021-07-29 | 2021-11-09 | 苏州浪潮智能科技有限公司 | Deep learning training platform |
| CN114693612A (en) * | 2022-03-16 | 2022-07-01 | 深圳大学 | Knee joint bone tumor detection method based on deep learning and related device |
| CN116737605A (en) * | 2023-08-11 | 2023-09-12 | 上海燧原科技有限公司 | Data prefetching method, device, equipment and medium based on chip multilevel storage |
| CN116882469A (en) * | 2023-09-06 | 2023-10-13 | 苏州浪潮智能科技有限公司 | Impulse neural network deployment method, device and equipment for emotion recognition |
| WO2023207228A1 (en) * | 2022-04-28 | 2023-11-02 | 重庆长安汽车股份有限公司 | Intelligent connected vehicle data training method and electronic device based on privacy data protection, and computer readable storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107992906A (en) * | 2018-01-02 | 2018-05-04 | 联想(北京)有限公司 | A kind of model treatment method, system, terminal device and server |
| CN108830143A (en) * | 2018-05-03 | 2018-11-16 | 深圳市中电数通智慧安全科技股份有限公司 | A kind of video analytic system based on deep learning |
| US20190349265A1 (en) * | 2016-11-14 | 2019-11-14 | Orange | Method and device for updating a predictive model of a variable relating to a mobile terminal |
| CN110705684A (en) * | 2019-08-22 | 2020-01-17 | 中国科学院计算技术研究所 | Environment self-adaptive learning method and system based on end cloud cooperation |
| US20200074058A1 (en) * | 2018-08-28 | 2020-03-05 | Samsung Electronics Co., Ltd. | Method and apparatus for training user terminal |
| CN110991905A (en) * | 2019-12-05 | 2020-04-10 | 支付宝(杭州)信息技术有限公司 | Risk model training method and device |
| CN111143097A (en) * | 2018-11-03 | 2020-05-12 | 千寻位置网络有限公司 | GNSS positioning service-oriented fault management system and method |
| CN111368788A (en) * | 2020-03-17 | 2020-07-03 | 北京迈格威科技有限公司 | Training method and device of image recognition model and electronic equipment |
| CN111444848A (en) * | 2020-03-27 | 2020-07-24 | 广州英码信息科技有限公司 | Specific scene model upgrading method and system based on federal learning |
| CN111521911A (en) * | 2020-05-21 | 2020-08-11 | 山东信通电子股份有限公司 | Intelligent monitoring method and equipment for power transmission line |
-
2020
- 2020-08-18 CN CN202010832070.9A patent/CN112115975B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190349265A1 (en) * | 2016-11-14 | 2019-11-14 | Orange | Method and device for updating a predictive model of a variable relating to a mobile terminal |
| CN107992906A (en) * | 2018-01-02 | 2018-05-04 | 联想(北京)有限公司 | A kind of model treatment method, system, terminal device and server |
| CN108830143A (en) * | 2018-05-03 | 2018-11-16 | 深圳市中电数通智慧安全科技股份有限公司 | A kind of video analytic system based on deep learning |
| US20200074058A1 (en) * | 2018-08-28 | 2020-03-05 | Samsung Electronics Co., Ltd. | Method and apparatus for training user terminal |
| CN111143097A (en) * | 2018-11-03 | 2020-05-12 | 千寻位置网络有限公司 | GNSS positioning service-oriented fault management system and method |
| CN110705684A (en) * | 2019-08-22 | 2020-01-17 | 中国科学院计算技术研究所 | Environment self-adaptive learning method and system based on end cloud cooperation |
| CN110991905A (en) * | 2019-12-05 | 2020-04-10 | 支付宝(杭州)信息技术有限公司 | Risk model training method and device |
| CN111368788A (en) * | 2020-03-17 | 2020-07-03 | 北京迈格威科技有限公司 | Training method and device of image recognition model and electronic equipment |
| CN111444848A (en) * | 2020-03-27 | 2020-07-24 | 广州英码信息科技有限公司 | Specific scene model upgrading method and system based on federal learning |
| CN111521911A (en) * | 2020-05-21 | 2020-08-11 | 山东信通电子股份有限公司 | Intelligent monitoring method and equipment for power transmission line |
Non-Patent Citations (4)
| Title |
|---|
| BU XUHUI等: "Model free adaptive iterative learning control for a class of nonlinear system with randomly varying iteration lengths", 《JOURNAL OF THE FRANKLIN INSTITUTE》, vol. 356, no. 5, 31 March 2019 (2019-03-31), pages 2491 - 2504 * |
| YING WANG等: "A Data-Driven Architecture for Personalized QoE Management in 5G Wireless Network", 《IEEE WIRELESS COMMUNICATIONS》, vol. 24, no. 1, 15 November 2016 (2016-11-15), pages 102 - 110, XP011641851, DOI: 10.1109/MWC.2016.1500184WC * |
| 刘耕等: "联邦学习在5G云边协同场景中的原理和应用综述", 《通讯世界》, vol. 27, no. 7, 25 July 2020 (2020-07-25), pages 50 - 52 * |
| 路星昊: "基于深度学习的图像分类方法及加速训练技术", 《中国优秀硕士学位论文全文数据库（信息科技辑）》, no. 2, 15 February 2020 (2020-02-15), pages 138 - 1407 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113626052A (en) * | 2021-07-29 | 2021-11-09 | 苏州浪潮智能科技有限公司 | Deep learning training platform |
| CN113626052B (en) * | 2021-07-29 | 2023-08-08 | 苏州浪潮智能科技有限公司 | Deep learning training platform |
| CN114693612A (en) * | 2022-03-16 | 2022-07-01 | 深圳大学 | Knee joint bone tumor detection method based on deep learning and related device |
| WO2023207228A1 (en) * | 2022-04-28 | 2023-11-02 | 重庆长安汽车股份有限公司 | Intelligent connected vehicle data training method and electronic device based on privacy data protection, and computer readable storage medium |
| CN116737605A (en) * | 2023-08-11 | 2023-09-12 | 上海燧原科技有限公司 | Data prefetching method, device, equipment and medium based on chip multilevel storage |
| CN116737605B (en) * | 2023-08-11 | 2023-11-14 | 上海燧原科技有限公司 | Data prefetching method, device, equipment and medium based on chip multilevel storage |
| CN116882469A (en) * | 2023-09-06 | 2023-10-13 | 苏州浪潮智能科技有限公司 | Impulse neural network deployment method, device and equipment for emotion recognition |
| CN116882469B (en) * | 2023-09-06 | 2024-02-02 | 苏州浪潮智能科技有限公司 | Impulse neural network deployment method, device and equipment for emotion recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112115975B (en) | 2024-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112115975A (en) | Deep learning network model fast iterative training method and equipment suitable for monitoring device | |
| CN110443969B (en) | Fire detection method and device, electronic equipment and storage medium | |
| US20230084869A1 (en) | System for simplified generation of systems for broad area geospatial object detection | |
| CN112613375B (en) | Tire damage detection and identification method and equipment | |
| CN112037142B (en) | Image denoising method, device, computer and readable storage medium | |
| CN111259812B (en) | Inland ship re-identification method and equipment based on transfer learning and storage medium | |
| KR20230171966A (en) | Image processing method and device and computer-readable storage medium | |
| CN113780435A (en) | Vehicle damage detection method, device, equipment and storage medium | |
| CN114418021B (en) | Model optimization method, device and computer program product | |
| CN116012815A (en) | Traffic element identification method, multi-task network model, training method and training device | |
| CN115620090A (en) | Model training method, low-illumination target re-recognition method and device and terminal equipment | |
| CN112269827B (en) | Data processing method and device, computer equipment and computer readable storage medium | |
| CN113704276A (en) | Map updating method and device, electronic equipment and computer readable storage medium | |
| CN115482436B (en) | Training method and device for image screening model and image screening method | |
| CN115984723A (en) | Road damage detection method, system, device, storage medium and computer equipment | |
| CN116964588A (en) | Target detection method, target detection model training method and device | |
| CN114419018A (en) | Image sampling method, system, device and medium | |
| CN114898182A (en) | Image data screening method and system based on target detection learning algorithm | |
| CN113191183A (en) | Unsupervised domain false label correction method and unsupervised domain false label correction device in personnel re-identification | |
| CN114299435A (en) | Scene clustering method and device in video and related equipment | |
| CN113192106A (en) | Livestock tracking method and device | |
| CN113392867A (en) | Image identification method and device, computer equipment and storage medium | |
| CN111062479A (en) | Model rapid upgrading method and device based on neural network | |
| CN114818880B (en) | Method and system for automatically identifying key operation flow of YOLOv3 railway | |
| CN113111804B (en) | Face detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |