CN115661595A - Multi-model dynamic fusion method in deep learning object detection and storage medium - Google Patents
Multi-model dynamic fusion method in deep learning object detection and storage medium Download PDFInfo
- Publication number
- CN115661595A CN115661595A CN202211301088.1A CN202211301088A CN115661595A CN 115661595 A CN115661595 A CN 115661595A CN 202211301088 A CN202211301088 A CN 202211301088A CN 115661595 A CN115661595 A CN 115661595A
- Authority
- CN
- China
- Prior art keywords
- object detection
- model
- deep learning
- models
- dynamic fusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a multi-model dynamic fusion method in deep learning object detection and a storage medium, comprising the following steps: step 1, obtaining an object detection model, and training the object detection model based on deep learning through an existing data set to obtain the model; or directly integrating the existing object detection model; step 2, dynamically loading model object detection model integration by using a configuration file; and step 3, a multi-model fusion algorithm comprises the following steps: (1) Loading and analyzing different series of models, and performing forward prediction on an input image to obtain intensive frames and class output; (2) Adding output boxes of different models to a list; (3) And inputting all the intensive prediction frames into a non-maximum suppression module to obtain an optimal object detection result. The invention realizes the dynamic fusion of different series of deep learning object detection models, increases the reusability of the deep learning models, and greatly improves the overall performance of the object detection models.
Description
Technical Field
The invention relates to an object detection method, in particular to a multi-model dynamic fusion method and a storage medium in deep learning object detection.
Background
Object detection is a machine-time vision technique that uses a computer and related algorithms to classify and locate objects contained in an image. The method can divide object detection into two types of traditional object detection technologies and deep learning-based object detection technologies, the traditional object detection technologies utilize manual feature design and aim at feature design and classifier training, and the method is difficult to obtain effective feature expression of a target object in an image, is particularly sensitive to environmental noise and causes great restriction on engineering application. The object detection technology based on deep learning can automatically extract effective characteristic expression of a target object in an image from training data to resist the influence of environmental noise, so that the object detection algorithm based on deep learning is applied to engineering application to a certain extent. In recent years, with the application of deep learning-based object detection algorithms to engineering, the problems of low object detection and identification precision and low generalization capability are gradually exposed, and how to improve the identification performance of the deep learning object detection algorithms is a problem to be solved urgently.
At present, an object detection algorithm based on deep learning mainly depends on the strong fitting capability of a neural network, can automatically learn the characteristics of an object to be detected in complex image data, but the model capacity cannot be simply increased by depending on the improvement of the complexity of a model structure. The capacity of the deep learning model is increased along with the increase of the structure and the number of layers of the neural network, the deep learning model enters a saturation state after reaching a certain degree, and the expressed characteristic is that the recognition of the model does not rise after reaching a certain numerical value, so that the phenomenon of training data overflow is generated. Once the model training is finished, the target classes solidified in the object detection model cannot be changed any more unless training from a new model is performed, resulting in a decrease in the flexibility of application of the object detection model.
Disclosure of Invention
The invention aims to overcome the defects and provides a multi-model dynamic fusion method and a storage medium in deep learning object detection.
According to a first aspect, the invention provides a method for dynamic fusion of multiple models in deep learning object detection, which comprises the following steps:
step 1, object detection model acquisition
The model used in the present invention can come from two aspects:
(1) Training an object detection model based on deep learning through an existing data set to obtain a model;
(2) The existing object detection model is directly integrated.
Step 2, integrating object detection models
2.1 dynamic Loading model
In the existing object detection model based on deep learning, once the model training is finished, the target class solidified in the object detection model cannot be changed any more unless the model training is carried out again, so that the application flexibility of the object detection model is reduced, and the model reusability is poor.
2.2. Multi-model fusion algorithm
The method aims to solve the problems that the existing deep learning-based object detection algorithm cannot perform class customization modification after training is completed and has poor reusability. In the invention, a method for fusing a plurality of models is invented, opencv is used for carrying out loading analysis on different series of models (yolo, ssd, fast rcnn), and forward prediction is carried out on an input image to obtain intensive frames and class output; then, adding output boxes of different models to a list; and finally, inputting all the intensive prediction frames into a non-maximum value suppression module to obtain an optimal object detection result.
According to a second aspect, the present invention also provides a computer-readable storage medium, on which a computer program is stored, the computer program being executable by a processor to implement the steps of the method for dynamic fusion of multiple models in deep learning object detection according to the present invention.
The invention has the beneficial effects that:
the method comprises the steps of dynamically appointing an object detection model for fusion in a configuration file appointing mode, loading the appointed object detection model by using a dnn module in opencv, carrying out forward reasoning and prediction on pictures to obtain an intensive target detection prediction frame, and carrying out post-processing on the intensive prediction frame by using a non-maximum suppression algorithm to output an object detection result. The dynamic fusion of different series deep learning object detection models (a Yolo series, an SSD series and an Rcnn series) is realized, the reusability of the deep learning models is increased, and the overall performance of the object detection models is greatly improved.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
FIG. 2 is a dynamically loaded object detection model according to a configuration file.
Detailed Description
As shown in fig. 1, a method for dynamic fusion of multiple models in deep learning object detection includes the following steps:
step 1, object detection model acquisition
The model used in the present invention can come from two aspects:
(1) The model obtained by training the object detection model based on deep learning through the existing data set has high customization degree, and can control the classes contained in the model according to the specific application scene;
(2) The method has the greatest characteristic that an existing object detection model is directly integrated, a new training object detection network is not needed, application can be quickly formed, and the method has the defects that customization cannot be realized, and the types contained in the model cannot be changed according to needs.
Step 2, integrating object detection models
The basic idea of the invention is to promote a weak-performance object detection model to a strong-performance object detection model. Weak performing object detectors can achieve slightly better results than random choices in machine learning theory, while strong object detectors can be very close to optimal object detectors. In actual object detection work, various object detection models such as Yolo, fast rcnn and ssd can be obtained according to labeling data, under the condition that model training fine tuning is not performed, the detection performance obtained by each type of model has no significant difference, and under the condition that the model complexity is equivalent, the performance indexes of the Yolo series object detection models on a COCO2017val data set are shown in table 1.
TABLE 1 comparison of model Performance in the yolo series
| Model (model) | YOLOv5-L | YOLOv6-L | YOLOv7-L |
| Size of image | 640 | 640 | 640 |
| APval | 49％ | 52.5％ | 51.2％ |
Under the condition that the single model performance does not have a lifting space, the direction for lifting the overall model identification performance is multi-model integration.
2.1 dynamic Loading model
In the existing object detection model based on deep learning, once the model training is finished, the target class solidified in the object detection model cannot be changed any more unless the model training is carried out again, so that the application flexibility of the object detection model is reduced, and the model reusability is poor. In the present invention, we use configuration files to specify the model that the fusion algorithm needs to load as shown in fig. 2, for example: model 1 can detect A, B, C, D, model 2 can detect E, F, G, model 3 can detect H, I, and configuration file config.
2.2. Multi-model fusion algorithm
The method aims to solve the problems that the existing deep learning-based object detection algorithm cannot perform class customization modification after training is completed and has poor reusability. In the invention, a method for fusing a plurality of models is invented, opencv is used for carrying out loading analysis on different series of models (yolo, ssd, fast rcnn), and forward prediction is carried out on an input image to obtain intensive frames and class output; then, adding output boxes of different models to a list; and finally, inputting all the intensive prediction frames into a non-maximum value suppression module to obtain an optimal object detection result.
Claims (3)
1. A multi-model dynamic fusion method in deep learning object detection is characterized by comprising the following steps:
step 1, object detection model acquisition
Training an object detection model based on deep learning through an existing data set to obtain a model; or directly integrating the existing object detection model;
step 2, dynamically loading model object detection model integration by using a configuration file;
step 3, a multi-model fusion algorithm comprises the following steps:
(1) Loading and analyzing different series of models, and performing forward prediction on an input image to obtain intensive frames and class output;
(2) Adding output boxes of different models to a list;
(3) And inputting all the intensive prediction frames into a non-maximum suppression module to obtain an optimal object detection result.
2. The method for multi-model dynamic fusion in deep learning object detection according to claim 1, characterized in that:
the object detection models include yolo, ssd, and fast rcnn.
3. The method for multi-model dynamic fusion in deep learning object detection according to claim 1, wherein in step (3):
and loading the specified object detection model by using a dnn module in opencv.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211301088.1A CN115661595A (en) | 2022-10-24 | 2022-10-24 | Multi-model dynamic fusion method in deep learning object detection and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211301088.1A CN115661595A (en) | 2022-10-24 | 2022-10-24 | Multi-model dynamic fusion method in deep learning object detection and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115661595A true CN115661595A (en) | 2023-01-31 |
Family
ID=84990444
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211301088.1A Pending CN115661595A (en) | 2022-10-24 | 2022-10-24 | Multi-model dynamic fusion method in deep learning object detection and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115661595A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115938546A (en) * | 2023-02-21 | 2023-04-07 | 四川大学华西医院 | Early gastric cancer image synthesis method, system, equipment and storage medium |
-
2022
- 2022-10-24 CN CN202211301088.1A patent/CN115661595A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115938546A (en) * | 2023-02-21 | 2023-04-07 | 四川大学华西医院 | Early gastric cancer image synthesis method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111462126B (en) | Semantic image segmentation method and system based on edge enhancement | |
| WO2023138300A1 (en) | Target detection method, and moving-target tracking method using same | |
| CN107533754B (en) | Reducing image resolution in deep convolutional networks | |
| CN104182772B (en) | A kind of gesture identification method based on deep learning | |
| CN111191583B (en) | Space target recognition system and method based on convolutional neural network | |
| CN109961107B (en) | Training method and device for target detection model, electronic equipment and storage medium | |
| Tan et al. | Novel initialization scheme for Fuzzy C-Means algorithm on color image segmentation | |
| El-Sayed et al. | Automated edge detection using convolutional neural network | |
| CN111145188B (en) | Image segmentation method based on ResNet and UNet models | |
| CN110991311A (en) | Target detection method based on dense connection deep network | |
| US9418440B2 (en) | Image segmenting apparatus and method | |
| US20220157041A1 (en) | Image classification method and apparatus | |
| CN110148117B (en) | Power equipment defect identification method and device based on power image and storage medium | |
| CN110008853B (en) | Pedestrian detection network and model training method, detection method, medium and equipment | |
| CN115661595A (en) | Multi-model dynamic fusion method in deep learning object detection and storage medium | |
| CN112131933A (en) | Rapid pedestrian detection method and system based on improved YOLO network | |
| Ma et al. | Towards no-reference image quality assessment based on multi-scale convolutional neural network | |
| CN111898560A (en) | Classification regression feature decoupling method in target detection | |
| WO2024078112A1 (en) | Method for intelligent recognition of ship outfitting items, and computer device | |
| Wang et al. | A review of object detection based on convolutional neural networks and deep learning | |
| CN112614199B (en) | Semantic segmentation image conversion method, device, computer equipment and storage medium | |
| CN111127393B (en) | Sample making method and system for radar image change detection, storage medium and equipment | |
| Singh et al. | Fotonnet: A hw-efficient object detection system using 3d-depth segmentation and 2d-dnn classifier | |
| Tsai et al. | Real-time automatic multilevel color video thresholding using a novel class-variance criterion | |
| Sumahasan et al. | Object Detection using Deep Learning Algorithm CNN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |