CN115760843A - Defect detection model training method and device, terminal device and storage medium - Google Patents
Defect detection model training method and device, terminal device and storage medium Download PDFInfo
- Publication number
- CN115760843A CN115760843A CN202211579114.7A CN202211579114A CN115760843A CN 115760843 A CN115760843 A CN 115760843A CN 202211579114 A CN202211579114 A CN 202211579114A CN 115760843 A CN115760843 A CN 115760843A
- Authority
- CN
- China
- Prior art keywords
- target
- detection
- detection model
- network
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 378
- 230000007547 defect Effects 0.000 title claims abstract description 96
- 238000012549 training Methods 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000003860 storage Methods 0.000 title claims abstract description 12
- 230000011218 segmentation Effects 0.000 claims abstract description 101
- 230000002950 deficient Effects 0.000 claims abstract description 34
- 238000000605 extraction Methods 0.000 claims description 24
- 238000002372 labelling Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 238000004891 communication Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 10
- 238000007689 inspection Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
Abstract
The application relates to the technical field of defect detection, in particular to a training method, a training device, terminal equipment and a storage medium of a defect detection model, and can solve the problems of poor precision and low generalization capability of the detection model to industrial visual detection to a certain extent. Establishing an industrial detection data set according to various detection images of industrial detection according to a preset rule; the method can realize training of a backbone network of the initial detection model based on an industrial detection data set; determining target detection prediction data and semantic segmentation prediction data of a first defective target by a first target to be detected data set with the first defective target through an alternative detection model, wherein the alternative detection model comprises a backbone network, a target detection network and a semantic segmentation network; further, a target detection model corresponding to the first defective target may be determined based on the target detection prediction data and the semantic segmentation prediction data; and a target detection model with strong generalization capability and high precision is quickly realized through a small amount of defective target data.
Description
Technical Field
The application relates to the technical field of defect detection, in particular to a training method and device of a defect detection model, terminal equipment and a storage medium.
Background
With the development of deep learning (deep learning) in the visual field, data sets such as ImageNet data set (1400 million image data) and JFT-300M data set (3 hundred million image data) are generally adopted as pre-training models in image processing; these known data sets are image data in natural environments, cover most of the life picture categories, and have a large difference from the imaging characteristics of industrial visual images.
However, in the implementation of industrial vision inspection, it is difficult to collect negative samples (defect data), so that the aspects of model accuracy, generalization capability and the like of the inspection of industrial vision images by using the pre-trained model corresponding to the ImageNet data set are limited, and the method is not suitable for industrial vision inspection.
Disclosure of Invention
In order to solve the problems of poor precision and low generalization capability of a model deeply learned through a known data set to industrial visual inspection, the application provides a training method and device of a defect detection model, terminal equipment and a storage medium.
The embodiment of the application is realized as follows:
a first aspect of an embodiment of the present application provides a method for training a defect detection model, including the following steps:
training a backbone network of an initial detection model based on an industrial detection data set, wherein the industrial detection data set is established by various detection images of industrial detection according to a preset rule;
determining target detection prediction data and semantic segmentation prediction data of a first defect target by a first target data set to be detected through an alternative detection model; the candidate detection model is determined based on a backbone network, a target detection network and a semantic segmentation network, a first target data set to be detected is an image with a first defect target, and the number of the images of the first target data set to be detected is smaller than that of the images of the industrial detection data set;
and determining a target detection model corresponding to the first defect target based on the target detection prediction data and the semantic segmentation prediction data, wherein the target detection model comprises a backbone network, an updated target detection network and an updated semantic segmentation network.
A second aspect of the embodiments of the present application provides a training apparatus for a defect detection model, which includes a first training module, a second training module, and an updating module; wherein,
the first training module is used for training a backbone network of the initial detection model based on an industrial detection data set, wherein the industrial detection data set is established by various detection images of industrial detection according to a preset rule;
the second training module is used for determining target detection prediction data and semantic segmentation prediction data of the first defect target through the alternative detection model by the first target data set to be detected; the alternative detection model is determined based on a backbone network, a target detection network and a semantic segmentation network, a first target data set to be detected is an image with a first defect target, and the number of the images of the first target data set to be detected is smaller than that of the images of the industrial detection data set;
and the updating module is used for determining a target detection model corresponding to the first defect target based on the target detection prediction data and the semantic segmentation prediction data, and the target detection model comprises a backbone network, an updated target detection network and an updated semantic segmentation network.
A third aspect of the embodiments of the present application provides a terminal device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the method for training the defect detection model of the first aspect in the disclosure when executing the computer program.
A fourth aspect of the embodiments of the present application provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program causes the processor to execute the steps of the training method for the defect detection model of the first aspect in the disclosure.
The beneficial effects of the application are that; establishing an industrial detection data set for various detection images of industrial detection according to a preset rule; the method can realize training of a backbone network of an initial detection model based on an industrial detection data set; determining target detection prediction data and semantic segmentation prediction data of a first defective target by a first target to be detected data set with the first defective target through an alternative detection model, wherein the alternative detection model comprises a backbone network, a target detection network and a semantic segmentation network; further, a target detection model corresponding to the first defective target may be determined based on the target detection prediction data and the semantic segmentation prediction data, the target detection model including a backbone network, an updated target detection network, and an updated semantic segmentation network; and a target detection model with strong generalization capability and high precision can be quickly realized through a small amount of defect target data.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
FIG. 1 is a flow chart illustrating a training method of a defect detection model according to an embodiment of the present application;
FIG. 2 is a schematic flow chart illustrating training of an initial detection model according to an embodiment of the present application;
FIG. 3 is a network architecture diagram illustrating an initial detection model according to an embodiment of the present application;
FIG. 4 is a flow chart illustrating a training method of a defect detection model according to an embodiment of the present application;
FIG. 5 shows a flow diagram for providing alternative detection model determination according to an embodiment of the present application;
FIG. 6 is a network architecture diagram illustrating an alternative detection model according to an embodiment of the present application;
FIG. 7 is a network architecture diagram illustrating yet another alternative detection model according to an embodiment of the present application;
FIG. 8 is a flow chart illustrating the acquisition of a first target to be detected data set provided by the embodiment of the present application;
FIG. 9 is a schematic flow chart illustrating determination of an object detection model provided in an embodiment of the present application;
FIG. 10 is a flow chart illustrating a training method for a defect detection model according to an embodiment of the present application;
FIG. 11 is a flow chart illustrating a training method for a defect detection model according to an embodiment of the present application;
fig. 12 shows a schematic structural diagram of a training apparatus for a defect detection model according to an embodiment of the present application.
Detailed Description
To make the objects, embodiments and advantages of the present application clearer, the following description of exemplary embodiments of the present application will clearly and completely describe the exemplary embodiments of the present application with reference to the accompanying drawings in the exemplary embodiments of the present application, and it is to be understood that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.
It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.
The terms "first," "second," "third," and the like in the description and claims of this application and in the foregoing drawings are used for distinguishing between similar or analogous objects or entities and are not necessarily intended to limit the order or sequence in which they are presented unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.
The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.
ImageNet data sets (1400 million image data), JFT-300M data sets (3 hundred million image data) and the like are commonly used data sets in the current image processing application field, the data sets are image data in natural environment, most of picture categories in life are covered by huge image data, and the data sets are used as parameters of a pre-training model for initialization, so that the generalization capability of the model is improved.
However, in the field of industrial visual detection, the imaging characteristics of the image have a large difference from those of data such as an ImageNet data set and a JFT-300M data set, and some industrial image data are microscopic imaging data, and if a model is trained by using these known data sets, the ability of deep learning of the model is reduced, and the model accuracy and generalization ability of the model for detecting the industrial visual image are insufficient. The image data in the industrial detection aims at various defect types under the conditions of various machines, various industrial camera environments, various industrial product backgrounds and the like.
In combination with the conditions of an actual industrial generation environment, difficulty in acquiring defect data, short online period of each detection model and the like, a training method for a fast-reading iterative defect detection model is needed to meet the requirement of industrial production on defect detection.
In order to solve the problems of poor precision and low generalization capability of a known data set deep learning model to industrial visual inspection, the embodiment of the application provides a training method, a device, terminal equipment and a storage medium of a defect detection model, and an industrial detection data set is established by various detection images of industrial detection according to a preset rule; the method can realize training of a backbone network of an initial detection model based on an industrial detection data set; determining target detection prediction data and semantic segmentation prediction data of a first defective target by a first target to be detected data set with the first defective target through an alternative detection model, wherein the alternative detection model comprises a backbone network, a target detection network and a semantic segmentation network; further, a target detection model corresponding to the first defective target may be determined based on the target detection prediction data and the semantic segmentation prediction data, the target detection model including a backbone network, an updated target detection network, and an updated semantic segmentation network; and a target detection model with strong generalization capability and high precision can be quickly realized through a small amount of defective target data.
The following describes a training method and apparatus, a terminal device, and a storage medium for a defect detection model according to embodiments of the present application in detail with reference to the accompanying drawings.
Fig. 1 shows a schematic flow chart of a training method for a defect detection model provided in an embodiment of the present application, and as shown in fig. 1, the embodiment of the present application provides a training method for a defect detection model.
The training method of the defect detection model comprises the following steps:
s110, training a backbone network of the initial detection model based on an industrial detection data set, wherein the industrial detection data set is established by various detection images of industrial detection according to preset rules.
The image data under natural environment such as ImageNet data set (1400 million image data), JFT-300M data set (3 hundred million image data) and the like can be used for establishing an industrial detection data set suitable for industrial detection environment through various detection images of industrial detection in order to improve the model deep learning capability in the industrial detection field, and the industrial detection data set is based on the existing various industrial detection images.
The detection model is trained by establishing the industrial detection data set, the generalization capability of the model is improved, namely, the detection equipment learns the rule behind each data in the industrial detection data set by deeply learning the adaptive capability of the industrial detection data set, and the trained network model can also provide corresponding defect detection output for data except the learning set with the same rule.
Fig. 2 shows a schematic flowchart of a process of training an initial detection model according to an embodiment of the present application, and as shown in fig. 2, step 110 is to train a backbone network of the initial detection model based on an industrial detection data set, where the industrial detection data set is created by using various detection images of industrial detection according to a preset rule, and includes:
s101, establishing an industrial detection data set, wherein the industrial detection data set divides the existing detection image into M major classes and N minor classes according to a preset rule.
Existing inspection images are existing image data that can cover a variety of inspection tools, a variety of industrial camera environments, a variety of industrial product backgrounds, and a variety of defect types. The existing detection images can be subjected to target detection labeling and semantic segmentation labeling based on targets needing to be detected, so that the images can be divided into M major classes and N minor classes according to a preset plan. Wherein the preset rule is determined based on the product qualification requirement of the industrial defect detection.
For example, an Industrial test dataset may be defined as Big Industrial data sets, BID for short; the industrial detection data set can comprise 50 major categories such as mobile phone frames, display screens, earphones, printing and wearing equipment, and 500 minor categories such as camera modules, inner cavities, charging ports, earphone ports, SIM card slots, short edges and long edges, and the total data set comprises 100 ten thousand data volumes.
And S102, training an initial detection model based on the industrial detection data set.
And pre-training the existing model through the established industrial detection data set to determine an initial detection model, particularly pre-training a backbone network of the initial detection model. The initial detection model comprises a backbone network and an initial prediction network.
Taking the efficiency rep in yolov6 as an example of a Backbone network, fig. 3 shows a network structure diagram of an initial detection model in the embodiment of the present application, and as shown in fig. 3, the initial detection model includes a Backbone network (Backbone) and an initial prediction network (classification head).
The backbone network is a network for extracting features, and is used for extracting information in picture data of the industrial detection data set for later network use. The backbone network is usually a network designed in an existing model, and frequently used are resnet, VGG and the like, and the backbone networks already have strong feature extraction capability. The initial detection model is obtained by replacing the model parameters of the backbone networks with an industrial detection data set and pre-training the existing model based on the industrial detection data set.
The classification head is a network for acquiring network output content, and makes prediction by using features extracted from a backbone network. In the industrial defect detection process, a small amount of target detection image data to be detected is required to further build and train an initial detection model. The specific process is as follows:
and S120, determining target detection prediction data and semantic segmentation prediction data of the first defective target through the first target data set to be detected through the alternative detection model.
The alternative detection model comprises a backbone network, a target detection network and a semantic segmentation network.
Fig. 4 shows a flowchart of a training method for a defect detection model provided by an embodiment of the present application, as shown in fig. 4, before the first target data set to be detected passes through the alternative detection model to determine target detection prediction data and semantic segmentation prediction data of the first defect target in step 120, the method further includes:
and S121, determining an alternative detection model based on the initial detection model, the target detection network and the semantic segmentation network.
Aiming at the environmental limitations of large difficulty and long period of field collection of industrial data sets, a target detection network and a semantic segmentation network are selected in the aspect of model selection, the required labeling information is richer, and the dependence on data volume can be reduced.
Fig. 5 shows a schematic flowchart of a process for determining an alternative detection model provided in an embodiment of the present application, and as shown in fig. 5, in step 121, the alternative detection model is determined based on an initial detection model, a target detection network, and a semantic segmentation network, which specifically includes the following steps:
s2101, an initial prediction network connected with an output end of a backbone network in the initial detection model is replaced by a feature extraction structure, and the feature extraction structure is used for performing feature fusion on a feature map in the backbone network.
S2102, determining an alternative detection model based on the backbone network, the feature extraction structure, the target detection network and the semantic segmentation network.
The target detection network and the semantic segmentation network respectively have application characteristics and advantages and disadvantages. The target detection network has disadvantages in the aspects of detection of small targets, targets with special angles and targets with special shapes, but can detect targets with certain overlapped areas; the semantic segmentation network is difficult to process for multi-target overlapping regions, but can carry out pixel-level classification on targets with arbitrary shapes.
The output end of the backbone network is in communication connection with the input end of the feature extraction structure; the output end of the feature extraction structure is in communication connection with the input end of the target detection network and the input end of the semantic segmentation network.
Continuing to take the initial detection model shown in fig. 3 as an example, fig. 6 shows a network structure diagram of an alternative detection model in the embodiment of the present application, and fig. 7 shows a network structure diagram of another alternative detection model in the embodiment of the present application, it should be noted that the alternative detection model includes a target detection network and a semantic segmentation network, and for clarity of display, the two diagrams shown in fig. 6 and fig. 7 are separated, as shown in fig. 6 and fig. 7, the alternative detection model includes a Backbone network (Backbone), a feature extraction structure (Neck), a target detection network (detection head), and a semantic segmentation network (segmentation head).
The Neck function is to perform feature fusion on feature graphs of different dimensions in the Backbone to obtain three scale features of P3, P4 and P5, and the fused features are used as input of a target detection network and a semantic segmentation network.
Before S120, a first target data set to be detected corresponding to the first target to be detected is also acquired. Wherein the first target data set to be detected is an image with a first defect target, and the number of images of the first target data set to be detected is smaller than that of the industrial detection data set.
It should be appreciated that the number of images of the first target dataset to be inspected may be derived from a smaller number of images than the industrial inspection dataset. For example, the number of images of the first target data set to be detected may be 100.
In order to further improve the detection performance, the embodiment of the application realizes further training of the alternative detection model by reducing the first target data set to be detected corresponding to the first target to be detected, which is required by model convergence.
It should be understood that, in the application of industrial defect detection, not only the first target to be detected but also the second target to be detected, i.e. the pth target to be detected, may be included, that is, the present application may further train the candidate detection model according to the data set of the small amount of image data corresponding to each target to be detected.
For example, taking scratch detection of the mobile phone outer frame as a first target to be detected as an example, a scratch image of the relevant mobile phone outer frame and a good product data image without scratch need to be acquired, and the number of the acquired scratch images is only 100 or more.
Fig. 8 is a schematic flow chart illustrating the first target to be detected data set acquisition provided in the embodiment of the present application, and as shown in fig. 8, the first target to be detected data set acquisition specifically includes the following steps:
s2201, performing target detection labeling and semantic segmentation labeling on the image data corresponding to the first target to be detected.
S2202, performing data enhancement on the marked image data to determine a first target data set to be detected.
Data enhancement performed on the annotated image data may include, but is not limited to, copy-paste, mosaic data enhancement.
And step 120 is executed by combining the establishment of the alternative detection model and the acquisition of the first target data set to be detected, wherein the first target data set to be detected determines target detection prediction data and semantic segmentation prediction data of the first defective target through the alternative detection model.
A first target data set to be detected enters a feature extraction structure through a backbone network of an alternative detection model, fused features are output, and target detection prediction data of a first defect target are determined through a target detection network; and the first target data set to be detected enters the feature extraction structure through the backbone network of the alternative detection model, the fused features are output, and the semantic segmentation prediction data of the first defective target is determined through the semantic segmentation network.
For example, in the determination of the target detection prediction data, the detection head is as shown in fig. 7, the finally obtained feature dimension is 8400 × n +5, n represents the defect class number, 5 represents the x, y, w, h coordinate position and width and height of the rectangular box, and the target confidence level.
The segmentation head and the detection head are two parallel branch networks which can be operated in parallel, and the two branches share the backhaul and the hack.
As shown in fig. 1, the method further includes: s130, determining a target detection model corresponding to the first defect target based on the target detection prediction data and the semantic segmentation prediction data, wherein the target detection model comprises a backbone network, an updated target detection network and an updated semantic segmentation network.
Fig. 9 shows a schematic flowchart of a process of determining an object detection model provided in the embodiment of the present application, and as shown in fig. 9, S130 determines an object detection model corresponding to a first defective object based on object detection prediction data and semantic segmentation prediction data, which specifically includes the following steps:
s301, determining target detection loss data based on the target detection prediction data.
S302, semantic segmentation loss data are determined based on the semantic segmentation prediction data.
And S303, carrying out inverse gradient propagation on the sum of the target detection loss data and the semantic segmentation loss data, and determining the updating data.
And S304, updating the target detection network and the semantic segmentation network based on the updating data.
In some embodiments, the target detection model further comprises an updated feature extraction structure.
Fig. 10 is a flowchart illustrating a training method of a defect detection model provided in an embodiment of the present application, and as shown in fig. 10, after determining, in step 130, a target detection model corresponding to a first defective target based on target detection prediction data and semantic segmentation prediction data, the training method of a defect detection model further includes:
s140, determining that the target detection model comprises an updated target detection network or an updated semantic segmentation network or both based on the preset detection requirement of the first defective target.
It should be appreciated that the trained object detection model can be used for detecting a first defect object (e.g., a scratch on the outer frame of a mobile phone) in practical applications.
And determining that the target detection model comprises an updated target detection network, or an updated semantic segmentation network, or both based on the preset detection requirement of the first defective target. It should be understood that only one of the dual branches may be selected for defect detection, for example, if only the defect position needs to be located, only the target detection network branch may be selected, and if the area size of the scratch area needs to be known, the semantic segmentation network branch may also be selected.
Fig. 11 shows a schematic flowchart of a training method for a defect detection model provided in an embodiment of the present application, and as shown in fig. 11, the training method for a defect detection model is applied to model training of three defect targets, which are a first defect target, a second defect target, and a third defect target, and the training method for a defect detection model specifically includes the following steps:
and S210, establishing an industrial detection data set.
S220, training a backbone network of the initial detection model based on the industrial detection data set.
And S231, acquiring image data of the first target to be detected aiming at the first defect target. S232, performing target detection labeling and semantic segmentation labeling on the image data corresponding to the first target to be detected. And S233, performing data enhancement on the marked image data to determine a first target data set to be detected. S234, determining target detection prediction data and semantic segmentation prediction data of the first defect target through the first target data set to be detected through the alternative detection model. S235, determining a target detection model corresponding to the first defect target based on the target detection prediction data and the semantic segmentation prediction data. And S236, deploying a target detection model based on the preset detection requirement of the first defect target.
And S241, acquiring image data of a second target to be detected aiming at the second defect target. And S242, performing target detection labeling and semantic segmentation labeling on the image data corresponding to the second target to be detected. And S243, performing data enhancement on the marked image data, and determining a second target data set to be detected. And S244, determining target detection prediction data and semantic segmentation prediction data of the second defect target by the second target data set to be detected through the alternative detection model. And S245, determining a target detection model corresponding to the second defect target based on the target detection prediction data and the semantic segmentation prediction data. And S246, deploying a target detection model based on the preset detection requirement of the second defect target.
And S251, acquiring image data of a third target to be detected aiming at the third defective target. And S252, performing target detection labeling and semantic segmentation labeling on the image data corresponding to the third target to be detected. And S253, performing data enhancement on the marked image data, and determining a third target data set to be detected. And S254, determining target detection prediction data and semantic segmentation prediction data of a third defect target by the third target data set to be detected through the alternative detection model. And S255, determining a target detection model corresponding to the third defective target based on the target detection prediction data and the semantic segmentation prediction data. And S256, deploying a target detection model based on the preset detection requirement of the third defect target.
It should be understood that the number of each detection item is not limited to three as described in the above embodiments, and may also be one or another number, which is determined according to the defect detection requirement, and each detection item updates the corresponding candidate detection model according to its corresponding target data set to be detected, so as to obtain the corresponding target detection model.
The embodiment of the application provides a training method of a defect detection model, which comprises the steps of establishing an industrial detection data set according to preset rules for various detection images of industrial detection; the method can realize training of a backbone network of an initial detection model based on an industrial detection data set; determining target detection prediction data and semantic segmentation prediction data of a first defective target by a first target to be detected data set with the first defective target through an alternative detection model, wherein the alternative detection model comprises a backbone network, a target detection network and a semantic segmentation network; further, a target detection model corresponding to the first defective target may be determined based on the target detection prediction data and the semantic segmentation prediction data, the target detection model including a backbone network, an updated target detection network, and an updated semantic segmentation network; and a target detection model with strong generalization capability and high precision can be quickly realized through a small amount of defect target data.
Fig. 12 is a schematic structural diagram of a training apparatus for a defect detection model according to an embodiment of the present application, and as shown in fig. 12, the training apparatus 1200 for a defect detection model includes a first training module 1210, a second training module 1220, and an updating module 1230.
The first training module is used for training a backbone network of an initial detection model based on an industrial detection data set, wherein the industrial detection data set is established by various detection images of industrial detection according to a preset rule;
the second training module is used for determining target detection prediction data and semantic segmentation prediction data of the first defect target through the alternative detection model by the first target data set to be detected; the alternative detection model comprises a backbone network, a target detection network and a semantic segmentation network, wherein a first target data set to be detected is an image with a first defect target, and the number of the images of the first target data set to be detected is smaller than that of the images of the industrial detection data set;
and the updating module is used for determining a target detection model corresponding to the first defect target based on the target detection prediction data and the semantic segmentation prediction data, and the target detection model comprises a backbone network, an updated target detection network and an updated semantic segmentation network.
In some embodiments, the update module further comprises a data processing unit and an update unit, wherein,
a data processing unit for determining target detection loss data based on the target detection prediction data.
And the data processing unit is also used for determining semantic segmentation loss data based on the semantic segmentation prediction data.
And the data processing unit is also used for carrying out inverse gradient propagation on the sum of the target detection loss data and the semantic segmentation loss data to determine the updating data.
And the updating unit is used for updating the target detection network and the semantic segmentation network based on the updating data.
In some embodiments, the training apparatus for a defect detection model further includes a model building module, where the model building module:
replacing an initial prediction network connected with the output end of the backbone network in the initial detection model with a feature extraction structure, wherein the feature extraction structure is used for performing feature fusion on a feature map in the backbone network;
and determining an alternative detection model based on the backbone network, the feature extraction structure, the target detection network and the semantic segmentation network.
In some embodiments, the updating unit is further configured to update the feature extraction structure to obtain the updated feature extraction structure.
In some embodiments, the model building module is further configured to:
the output end of the backbone network is in communication connection with the input end of the feature extraction structure;
the output end of the feature extraction structure is in communication connection with the input end of the target detection network and the input end of the semantic segmentation network.
In some embodiments, the training apparatus for the defect inspection model further comprises a target data obtaining module, configured to:
performing target detection labeling and semantic segmentation labeling on image data corresponding to a first target to be detected;
and performing data enhancement on the marked image data to determine a first target data set to be detected.
In some embodiments, the training apparatus for the defect detection model further includes a deployment module, and the deployment module is configured to:
and determining that the target detection model comprises an updated target detection network, or an updated semantic segmentation network, or both based on the preset detection requirement of the first defective target.
The embodiment of the application provides a training device of a defect detection model, which comprises a first training module, a second training module and an updating module; establishing an industrial detection data set for various detection images of industrial detection according to a preset rule; the method can realize training of a backbone network of an initial detection model based on an industrial detection data set; determining target detection prediction data and semantic segmentation prediction data of a first defective target by a first target data set to be detected with the first defective target through an alternative detection model, wherein the alternative detection model comprises a backbone network, a target detection network and a semantic segmentation network; further, a target detection model corresponding to the first defective target may be determined based on the target detection prediction data and the semantic segmentation prediction data, the target detection model including a backbone network, an updated target detection network, and an updated semantic segmentation network; and a target detection model with strong generalization capability and high precision can be quickly realized through a small amount of defect target data.
The terminal device further provided in this embodiment of the present application includes a memory and a processor, where the memory stores a computer program, and the processor executes the computer program, where the computer program is used to implement the method for training the defect detection model, and the implementation principle and technical effects are similar to those of the method embodiment and are not described herein again.
The embodiment of the present application further provides a computer storage medium, where a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to perform the method for training the defect detection model, which is similar to the method embodiment in the foregoing implementation principle and technical effect and is not described herein again.
The following paragraphs will provide a comparative listing of Chinese terms and their corresponding English terms referred to in this application for ease of reading and understanding.
The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the foregoing discussion in some embodiments is not intended to be exhaustive or to limit the implementations to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.
Claims (10)
1. A training method of a defect detection model is characterized by comprising the following steps:
training a backbone network of an initial detection model based on an industrial detection data set, wherein the industrial detection data set is established by various detection images of industrial detection according to a preset rule;
determining target detection prediction data and semantic segmentation prediction data of a first defect target by a first target data set to be detected through an alternative detection model; wherein the alternative detection model is determined based on the backbone network, an object detection network and a semantic segmentation network, the first target data set to be detected is an image of a first defect object, and the number of images of the first target data set to be detected is smaller than that of the industrial detection data set;
and determining a target detection model corresponding to the first defect target based on the target detection prediction data and the semantic segmentation prediction data, wherein the target detection model comprises the backbone network, an updated target detection network and an updated semantic segmentation network.
2. The method for training the defect detection model according to claim 1, wherein the determining the target detection model corresponding to the first defective target based on the target detection prediction data and the semantic segmentation prediction data comprises:
determining target detection loss data based on the target detection prediction data;
determining semantic segmentation loss data based on the semantic segmentation prediction data;
transmitting the sum of the target detection loss data and the semantic segmentation loss data in a reverse gradient manner, and determining updating data;
updating the target detection network and the semantic segmentation network based on the update data.
3. The method for training the defect detection model according to claim 1, wherein before determining the target detection prediction data and the semantic segmentation prediction data of the first defective target by the candidate detection model, the first target data set to be detected further comprises:
replacing an initial prediction network connected with the output end of the backbone network in the initial detection model with a feature extraction structure, wherein the feature extraction structure is used for performing feature fusion on a feature map in the backbone network;
and determining the alternative detection model based on the backbone network, the feature extraction structure, the target detection network and the semantic segmentation network.
4. The method for training the defect detection model according to claim 3, wherein in the step of determining the target detection model corresponding to the first defective target based on the target detection prediction data and the semantic segmentation prediction data, the target detection model further comprises an updated feature extraction structure.
5. The method for training the defect detection model according to claim 3, wherein determining the candidate detection model based on the backbone network, the feature extraction structure, the target detection network, and the semantic segmentation network comprises:
the output end of the backbone network is in communication connection with the input end of the feature extraction structure;
and the output end of the feature extraction structure is in communication connection with the input end of the target detection network and the input end of the semantic segmentation network.
6. The method for training the defect detection model according to claim 1, wherein before determining the target detection prediction data and the semantic segmentation prediction data of the first defective target by the candidate detection model, the first target data set to be detected further comprises:
performing target detection labeling and semantic segmentation labeling on the image data corresponding to the first target to be detected;
and performing data enhancement on the marked image data to determine the first target data set to be detected.
7. The method for training the defect detection model according to claim 1, wherein after determining the target detection model corresponding to the first defective target based on the target detection prediction data and the semantic segmentation prediction data, the method further comprises:
and determining that the target detection model comprises an updated target detection network or an updated semantic segmentation network or both based on the preset detection requirement of the first defective target.
8. A training device for a defect detection model is characterized by comprising:
the first training module is used for training a backbone network of an initial detection model based on an industrial detection data set, wherein the industrial detection data set is established by various detection images of industrial detection according to a preset rule;
the second training module is used for determining target detection prediction data and semantic segmentation prediction data of the first defect target through the alternative detection model by the first target data set to be detected; the alternative detection model is determined based on the backbone network, a target detection network and a semantic segmentation network, the first target data set to be detected is an image of a target with a first defect, and the number of images of the first target data set to be detected is smaller than that of the industrial detection data set;
and the updating module is used for determining a target detection model corresponding to the first defective target based on the target detection prediction data and the semantic segmentation prediction data, wherein the target detection model comprises the backbone network, an updated target detection network and an updated semantic segmentation network.
9. A terminal device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the training method of the defect detection model according to any one of claims 1 to 7.
10. A computer storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when being executed by a processor, causes the processor to carry out the steps of the method of training a defect detection model according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211579114.7A CN115760843A (en) | 2022-12-06 | 2022-12-06 | Defect detection model training method and device, terminal device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211579114.7A CN115760843A (en) | 2022-12-06 | 2022-12-06 | Defect detection model training method and device, terminal device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115760843A true CN115760843A (en) | 2023-03-07 |
Family
ID=85346613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211579114.7A Pending CN115760843A (en) | 2022-12-06 | 2022-12-06 | Defect detection model training method and device, terminal device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115760843A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116468725A (en) * | 2023-06-13 | 2023-07-21 | 北京航空航天大学杭州创新研究院 | Industrial defect detection method, device and storage medium based on pre-training model |
-
2022
- 2022-12-06 CN CN202211579114.7A patent/CN115760843A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116468725A (en) * | 2023-06-13 | 2023-07-21 | 北京航空航天大学杭州创新研究院 | Industrial defect detection method, device and storage medium based on pre-training model |
| CN116468725B (en) * | 2023-06-13 | 2023-09-05 | 北京航空航天大学杭州创新研究院 | Industrial defect detection method, device and storage medium based on pre-training model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109493346B (en) | Stomach cancer pathological section image segmentation method and device based on multiple losses | |
| CN110379020B (en) | Laser point cloud coloring method and device based on generation countermeasure network | |
| CN106548169B (en) | Fuzzy literal Enhancement Method and device based on deep neural network | |
| KR102190527B1 (en) | Apparatus and method for automatic synthesizing images | |
| US20210118140A1 (en) | Deep model training method and apparatus, electronic device, and storage medium | |
| CN110619605B (en) | Parallel meteorological element downscaling method and device, electronic equipment and storage medium | |
| CN112149547A (en) | Remote sensing image water body identification based on image pyramid guidance and pixel pair matching | |
| CN111738036B (en) | Image processing method, device, equipment and storage medium | |
| CN111950528A (en) | Chart recognition model training method and device | |
| CN113487610B (en) | Herpes image recognition method and device, computer equipment and storage medium | |
| CN110599453A (en) | Panel defect detection method and device based on image fusion and equipment terminal | |
| CN115131797A (en) | Scene text detection method based on feature enhancement pyramid network | |
| CN117152484B (en) | Small target cloth flaw detection method based on improved YOLOv5s | |
| CN113159043A (en) | Feature point matching method and system based on semantic information | |
| CN115760843A (en) | Defect detection model training method and device, terminal device and storage medium | |
| CN116310745B (en) | Image processing method, data processing method, related device and storage medium | |
| CN113469997A (en) | Method, device, equipment and medium for detecting plane glass | |
| CN114429636B (en) | Image scanning identification method and device and electronic equipment | |
| CN112365513A (en) | Model training method and device | |
| Meng et al. | A mobilenet-SSD model with FPN for waste detection | |
| CN113223011B (en) | Small sample image segmentation method based on guide network and full-connection conditional random field | |
| CN113112511B (en) | Method and device for correcting test paper, storage medium and electronic equipment | |
| CN116563840B (en) | Scene text detection and recognition method based on weak supervision cross-mode contrast learning | |
| CN115147597A (en) | Positioning method and device of photovoltaic module and electronic equipment | |
| CN112580750A (en) | Image recognition method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |