CN118135206A - Hierarchical detection method, system, equipment and medium for semi-supervised learning - Google Patents
Hierarchical detection method, system, equipment and medium for semi-supervised learning Download PDFInfo
- Publication number
- CN118135206A CN118135206A CN202410572052.XA CN202410572052A CN118135206A CN 118135206 A CN118135206 A CN 118135206A CN 202410572052 A CN202410572052 A CN 202410572052A CN 118135206 A CN118135206 A CN 118135206A
- Authority
- CN
- China
- Prior art keywords
- data set
- positioning
- label
- patch
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 95
- 238000000034 method Methods 0.000 claims description 24
- 230000007547 defect Effects 0.000 claims description 22
- 238000002372 labelling Methods 0.000 claims description 15
- 239000010410 layer Substances 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 238000003860 storage Methods 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 230000004807 localization Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 239000002356 single layer Substances 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 description 7
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000009966 trimming Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 229910000679 solder Inorganic materials 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/06—Recognition of objects for industrial automation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a hierarchical detection method for semi-supervised learning, which comprises the steps of designing a hierarchical detection model to obtain preliminary positioning and classification; self-adaptively generating a label-free training data set for fine adjustment through a positioning network model in a hierarchical detection model, wherein the data set comprises target information to be detected of the positioning model and background information with noise, and performing label matching on the generated label-free training data set through maximum cross-over ratio and similarity, so that labels are distributed to the background information and the target information to be detected, and a training data set with the distributed labels is generated; the decoupling classification head of the hierarchical detection model is finely adjusted by generating a training data set with distributed labels, so that the robustness of the network is enhanced; the invention can establish a hierarchical detection network, and effectively enhance the robustness of a detection model based on a semi-supervised fine tuning strategy, and can be used for various detection tasks.
Description
Technical Field
The invention relates to the technical field of intelligent manufacturing automation detection, in particular to a hierarchical detection method, system, equipment and medium for semi-supervised learning.
Background
Target detection is a common task in the industry and is widely used in the fields of production quality management, defect detection and automated production. A high-precision detection model can effectively ensure the reliability of a production quality evaluation system.
With the advent of deep learning and artificial intelligence technology, industrial detection techniques have generally employed convolutional neural networks to obtain advanced features for locating targets to be detected. However, due to the black box mode of the deep learning, when the deep learning is adopted to detect the target, the problem that defects exist in the positioning detection model cannot be intuitively found, so that unpredictable risks are caused in practical application. The invention provides a hierarchical detection method for semi-supervised learning, which can detect a target to be detected with high precision, and can enable a network to self-adaptively find defects existing in the network, thereby improving the robustness and the interpretability of a model.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a hierarchical detection method of semi-supervised learning, which solves the problems that the traditional defect detection framework based on deep learning is weak in interpretation, and potential network defects are difficult to adjust once training is completed; according to the invention, by constructing the hierarchical detection model and purposefully designing a fine adjustment strategy based on semi-supervised learning for the hierarchical detection model, high-precision target detection is realized, the robustness of the detection model is effectively enhanced, and the hierarchical detection model can be used for various detection tasks.
In order to solve the technical problems, the invention provides the following technical scheme: a hierarchical detection method for semi-supervised learning includes the following steps:
S1, designing a hierarchical detection model, and detecting a target to be detected by respectively designing a positioning network model and a classification network model to obtain preliminary positioning and classification;
s2, adaptively generating a label-free training data set for fine adjustment through a positioning network model in the hierarchical detection model The data set comprises target information to be detected of a positioning network model and background information with noise, and is stored in the form of a patch;
s3, generating a label-free training data set Label matching is carried out through the maximum cross-over ratio and the similarity, so that labels are distributed to background information and target information to be detected, and a training data set/>, after labels are distributed, is generated；
S4, generating a training data set with assigned labelsAnd fine tuning is carried out on the decoupling classification head of the hierarchical detection model, so that the robustness of the network is enhanced.
Further, in step S1, the hierarchical detection model includes two stages, including a positioning stage and a classifying stage, the positioning stage adopts a pre-trained positioning network model, and is responsible for outputting positioning information of all objects to be detected on the image data, and storing the positioning information in the form of 32×32 pixel small patches, the patches include positioning information output by the positioning network model, the information is sent to the classifying network model in the classifying task, the model is responsible for classifying each patch, the output patch includes positioning information of the objects to be detected and background information with noise, and then fine tuning is performed on the classifying network model based on the background information with noise and the positioning information, so as to filter noise in the background information output by the positioning network model.
Further, in step S2, a label-free training data set for fine tuning is adaptively generated by means of a localization model in the hierarchical detection modelThe specific process comprises the following steps:
S201, preparing an original data set Labeling the target to be detected, wherein the label contains the position information of the target to be detected and the category information of the target to be detected;
S202, labeling the original data set Inputting the data into a positioning network model designed in the S1, and generating a large quantity of unlabeled data sets/>The data set contains target information to be detected of the positioning model and background information with noise, wherein the background information with noise is regarded as defect information of the positioning model, and the data of the data set is stored as 32 x 32 small blocks patch.
Further, in step S3, for the generated unlabeled training data setThe label matching is carried out through the maximum cross-over ratio and the similarity, and the specific process comprises the following steps:
S301, because the patch generated in S2 is unlabeled, generating an unlabeled training data set Each patch of the plurality is associated with a base dataset/>Each annotation frame on the corresponding image in (a) is matched, namely when the ith small block/>In the basic dataset/>Extracted from the j-th image in the list, then the/>Matching with all the labeling frames in the image is needed;
S302, in the matching process, the two indexes of the maximum cross ratio and the similarity are weighted to obtain a final matching result, and the matching result is calculated as follows:
；
Wherein Th represents the matching result of the patch of the label to be allocated and the labeling frame of the corresponding image, Weight coefficient representing maximum cross-over ratio,/>The weight coefficient representing the similarity, SIM represents the similarity of Gaussian distribution of a patch and a labeling frame of a corresponding image, GIOU represents the regularized maximum intersection ratio, and the calculation formula is as follows:
；
wherein IOU represents the maximum cross-over ratio between the patch and the label box of the corresponding image, Representing the minimum matrix area of a label box containing a patch of tiles and corresponding images,/>Representing the sum of the areas between the patch and the callout box, SIM (A, B) represents the calculation formula that solves the similarity between patch A and callout box B:
；
Wherein A and B represent the coordinate information of the patch A and the label frame B in the original image data, Representation/>Regularization;
S303, after calculating the matching result Th, combining Th with a set threshold value Comparing, if Th is larger than threshold, marking the label of the patch as label frame label of the corresponding image, if Th is smaller than/>The label of the patch is noted as background, and the formula is as follows:
。
further, in step S4, a training data set with assigned labels is generated Fine tuning the decoupling classification head of the hierarchical detection model, wherein the specific process comprises the following steps:
S401, for the generated data set after label distribution The data is adopted to enhance a decoupling classifier, wherein the decoupling classifier comprises a single-layer attention mechanism and a layer of fully-connected neural network, and a generated data set/>, after labels are distributed, is generatedAfter the decoupling classifier is input, the microscopic features of the data are extracted through an attention mechanism, classification results are given through a full-connection layer, and the fine tuning process only adjusts the classification result output layer of the decoupling classifier, namely the full-connection neural network, so that parameters of the attention layer are frozen;
S402, before fine tuning, the classification head of the classifier is provided with N outputs which are respectively used for outputting N categories of targets, in the fine tuning process, the classification head of the classifier is set to be N+1, besides the targets of the N categories can be output, whether the input small patch is a background can be judged, if the input small patch is judged to be the background, the classification head judges that the small patch is the error information output by the positioning model, and the robustness of the positioning model is enhanced by filtering the background information.
Furthermore, the present invention also provides a hierarchical detection system for semi-supervised learning, which includes: the hierarchical detection model construction module is used for detecting targets to be detected by respectively designing a positioning network model and a classification network model to obtain preliminary positioning and classification; a data set generation module for adaptively generating a label-free training data set for fine tuning by a positioning network model in a hierarchical detection modelThe data set comprises target information to be detected of a positioning model and background information with noise, and is stored in the form of a patch; a label matching module for matching the generated label-free training data setLabel matching is carried out through the maximum cross-over ratio and the similarity, so that labels are distributed to background information and target information to be detected, and a training data set/>, after labels are distributed, is generated; Decoupling classifier trimming module that generates training data set/>, after label assignment, by generating training data set/>And fine tuning is carried out on the decoupling classification head of the hierarchical detection model, so that the robustness of the network is enhanced.
Further, the present invention also provides an electronic device, including: the system comprises a memory and a processor, wherein the memory is used for storing a computer program, and the processor runs the computer program to enable the electronic equipment to execute the hierarchical detection method of the semi-supervised learning.
Furthermore, the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the hierarchical detection method of the semi-supervised learning when being executed by a processor.
By means of the technical scheme, the invention provides a hierarchical detection method for semi-supervised learning, which has at least the following beneficial effects:
The invention can strengthen the robustness of the detection network by carrying out semi-supervised training fine adjustment on the hierarchical detection model, can lead the noise possibly existing in the self-adaptive prediction positioning network model of the decoupling classifier, strengthen the denoising capability of the self-adaptive prediction positioning network model by fine adjustment, filter the noise generated by the positioning network model, strengthen the robustness of the hierarchical detection model, and solve the problems that the traditional defect detection frame based on deep learning has weak interpretability, potential network defects are difficult to adjust once training is finished, and the like.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a hierarchical detection flow chart of semi-supervised learning proposed by the present invention;
FIG. 2 is a schematic diagram of a hierarchical detection framework for semi-supervised learning according to the present invention;
fig. 3 is a block diagram of a hierarchical detection system for semi-supervised learning provided by the present invention.
In the figure: 501. the hierarchical detection model building module; 502. a data set generation module; 503. a tag matching module; 504. and a decoupling classifier fine tuning module.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description. Therefore, the realization process of how to apply the technical means to solve the technical problems and achieve the technical effects can be fully understood and implemented.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in a method of implementing an embodiment described above may be implemented by a program to instruct related hardware, and thus, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Referring to fig. 1-3, a specific implementation manner of the present embodiment is shown, as shown in fig. 2, in this embodiment, by taking solder joint defect detection of an integrated circuit as an example, and showing a hierarchical detection manner proposed in this patent, by using a manner based on semi-supervised fine tuning, a positioning network model of a hierarchical detection model outputs solder joint information and background information with noise on the integrated circuit, and adaptively marks the background information as noise, so that a network adaptively discovers potential defects of itself, and adjusts parameters of a network structure based on defect information, thereby implementing fine tuning on the hierarchical detection model, and finally filtering potential noise and defects in the network model; the robustness of the hierarchical detection model to the detection task is improved, and the network training strategy is suitable for multiple types of detection network training; the method solves the problems that the defect detection framework based on deep learning is poor in interpretability, and potential network defects are difficult to adjust once training is completed.
Referring to fig. 1, the present embodiment provides a hierarchical detection method for semi-supervised learning, which includes the following steps:
S1, designing a hierarchical detection model, and detecting a target to be detected by respectively designing a positioning network model and a classification network model to obtain preliminary positioning and classification;
As a preferred implementation manner of the step S1, in the step S1, the hierarchical detection model includes two stages, namely a positioning stage and a classification stage, the positioning stage adopts a pre-trained positioning network model, is responsible for outputting positioning information of all objects to be detected on image data, and is stored in the form of 32×32 pixel small patches, the patches include the positioning information output by the positioning network model, and the information is sent to the classification network model in the classification task, and the model is responsible for classifying each patch;
It is noted that, because the detection precision of the positioning model is lower than that of the training positioning model, the patch output by the positioning model comprises the positioning information of the target to be detected and the background information with noise, and then the classification network model is finely adjusted based on the background information with noise and the positioning information, so that the noise in the background information output by the positioning network model is filtered.
S2, adaptively generating a label-free training data set for fine adjustment through a positioning network model in the hierarchical detection modelThe data set comprises target information to be detected of a positioning network model and background information with noise, and is stored in the form of a patch;
As a preferred embodiment of step S2, the label-free training data set for fine tuning is adaptively generated by means of a localization model in the hierarchical detection model The specific process comprises the following steps:
S201, preparing an original data set Labeling the target to be detected, wherein the label contains the position information of the target to be detected and the category information of the target to be detected;
S202, labeling the original data set Inputting the data into a positioning network model designed in the S1, and generating a large quantity of unlabeled data sets/>The data set contains target information to be detected of the positioning model and background information with noise, wherein the background information with noise is regarded as defect information of the positioning model, and the data of the data set is stored as 32 x 32 small blocks patch.
S3, generating a label-free training data setLabel matching is carried out through the maximum cross-over ratio and the similarity, so that labels are distributed to background information and target information to be detected, and a training data set/>, after labels are distributed, is generated；
As a preferred embodiment of step S3, for the generated unlabeled training data setThe label matching is carried out through the maximum cross-over ratio and the similarity, and the specific process comprises the following steps:
S301, because the patch generated in S2 is unlabeled, generating an unlabeled training data set Each patch of the plurality is associated with a base dataset/>Each annotation frame on the corresponding image in (a) is matched, namely when the ith small block/>In the basic dataset/>Extracted from the j-th image in the list, then the/>Matching with all the labeling frames in the image is needed;
S302, in the matching process, the two indexes of the maximum cross ratio and the similarity are weighted to obtain a final matching result, and the matching result is calculated as follows:
；
Wherein Th represents the matching result of the patch of the label to be allocated and the labeling frame of the corresponding image, Weight coefficient representing maximum cross-over ratio,/>The weight coefficient representing the similarity, SIM represents the similarity of Gaussian distribution of a patch and a labeling frame of a corresponding image, GIOU represents the regularized maximum intersection ratio, and the calculation formula is as follows:
；
wherein IOU represents the maximum cross-over ratio between the patch and the label box of the corresponding image, Representing the minimum matrix area of a label box containing a patch of tiles and corresponding images,/>Representing the sum of the areas between the patch and the callout box, SIM (A, B) represents the calculation formula that solves the similarity between patch A and callout box B:
；
Wherein A and B represent the coordinate information of the patch A and the label frame B in the original image data, Representation/>Regularization;
S303, after calculating the matching result Th, combining Th with a set threshold value Comparing, if Th is larger than threshold, marking the label of the patch as label frame label of the corresponding image, if Th is smaller than/>The label of the patch is noted as background, and the formula is as follows:
。
S4, generating a training data set with assigned labels And fine tuning is carried out on the decoupling classification head of the hierarchical detection model, so that the robustness of the network is enhanced.
As a preferred embodiment of step S4, the training data set after the label assignment is generatedFine tuning the decoupling classification head of the hierarchical detection model, wherein the specific process comprises the following steps:
S401, for the generated data set after label distribution The data is adopted to enhance a decoupling classifier, wherein the decoupling classifier comprises a single-layer attention mechanism and a layer of fully-connected neural network, and a generated data set/>, after labels are distributed, is generatedAfter the decoupling classifier is input, the microscopic features of the data are extracted through an attention mechanism, classification results are given through a full-connection layer, and the fine tuning process only adjusts the classification result output layer of the decoupling classifier, namely the full-connection neural network, so that parameters of the attention layer are frozen;
S402, before fine tuning, the classification head of the classifier is provided with N outputs which are respectively used for outputting N categories of targets, in the fine tuning process, the classification head of the classifier is set to be N+1, besides the targets of the N categories can be output, whether the input small patch is a background can be judged, if the input small patch is judged to be the background, the classification head judges that the small patch is the error information output by the positioning model, and the robustness of the positioning model is enhanced by filtering the background information.
Furthermore, the present invention also provides a hierarchical detection system for semi-supervised learning, which includes: the hierarchical detection model construction module is used for detecting targets to be detected by respectively designing a positioning network model and a classification network model to obtain preliminary positioning and classification; a data set generation module for adaptively generating a label-free training data set for fine tuning by a positioning network model in a hierarchical detection modelThe data set comprises target information to be detected of a positioning model and background information with noise, and is stored in the form of a patch; a label matching module for matching the generated label-free training data setLabel matching is carried out through the maximum cross-over ratio and the similarity, so that labels are distributed to background information and target information to be detected, and a training data set/>, after labels are distributed, is generated; Decoupling classifier trimming module that generates training data set/>, after label assignment, by generating training data set/>And fine tuning is carried out on the decoupling classification head of the hierarchical detection model, so that the robustness of the network is enhanced.
Specifically, the invention also provides a hierarchical detection system for semi-supervised learning, which comprises: the hierarchical detection model construction module 501 is configured to detect a target to be detected by respectively designing a positioning network model and a classification network model to obtain preliminary positioning and classification; a data set generation module 502, wherein the data set generation module 502 is configured to adaptively generate a label-free training data set for fine tuning through a positioning network model in a hierarchical detection modelThe data set comprises target information to be detected of a positioning model and background information with noise, and is stored in the form of a patch; a label matching module 503, wherein the label matching module 503 is used for generating label-free training data set/>Label matching is carried out through the maximum cross-over ratio and the similarity, so that labels are distributed to background information and target information to be detected, and a training data set/>, after labels are distributed, is generated; A decoupled classifier trimming module 504, the decoupled classifier trimming module 504 generates a training dataset/>, after label assignment, by generating a training dataset/>And fine tuning is carried out on the decoupling classification head of the hierarchical detection model, so that the robustness of the network is enhanced.
Specifically, the invention further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the hierarchical detection method of semi-supervised learning when being executed by a processor.
The method has the advantages that the construction of a hierarchical detection model and the training based on semi-supervised learning are realized, potential defects of a network are found in a self-adaptive mode based on semi-supervised fine tuning, parameters of a network structure are adjusted based on defect information, potential noise and defects of the network can be filtered, and the problems that the defect detection frame based on deep learning is poor in interpretation, potential network defects are difficult to adjust once training is completed and the like are solved.
In summary, the robustness of the hierarchical detection model to the detection task can be improved by the semi-supervised training method of the hierarchical detection model, and the network training strategy is suitable for multiple types of detection network training.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
The foregoing embodiments have been presented in a detail description of the invention, and are presented herein with a particular application to the understanding of the principles and embodiments of the invention, the foregoing embodiments being merely intended to facilitate an understanding of the method of the invention and its core concepts; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (8)
1. The hierarchical detection method for semi-supervised learning is characterized by comprising the following steps of:
S1, designing a hierarchical detection model, and detecting a target to be detected by respectively designing a positioning network model and a classification network model to obtain preliminary positioning and classification;
s2, adaptively generating a label-free training data set for fine adjustment through a positioning network model in the hierarchical detection model The data set comprises target information to be detected of a positioning network model and background information with noise, and is stored in the form of a patch;
s3, generating a label-free training data set Label matching is carried out through the maximum cross-over ratio and the similarity, so that labels are distributed to background information and target information to be detected, and a training data set/>, after labels are distributed, is generated；
S4, generating a training data set with assigned labelsAnd fine tuning is carried out on the decoupling classification head of the hierarchical detection model, so that the robustness of the network is enhanced.
2. The hierarchical detection method for semi-supervised learning as set forth in claim 1, wherein: in step S1, the hierarchical detection model includes two stages, namely a positioning stage and a classifying stage, the positioning stage adopts a pre-trained positioning network model, and is responsible for outputting positioning information of all targets to be detected on image data, and storing the positioning information in the form of 32×32 pixel small patches, the patches include positioning information output by the positioning network model, the information is sent to the classifying network model in the classifying task, the model is responsible for classifying each patch, the output patch includes positioning information of the targets to be detected and background information with noise, and then fine tuning is performed on the classifying network model based on the background information with noise and the positioning information, so as to filter noise in the background information output by the positioning network model.
3. The hierarchical detection method for semi-supervised learning as set forth in claim 1, wherein: in step S2, a label-free training dataset for fine tuning is adaptively generated by a localization model in the hierarchical detection modelThe specific process comprises the following steps:
S201, preparing an original data set Labeling the target to be detected, wherein the label contains the position information of the target to be detected and the category information of the target to be detected;
S202, labeling the original data set Inputting the data into a positioning network model designed in the S1, and generating a large quantity of unlabeled data sets/>The data set contains target information to be detected of the positioning model and background information with noise, wherein the background information with noise is regarded as defect information of the positioning model, and the data of the data set is stored as 32 x 32 small blocks patch.
4. The hierarchical detection method for semi-supervised learning as set forth in claim 1, wherein: in step S3, for the generated unlabeled training data setThe label matching is carried out through the maximum cross-over ratio and the similarity, and the specific process comprises the following steps:
S301, because the patch generated in S2 is unlabeled, generating an unlabeled training data set Each patch of the plurality is associated with a base dataset/>Each annotation frame on the corresponding image in (a) is matched, namely when the ith small block/>In the basic dataset/>Extracted from the j-th image in the list, then the/>Matching with all the labeling frames in the image is needed;
S302, in the matching process, the two indexes of the maximum cross ratio and the similarity are weighted to obtain a final matching result, and the matching result is calculated as follows:
；
Wherein Th represents the matching result of the patch of the label to be allocated and the labeling frame of the corresponding image, Weight coefficient representing maximum cross-over ratio,/>The weight coefficient representing the similarity, SIM represents the similarity of Gaussian distribution of a patch and a labeling frame of a corresponding image, GIOU represents the regularized maximum intersection ratio, and the calculation formula is as follows:
；
wherein IOU represents the maximum cross-over ratio between the patch and the label box of the corresponding image, Representing the minimum matrix area of a label box containing a patch of tiles and corresponding images,/>Representing the sum of the areas between the patch and the callout box, SIM (A, B) represents the calculation formula that solves the similarity between patch A and callout box B:
；
Wherein A and B represent the coordinate information of the patch A and the label frame B in the original image data, Representation/>Regularization;
S303, after calculating the matching result Th, combining Th with a set threshold value Comparing, if Th is larger than threshold, marking the label of the patch as label frame label of the corresponding image, if Th is smaller than/>The label of the patch is noted as background, and the formula is as follows:
。
5. the hierarchical detection method for semi-supervised learning as set forth in claim 1, wherein: in step S4, a training data set with assigned labels is generated Fine tuning the decoupling classification head of the hierarchical detection model, wherein the specific process comprises the following steps:
S401, for the generated data set after label distribution The data is adopted to enhance a decoupling classifier, wherein the decoupling classifier comprises a single-layer attention mechanism and a layer of fully-connected neural network, and a generated data set/>, after labels are distributed, is generatedAfter the decoupling classifier is input, the microscopic features of the data are extracted through an attention mechanism, classification results are given through a full-connection layer, and the fine tuning process only adjusts the classification result output layer of the decoupling classifier, namely the full-connection neural network, so that parameters of the attention layer are frozen;
S402, before fine tuning, the classification head of the classifier is provided with N outputs which are respectively used for outputting N categories of targets, in the fine tuning process, the classification head of the classifier is set to be N+1, besides the targets of the N categories can be output, whether the input small patch is a background can be judged, if the input small patch is judged to be the background, the classification head judges that the small patch is the error information output by the positioning model, and the robustness of the positioning model is enhanced by filtering the background information.
6. A hierarchical detection system for semi-supervised learning, comprising:
The hierarchical detection model construction module is used for detecting targets to be detected by respectively designing a positioning network model and a classification network model to obtain preliminary positioning and classification;
A data set generation module for adaptively generating a label-free training data set for fine tuning by a positioning network model in a hierarchical detection model The data set comprises target information to be detected of a positioning model and background information with noise, and is stored in the form of a patch;
a label matching module for matching the generated label-free training data set Label matching is carried out through the maximum cross-over ratio and the similarity, so that labels are distributed to background information and target information to be detected, and a training data set/>, after labels are distributed, is generated；
The decoupling classifier fine tuning module is used for generating a training data set with assigned labelsAnd fine tuning is carried out on the decoupling classification head of the hierarchical detection model, so that the robustness of the network is enhanced.
7. An electronic device, comprising: a memory for storing a computer program, and a processor that runs the computer program to cause the electronic device to perform the semi-supervised learning hierarchical detection method of any of claims 1-5.
8. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the hierarchical detection method of semi-supervised learning of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410572052.XA CN118135206B (en) | 2024-05-10 | 2024-05-10 | Hierarchical detection method, system, equipment and medium for semi-supervised learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410572052.XA CN118135206B (en) | 2024-05-10 | 2024-05-10 | Hierarchical detection method, system, equipment and medium for semi-supervised learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118135206A true CN118135206A (en) | 2024-06-04 |
| CN118135206B CN118135206B (en) | 2024-08-02 |
Family
ID=91242052
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410572052.XA Active CN118135206B (en) | 2024-05-10 | 2024-05-10 | Hierarchical detection method, system, equipment and medium for semi-supervised learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118135206B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220261599A1 (en) * | 2021-02-18 | 2022-08-18 | Irida Labs S.A. | Annotating unlabeled images using convolutional neural networks |
| WO2022237153A1 (en) * | 2021-05-14 | 2022-11-17 | 上海商汤智能科技有限公司 | Target detection method and model training method therefor, related apparatus, medium, and program product |
| CN117726855A (en) * | 2023-11-15 | 2024-03-19 | 联合汽车电子有限公司 | Target detection method, machine vision device, storage medium and controller |
-
2024
- 2024-05-10 CN CN202410572052.XA patent/CN118135206B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220261599A1 (en) * | 2021-02-18 | 2022-08-18 | Irida Labs S.A. | Annotating unlabeled images using convolutional neural networks |
| WO2022237153A1 (en) * | 2021-05-14 | 2022-11-17 | 上海商汤智能科技有限公司 | Target detection method and model training method therefor, related apparatus, medium, and program product |
| CN117726855A (en) * | 2023-11-15 | 2024-03-19 | 联合汽车电子有限公司 | Target detection method, machine vision device, storage medium and controller |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118135206B (en) | 2024-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106971152B (en) | Method for detecting bird nest in power transmission line based on aerial images | |
| CN109919934B (en) | Liquid crystal panel defect detection method based on multi-source domain deep transfer learning | |
| CN110084299B (en) | Target detection method and device based on multi-head fusion attention | |
| CN110781262B (en) | Semantic map construction method based on visual SLAM | |
| CN112434586B (en) | Multi-complex scene target detection method based on domain self-adaptive learning | |
| CN112528845B (en) | Physical circuit diagram identification method based on deep learning and application thereof | |
| CN111914720B (en) | Method and device for identifying insulator burst of power transmission line | |
| CN114022586B (en) | Defect image generation method based on countermeasure generation network | |
| CN109087294A (en) | A kind of product defects detection method, system and computer readable storage medium | |
| CN117576379B (en) | Target detection method based on meta-learning combined attention mechanism network model | |
| CN116385958A (en) | Edge intelligent detection method for power grid inspection and monitoring | |
| CN116266387A (en) | YOLOV4 image recognition algorithm and system based on re-parameterized residual error structure and coordinate attention mechanism | |
| CN116258990A (en) | Cross-modal affinity-based small sample reference video target segmentation method | |
| Zhao et al. | Adaptive vision inspection for multi-type electronic products based on prior knowledge | |
| CN112884135B (en) | Data annotation correction method based on frame regression | |
| CN118135206B (en) | Hierarchical detection method, system, equipment and medium for semi-supervised learning | |
| CN117576453A (en) | Cross-domain armored target detection method, system, electronic equipment and storage medium | |
| CN117495786A (en) | Defect detection meta-model construction method, defect detection method, device and medium | |
| CN116894113A (en) | Data security classification method and data security management system based on deep learning | |
| CN115204318B (en) | Event automatic hierarchical classification method and electronic equipment | |
| CN116524296A (en) | Training method and device of equipment defect detection model and equipment defect detection method | |
| CN110647917A (en) | Model multiplexing method and system | |
| CN116091784A (en) | Target tracking method, device and storage medium | |
| CN115601610A (en) | Fabric flaw detection method based on improved EfficientDet model | |
| CN112287938B (en) | Text segmentation method, system, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |