CN116563641A - Surface defect identification method and system based on small target detection - Google Patents
Surface defect identification method and system based on small target detection Download PDFInfo
- Publication number
- CN116563641A CN116563641A CN202310624084.5A CN202310624084A CN116563641A CN 116563641 A CN116563641 A CN 116563641A CN 202310624084 A CN202310624084 A CN 202310624084A CN 116563641 A CN116563641 A CN 116563641A
- Authority
- CN
- China
- Prior art keywords
- defect
- module
- sets
- recognition
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 230000007547 defect Effects 0.000 title claims abstract description 258
- 238000001514 detection method Methods 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000004458 analytical method Methods 0.000 claims abstract description 107
- 238000000605 extraction Methods 0.000 claims description 55
- 238000012549 training Methods 0.000 claims description 26
- 239000013598 vector Substances 0.000 claims description 23
- 238000012795 verification Methods 0.000 claims description 15
- 238000012360 testing method Methods 0.000 claims description 14
- 238000013507 mapping Methods 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 11
- 238000012545 processing Methods 0.000 abstract description 8
- 230000006870 function Effects 0.000 description 10
- 238000010276 construction Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013501 data transformation Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/54—Extraction of image or video features relating to texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a surface defect identification method and a system based on small target detection, and relates to the technical field of defect identification, wherein the method comprises the following steps: acquiring a real-time image set; extracting features to obtain an image feature set; carrying out small target recognition, and obtaining a plurality of recognition area sets according to recognition results; carrying out loss analysis on the plurality of identification area sets by using a softmax loss function, judging whether a preset threshold is met according to a loss value in a loss analysis result, and if so, setting the plurality of identification area sets as target area sets; obtaining a plurality of target area videos; outputting a plurality of surface defect sets; the method has the advantages that the surface defect identification result set is obtained, the technical problems that tiny defects are difficult to detect and the defect identification accuracy is poor due to poor processing effect on defect images in the prior art are solved, and the technical effect of improving the defect identification accuracy is achieved.
Description
Technical Field
The invention relates to the technical field of defect identification, in particular to a surface defect identification method and system based on small target detection.
Background
The defect detection is an important program in the production and manufacturing industry, can also be used for equipment maintenance management, can effectively prevent the output of unqualified products by performing defect identification, and has important reference significance for equipment maintenance. Most of defect recognition depends on image processing, but sometimes the defective area is too small, so that defects are difficult to detect after image processing.
In summary, the prior art has the technical problems that the processing effect on the defect image is poor, so that the micro defect is difficult to detect, and the defect identification accuracy is poor.
Disclosure of Invention
The invention provides a surface defect identification method and a system based on small target detection, which are used for solving the technical problems that in the prior art, small defects are difficult to detect and the defect identification accuracy is poor due to poor processing effect on defect images.
According to a first aspect of the present invention, there is provided a surface defect recognition method based on small target detection, comprising: collecting real-time images of a target detection object to obtain a real-time image set; preprocessing the real-time image set, and extracting features of the preprocessed real-time image set by using a feature extraction model to obtain an image feature set; performing small target recognition on the image feature set by using a classifier, and obtaining a plurality of recognition area sets according to recognition results; carrying out loss analysis on the plurality of identification area sets by using a softmax loss function, judging whether a preset threshold is met according to a loss value in a loss analysis result, and if so, setting the plurality of identification area sets as target area sets; video acquisition is carried out on the target area set in a preset time window, and a plurality of target area videos are obtained; inputting the videos of the multiple target areas into a defect recognition module in a defect recognition model, and outputting multiple surface defect sets, wherein the defect recognition module is constructed based on a slow network; and inputting the plurality of surface defect sets into a defect matching module in the defect recognition model to obtain a surface defect recognition result set.
According to a second aspect of the present invention, there is provided a surface defect recognition system based on small object detection, comprising: the real-time image acquisition module is used for acquiring a real-time image of the target detection object to obtain a real-time image set; the feature extraction module is used for preprocessing the real-time image set, and extracting features of the preprocessed real-time image set by using a feature extraction model to obtain an image feature set; the small target recognition module is used for carrying out small target recognition on the image feature set by using a classifier, and a plurality of recognition area sets are obtained according to recognition results; the loss analysis module is used for carrying out loss analysis on the plurality of identification area sets by using a softmax loss function, judging whether a preset threshold is met according to a loss value in a loss analysis result, and if so, setting the plurality of identification area sets as target area sets; the video acquisition module is used for acquiring videos of the target area set in a preset time window to obtain videos of a plurality of target areas; the surface defect set output module is used for inputting the videos of the plurality of target areas into a defect recognition module in a defect recognition model to output a plurality of surface defect sets, wherein the defect recognition module is constructed based on a slow network; the defect recognition result acquisition module is used for inputting the plurality of surface defect sets into the defect matching module in the defect recognition model to obtain the surface defect recognition result set.
According to the surface defect identification method based on small target detection, the real-time image of the target detection object is acquired to obtain a real-time image set; preprocessing the real-time image set, and extracting features of the preprocessed real-time image set by using a feature extraction model to obtain an image feature set; performing small target recognition on the image feature set by using a classifier, and obtaining a plurality of recognition area sets according to recognition results; carrying out loss analysis on the plurality of identification area sets by using a softmax loss function, judging whether a preset threshold is met according to a loss value in a loss analysis result, and if so, setting the plurality of identification area sets as target area sets; video acquisition is carried out on the target area set in a preset time window, and a plurality of target area videos are obtained; inputting the videos of the multiple target areas into a defect recognition module in a defect recognition model, and outputting multiple surface defect sets, wherein the defect recognition module is constructed based on a slow network; and inputting the plurality of surface defect sets into a defect matching module in a defect recognition model to obtain a surface defect recognition result set, thereby achieving the technical effects of improving the image processing effect and further improving the defect recognition accuracy.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following brief description will be given of the drawings used in the description of the embodiments or the prior art, it being obvious that the drawings in the description below are only exemplary and that other drawings can be obtained from the drawings provided without the inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a surface defect recognition method based on small target detection according to an embodiment of the present invention;
FIG. 2 is a flow chart of obtaining an image feature set according to an embodiment of the present invention;
FIG. 3 is a flow chart of obtaining a surface defect set output by a defect recognition layer according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a surface defect recognition system based on small target detection according to an embodiment of the present invention.
Reference numerals illustrate: the system comprises a real-time image acquisition module 11, a feature extraction module 12, a small target identification module 13, a loss analysis module 14, a video acquisition module 15, a surface defect set output module 16 and a defect identification result acquisition module 17.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In order to solve the technical problems of difficult detection of tiny defects and poor defect identification accuracy caused by poor processing effect on defect images in the prior art, the inventor obtains the surface defect identification method and system based on small target detection through creative labor.
Example 1
Fig. 1 is a diagram of a surface defect recognition method based on small target detection according to an embodiment of the present invention, as shown in fig. 1, where the method includes:
step S100: collecting real-time images of a target detection object to obtain a real-time image set;
specifically, the target detection object refers to any type of element to be subjected to surface defect detection, such as an automobile part, steel, a pipeline and the like, and a real-time image of the target detection object is acquired through equipment such as an intelligent camera and the like to obtain a real-time image set.
Step S200: preprocessing the real-time image set, and extracting features of the preprocessed real-time image set by using a feature extraction model to obtain an image feature set;
as shown in fig. 2, step S200 of the embodiment of the present invention includes:
step S210: constructing the feature extraction model based on a convolutional neural network, wherein the feature extraction model comprises an input layer, a texture feature analysis layer, a shape feature analysis layer and an output layer;
step S220: inputting the preprocessed real-time image set into a texture feature analysis layer in the feature extraction model to obtain a texture feature vector set;
step S230: inputting the preprocessed real-time image set into a shape feature analysis layer in the feature extraction model to obtain a shape feature vector set;
step S240: and obtaining an image feature set according to the texture feature vector set and the shape feature vector set.
Specifically, the real-time image set is subjected to preprocessing, and the preprocessing includes denoising, smoothing, enhancement and other operations, such as median filtering, mean filtering and the like. And further, feature extraction is carried out on the preprocessed real-time image set by using a feature extraction model to obtain an image feature set, wherein the feature extraction model is a convolutional neural network model in machine learning, and is obtained by obtaining historical data for supervision training and verification. The feature is a recognizable object in the image, and the feature extraction is performed on the recognizable object to obtain an image feature set.
Specifically, the feature extraction model is constructed based on a convolutional neural network, and features in historical data can be marked, so that the feature extraction model is trained and tested as training data, and a feature extraction model meeting requirements is obtained. The feature extraction model comprises an input layer, a texture feature analysis layer, a shape feature analysis layer and an output layer, wherein the texture feature analysis layer and the shape feature analysis layer are connected in parallel, that is, the preprocessed real-time image set is required to be respectively input into the texture feature analysis layer and the shape feature analysis layer in the feature extraction model to obtain a texture feature vector set and a shape feature vector set, the texture feature is a global feature and reflects the visual feature of a homogeneity phenomenon in an image, the surface tissue structure arrangement attribute with slow transformation or periodical change of the surface of an object is reflected, and the texture feature vector can be represented by the gray scale or pixel distribution of pixels and the surrounding space neighborhood; the shape feature vector may be represented by a contour of the identifiable object, where the real-time image set includes a plurality of images, and each image may have a plurality of identifiable objects, so that a plurality of texture feature vectors may be obtained to form a texture feature vector set, and a plurality of shape feature vectors may be obtained to form a shape feature vector set. And obtaining an image feature set according to the texture feature vector set and the shape feature vector set, wherein the image feature set comprises a shape and an appearance texture, and providing data support for subsequent defect recognition by obtaining the image feature set.
Step S300: performing small target recognition on the image feature set by using a classifier, and obtaining a plurality of recognition area sets according to recognition results;
specifically, the two classifiers can only output two results, for example, only 0 and 1, and in this embodiment, the two classifiers are used for identifying a small target and a large target, specifically, a threshold value can be set for the contour area of the shape of the feature according to the shape feature vector set in the image feature set, the image feature with the contour area exceeding the threshold value is the large target, and the image feature with the contour area less than or equal to the threshold value is the small target. The small target is a tiny object in an image, the image may be blurred when the tiny image is enlarged, and the resolution is not high, so that the image gray value or the pixel value distribution of the small target is different from that of the large target, a pixel threshold or a gray threshold is set based on the small target, the small target recognition is carried out on the image feature set by taking the pixel threshold or the gray threshold as a classification standard of a two-classifier, a plurality of recognition area sets are obtained according to the recognition result, and the plurality of recognition area sets comprise a plurality of small targets and a plurality of large targets.
Step S400: carrying out loss analysis on the plurality of identification area sets by using a softmax loss function, judging whether a preset threshold is met according to a loss value in a loss analysis result, and if so, setting the plurality of identification area sets as target area sets;
specifically, the softmax loss function is constructed based on the overfunction-log, and if the probability value of correct classification is closer to 1, the classification effect is good, and the corresponding loss function-log is close to 0; if the probability of correct classification is closer to 0, the classification effect is poor, and the corresponding loss function-log is larger. That is, if more regions are identified correctly in the plurality of identified region sets, the corresponding loss value should be smaller, the closer to 0, the better.
Wherein, softmax loss function is:
wherein G (X) is a loss value, X i For identifying the region, n is the number of the identification regions in the ith identification region set, n is an integer of 1 or more, p (x) i ) Is the ratio of the number of successfully identified regions in the ith identified region to the total number of identified regions in the ith identified region set.
In colloquial terms, the multiple recognition area sets represent multiple small targets, the closer the loss value of G (X) is to 0, which indicates that the higher the recognition accuracy of the small targets is, a threshold may be set, where the threshold is a preset threshold, that is, when the small targets are recognized, it is difficult to completely and accurately recognize all the small targets, or the recognized small targets may have errors, but as long as the accuracy reaches a certain height, the accuracy of the subsequent defect recognition is not affected. And judging whether the loss value G (X) meets a preset threshold value according to the loss analysis result, if so, setting the multiple recognition area sets as target area sets, and if not, re-recognizing until the loss value meets the preset threshold value.
Step S500: video acquisition is carried out on the target area set in a preset time window, and a plurality of target area videos are obtained;
specifically, the preset time window refers to a preset period of time, and can be set by itself, and video acquisition is performed on each identification area in the target area set within the preset period of time, so as to obtain a plurality of target area videos.
Step S600: inputting the videos of the multiple target areas into a defect recognition module in a defect recognition model, and outputting multiple surface defect sets, wherein the defect recognition module is constructed based on a slow network;
as shown in fig. 3, step S600 in the embodiment of the present invention includes:
step S610: performing primary picture extraction on the multiple target area videos according to a first preset step length to obtain a first picture set, wherein the first preset step length is the number of picture frames extracted per second when primary picture extraction is performed in advance;
step S620: performing secondary picture extraction on the videos of the multiple target areas according to a second preset step length, and performing pooling treatment to obtain a second picture set, wherein the second preset step length is the number of picture frames extracted per second when the secondary picture extraction is performed in advance, and the second preset step length is larger than the first preset step length;
step S630: constructing the defect identification module by taking a slow fast network as a basic framework, wherein the defect identification module comprises a slow analysis channel, a fast analysis channel and a defect identification layer;
step S640: and inputting the first picture set into the slow analysis channel, inputting the second picture set into the fast analysis channel, and obtaining the surface defect set output by the defect identification layer.
The step S630 of the embodiment of the present invention includes:
step S631: obtaining a plurality of sample first picture sets and a plurality of sample second picture sets based on images of a plurality of target detection objects in a past period of time;
step S632: traversing the plurality of sample first picture sets and the plurality of sample second picture sets to identify surface defects corresponding to each sample, thereby obtaining a plurality of sample surface defect sets;
step S633: and taking the first picture sets of the samples, the second picture sets of the samples and the defect sets of the surfaces of the samples as module sample data, and obtaining the defect identification module based on the module sample data.
The step S633 of the embodiment of the present invention includes:
step S6331: generating a slow analysis channel, a fast analysis channel and a defect identification layer by taking the slow fast network as a basic framework, wherein the slow analysis channel and the fast analysis channel are full convolution layers, the defect identification layer is a full connection layer, and the defect identification layer is connected with the slow analysis channel and the fast analysis channel;
step S6332: dividing and marking the module sample data according to a preset dividing proportion, so as to obtain module training data, module verification data and module test data;
step S6333: and performing supervision training on the slow analysis channel, the fast analysis channel and the defect identification layer by using module training data, performing verification test on the slow analysis channel, the fast analysis channel and the defect identification layer by using module verification data and module test data respectively, and obtaining the defect identification module after the preset condition is met.
Specifically, the multiple target area videos are input into a defect recognition module in a defect recognition model, and multiple surface defect sets are output, wherein the defect recognition module is constructed based on a slow network, the slow network uses a slow high-resolution CNN (slow channel) to analyze static content in the videos, and simultaneously uses a fast low-resolution CNN (fast channel) to analyze dynamic content in the videos, that is, the defects can change along with time, but the changes are difficult to see in a short time, so that the videos need to be collected for recognizing the surface defects.
Specifically, the process of inputting the videos of the multiple target areas into a defect recognition module in a defect recognition model and outputting multiple surface defect sets is as follows: and carrying out image extraction on the multiple target area videos for one time according to a first preset step length to obtain a first image set, wherein the first preset step length is the number of image frames extracted per second when the image extraction is carried out for one time in advance, the first preset step length is smaller, for example, 3 frames of images are extracted per second, the images in the first image set are fewer, the resolution ratio is higher, the details contained in the images are more, and the small targets can be distinguished.
And carrying out secondary picture extraction on the videos of the plurality of target areas according to a second preset step length, and carrying out pooling treatment to obtain a second picture set, wherein the second preset step length is the number of picture frames extracted per second when the secondary picture extraction is carried out, the second preset step length is larger than the first preset step length, image extraction treatment is carried out on the videos of the plurality of target areas according to the second preset step length, and downsampling treatment is carried out on an image extraction result, wherein the downsampling treatment is used for reducing the image resolution and generating a thumbnail of a corresponding image. Compared with the first preset step length, the second preset step length is appropriately increased, for example, 15 frames per second are extracted, more images are in the second picture set, but the resolution is lower after pooling treatment, the details contained in the second picture set are less, and a large target can be effectively identified.
And constructing the defect identification module by taking a Slow Fast network as a basic framework, wherein the Slow Fast neural network consists of a Slow channel and a Fast channel, and after the data of the Fast channel is converted, the result can be sent into the Slow channel through lateral connection and fused with the data of the Slow channel, so that the defect identification module is constructed, and comprises a Slow analysis channel, a Fast analysis channel and a defect identification layer. And inputting the first picture set into the slow analysis channel, inputting the second picture set into the fast analysis channel, and obtaining the surface defect set output by the defect identification layer. The slow analysis channel is used for identifying details of a defect area in the image, such as a defect position and the like, according to the image with higher resolution in the first picture set, and the fast analysis channel is used for identifying defect transfer conditions, such as whether defects in a target detection object are close or not, according to the image with lower resolution but a larger number of images in the second picture set, so that an identification result of the current defect position is output, and defect identification accuracy is improved.
Specifically, the process of constructing the defect recognition module is as follows: based on images of a plurality of target detection objects in a past period of time, a plurality of sample first screen sets and a plurality of sample second screen sets are obtained, the plurality of target detection objects being the same as the model of the target detection object in step S100. And traversing the plurality of sample first picture sets and the plurality of sample second picture sets to identify the surface defects corresponding to each sample, thereby obtaining a plurality of sample surface defect sets, taking the plurality of sample first picture sets and the plurality of sample second picture sets and the plurality of sample surface defect sets as module sample data, and training the defect identification module based on the module sample data.
The process of obtaining the defect identification module based on the module sample data is as follows: and generating a slow analysis channel, a fast analysis channel and a defect identification layer by taking the slow fast network as a basic framework, wherein the slow analysis channel and the fast analysis channel are full convolution layers, the defect identification layer is a full connection layer, the defect identification layer is connected with the slow analysis channel and the fast analysis channel, and after the result of the fast analysis channel is subjected to data transformation, the result can be sent into the slow analysis channel through lateral connection and fused with the data of the slow analysis channel. And dividing the module sample data into a training data set, a test data set and a verification data set according to a preset dividing proportion, and marking the dividing result, for example, the module sample data can be divided into module training data, module verification data and module test data according to the proportion of 70%, 20% and 10%. The method comprises the steps of performing supervised training on a slow analysis channel, a fast analysis channel and a defect recognition layer through module training data, dividing the module training data into a plurality of batches of sample data, performing supervised training on a model through the sample data of each batch, updating model parameters, performing supervised training on the model through training data of the next batch, presetting a training accuracy index, wherein the training accuracy index is a preset condition, stopping iterative training when the accuracy of a model output result is greater than the training accuracy index, performing test training on the slow analysis channel, the fast analysis channel and the defect recognition layer through module testing data, performing verification training on the model through module verification data when the accuracy of the model output result is greater than the preset condition, and obtaining the slow analysis channel, the fast analysis channel and the defect recognition layer when the accuracy of the model output result is greater than the preset condition.
Step S700: and inputting the plurality of surface defect sets into a defect matching module in the defect recognition model to obtain a surface defect recognition result set.
The step S700 of the embodiment of the present invention includes:
step S710: acquiring a historical surface defect set and a historical surface defect recognition result set of target area detection, and constructing a mapping relation between the surface defects and the surface defect recognition results;
step S720: and constructing the defect matching module based on the mapping relation, and obtaining the surface defect recognition result set after inputting the plurality of surface defect sets into the defect matching module.
Specifically, the defect matching module comprises a plurality of groups of historical surface defect sets and historical surface defect recognition result sets which are in one-to-one correspondence, the surface defect sets are input into the defect matching module in the defect recognition model, the same historical surface defect sets are matched in the defect matching module, the corresponding historical surface defect recognition result sets are matched based on one-to-one correspondence between the historical surface defect sets and the historical surface defect recognition result sets, the historical surface defect recognition result sets are used as the surface defect recognition result sets, and the surface defect recognition result sets comprise information such as defect types, defect positions and the like.
Specifically, a historical surface defect set and a historical surface defect recognition result set of target area detection are obtained, a mapping relation between surface defects and the surface defect recognition result is built, a defect matching module is built based on the mapping relation, after the surface defect sets are input into the defect matching module, similarity between the surface defect sets and the historical surface defect set is calculated through the defect matching module, a historical surface defect set with the highest similarity is obtained, and the historical surface defect recognition result set corresponding to the historical surface defect set with the highest similarity is used as the surface defect recognition result set, so that defect recognition accuracy is improved.
Based on the analysis, the invention provides a surface defect identification method based on small target detection, in the embodiment, a real-time image of a target detection object is acquired to obtain a real-time image set; preprocessing the real-time image set, and extracting features of the preprocessed real-time image set by using a feature extraction model to obtain an image feature set; performing small target recognition on the image feature set by using a classifier, and obtaining a plurality of recognition area sets according to recognition results; carrying out loss analysis on the plurality of identification area sets by using a softmax loss function, judging whether a preset threshold is met according to a loss value in a loss analysis result, and if so, setting the plurality of identification area sets as target area sets; video acquisition is carried out on the target area set in a preset time window, and a plurality of target area videos are obtained; inputting the videos of the multiple target areas into a defect recognition module in a defect recognition model, and outputting multiple surface defect sets, wherein the defect recognition module is constructed based on a slow network; and inputting the plurality of surface defect sets into a defect matching module in the defect recognition model to obtain a surface defect recognition result set, thereby achieving the technical effects of improving the image processing effect and further improving the surface defect recognition accuracy.
Example two
Based on the same inventive concept as the surface defect recognition method based on small target detection in the foregoing embodiment, as shown in fig. 4, the present invention further provides a surface defect recognition system based on small target detection, where the system includes:
the real-time image acquisition module 11 is used for acquiring a real-time image of the target detection object to obtain a real-time image set;
the feature extraction module 12 is configured to pre-process the real-time image set, and perform feature extraction on the pre-processed real-time image set by using a feature extraction model to obtain an image feature set;
a small target recognition module 13, where the small target recognition module 13 is configured to perform small target recognition on the image feature set by using a classifier, and obtain a plurality of recognition area sets according to a recognition result;
the loss analysis module 14 is configured to perform loss analysis on the plurality of identification area sets by using a softmax loss function, determine whether a preset threshold is met according to a loss value in a loss analysis result, and if so, set the plurality of identification area sets as a target area set;
the video acquisition module 15 is used for carrying out video acquisition on the target area set in a preset time window by the video acquisition module 15 to obtain a plurality of target area videos;
the surface defect set output module 16, wherein the surface defect set output module 16 is configured to input the plurality of target area videos into a defect recognition module in a defect recognition model, and output a plurality of surface defect sets, and the defect recognition module is constructed based on a slow network;
a defect recognition result obtaining module 17, where the defect recognition result obtaining module 17 is configured to input the plurality of surface defect sets into a defect matching module in a defect recognition model, and obtain a surface defect recognition result set.
Further, the system further comprises:
the feature extraction model construction module is used for constructing the feature extraction model based on a convolutional neural network, and the feature extraction model comprises an input layer, a texture feature analysis layer, a shape feature analysis layer and an output layer;
the texture feature analysis module is used for inputting the preprocessed real-time image set into a texture feature analysis layer in the feature extraction model to obtain a texture feature vector set;
the shape feature analysis module is used for inputting the preprocessed real-time image set into a shape feature analysis layer in the feature extraction model to obtain a shape feature vector set;
and the data integration module is used for obtaining an image feature set according to the texture feature vector set and the shape feature vector set.
Further, the system further comprises:
the Softmax loss function is:
wherein G (X) is a loss value, X i For identifying the region, n is the number of the identification regions in the ith identification region set, n is an integer of 1 or more, p (x) i ) Is the ratio of the number of successfully identified regions in the ith identified region to the total number of identified regions in the ith identified region set.
Further, the system further comprises:
the primary picture extraction module is used for carrying out primary picture extraction on the multiple target area videos according to a first preset step length to obtain a first picture set, wherein the first preset step length is a picture frame number extracted every second when carrying out primary picture extraction in advance;
the secondary picture extraction module is used for carrying out secondary picture extraction on the videos of the plurality of target areas according to a second preset step length, and carrying out pooling treatment to obtain a second picture set, wherein the second preset step length is the number of picture frames extracted per second when the secondary picture extraction is carried out, and the second preset step length is larger than the first preset step length;
the foundation framework building module is used for building the defect identification module by taking a slow network as a foundation framework, wherein the defect identification module comprises a slow analysis channel, a fast analysis channel and a defect identification layer;
and the defect output module is used for inputting the first picture set into the slow analysis channel, inputting the second picture set into the fast analysis channel and obtaining the surface defect set output by the defect identification layer.
Further, the system further comprises:
a first sample acquisition module for acquiring a plurality of sample first screen sets and a plurality of sample second screen sets based on images of a plurality of target detection objects in a past period of time;
the second sample acquisition module is used for traversing the plurality of sample first picture sets and the plurality of sample second picture sets to identify the surface defects corresponding to each sample so as to obtain a plurality of sample surface defect sets;
and the module sample data acquisition module is used for taking the plurality of sample first picture sets, the plurality of sample second picture sets and the plurality of sample surface defect sets as module sample data and acquiring the defect identification module based on the module sample data.
Further, the system further comprises:
the channel construction module is used for generating a slow analysis channel, a fast analysis channel and a defect identification layer by taking the slow fast network as a basic framework, wherein the slow analysis channel and the fast analysis channel are full convolution layers, the defect identification layer is a full connection layer, and the defect identification layer is connected with the slow analysis channel and the fast analysis channel;
the data dividing and marking module is used for dividing and marking the module sample data according to a preset dividing proportion so as to obtain module training data, module verification data and module test data;
and the verification test module is used for performing supervision training on the slow analysis channel, the fast analysis channel and the defect identification layer by using module training data, performing verification test on the slow analysis channel, the fast analysis channel and the defect identification layer by using module verification data and module test data respectively, and obtaining the defect identification module after the preset condition is met.
Further, the system further comprises:
the mapping relation construction module is used for acquiring a historical surface defect set and a historical surface defect recognition result set detected by the target area and constructing a mapping relation between the surface defects and the surface defect recognition result;
the defect identification result acquisition module is used for constructing the defect matching module based on the mapping relation, and acquiring the surface defect identification result set after inputting the plurality of surface defect sets into the defect matching module.
A specific example of a surface defect recognition method based on small target detection in the foregoing embodiment is also applicable to a surface defect recognition system based on small target detection in this embodiment, and by the foregoing detailed description of a surface defect recognition method based on small target detection, those skilled in the art can clearly know a surface defect recognition system based on small target detection in this embodiment, so that details thereof will not be described herein for brevity.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A method for identifying surface defects based on small target detection, the method comprising:
collecting real-time images of a target detection object to obtain a real-time image set;
preprocessing the real-time image set, and extracting features of the preprocessed real-time image set by using a feature extraction model to obtain an image feature set;
performing small target recognition on the image feature set by using a classifier, and obtaining a plurality of recognition area sets according to recognition results;
carrying out loss analysis on the plurality of identification area sets by using a softmax loss function, judging whether a preset threshold is met according to a loss value in a loss analysis result, and if so, setting the plurality of identification area sets as target area sets;
video acquisition is carried out on the target area set in a preset time window, and a plurality of target area videos are obtained;
inputting the videos of the multiple target areas into a defect recognition module in a defect recognition model, and outputting multiple surface defect sets, wherein the defect recognition module is constructed based on a slow network;
and inputting the plurality of surface defect sets into a defect matching module in the defect recognition model to obtain a surface defect recognition result set.
2. The method of claim 1, wherein the method comprises:
constructing the feature extraction model based on a convolutional neural network, wherein the feature extraction model comprises an input layer, a texture feature analysis layer, a shape feature analysis layer and an output layer;
inputting the preprocessed real-time image set into a texture feature analysis layer in the feature extraction model to obtain a texture feature vector set;
inputting the preprocessed real-time image set into a shape feature analysis layer in the feature extraction model to obtain a shape feature vector set;
and obtaining an image feature set according to the texture feature vector set and the shape feature vector set.
3. The method of claim 1, wherein the method comprises:
the Softmax loss function is:
wherein G (X) is a loss value, X i For identifying the region, n is the number of the identification regions in the ith identification region set, n is an integer of 1 or more, p (x) i ) Is the ratio of the number of successfully identified regions in the ith identified region to the total number of identified regions in the ith identified region set.
4. The method of claim 1, wherein the method comprises:
performing primary picture extraction on the multiple target area videos according to a first preset step length to obtain a first picture set, wherein the first preset step length is the number of picture frames extracted per second when primary picture extraction is performed in advance;
performing secondary picture extraction on the videos of the multiple target areas according to a second preset step length, and performing pooling treatment to obtain a second picture set, wherein the second preset step length is the number of picture frames extracted per second when the secondary picture extraction is performed in advance, and the second preset step length is larger than the first preset step length;
constructing the defect identification module by taking a slow fast network as a basic framework, wherein the defect identification module comprises a slow analysis channel, a fast analysis channel and a defect identification layer;
and inputting the first picture set into the slow analysis channel, inputting the second picture set into the fast analysis channel, and obtaining the surface defect set output by the defect identification layer.
5. The method of claim 4, wherein constructing the defect recognition module comprises:
obtaining a plurality of sample first picture sets and a plurality of sample second picture sets based on images of a plurality of target detection objects in a past period of time;
traversing the plurality of sample first picture sets and the plurality of sample second picture sets to identify surface defects corresponding to each sample, thereby obtaining a plurality of sample surface defect sets;
and taking the first picture sets of the samples, the second picture sets of the samples and the defect sets of the surfaces of the samples as module sample data, and obtaining the defect identification module based on the module sample data.
6. The method according to claim 5, comprising:
generating a slow analysis channel, a fast analysis channel and a defect identification layer by taking the slow fast network as a basic framework, wherein the slow analysis channel and the fast analysis channel are full convolution layers, the defect identification layer is a full connection layer, and the defect identification layer is connected with the slow analysis channel and the fast analysis channel;
dividing and marking the module sample data according to a preset dividing proportion, so as to obtain module training data, module verification data and module test data;
and performing supervision training on the slow analysis channel, the fast analysis channel and the defect identification layer by using module training data, performing verification test on the slow analysis channel, the fast analysis channel and the defect identification layer by using module verification data and module test data respectively, and obtaining the defect identification module after the preset condition is met.
7. The method of claim 1, wherein the method comprises:
acquiring a historical surface defect set and a historical surface defect recognition result set of target area detection, and constructing a mapping relation between the surface defects and the surface defect recognition results;
and constructing the defect matching module based on the mapping relation, and obtaining the surface defect recognition result set after inputting the plurality of surface defect sets into the defect matching module.
8. A surface defect recognition system based on small object detection, the system comprising:
the real-time image acquisition module is used for acquiring a real-time image of the target detection object to obtain a real-time image set;
the feature extraction module is used for preprocessing the real-time image set, and extracting features of the preprocessed real-time image set by using a feature extraction model to obtain an image feature set;
the small target recognition module is used for carrying out small target recognition on the image feature set by using a classifier, and a plurality of recognition area sets are obtained according to recognition results;
the loss analysis module is used for carrying out loss analysis on the plurality of identification area sets by using a softmax loss function, judging whether a preset threshold is met according to a loss value in a loss analysis result, and if so, setting the plurality of identification area sets as target area sets;
the video acquisition module is used for acquiring videos of the target area set in a preset time window to obtain videos of a plurality of target areas;
the surface defect set output module is used for inputting the videos of the plurality of target areas into a defect recognition module in a defect recognition model to output a plurality of surface defect sets, wherein the defect recognition module is constructed based on a slow network;
the defect recognition result acquisition module is used for inputting the plurality of surface defect sets into the defect matching module in the defect recognition model to obtain the surface defect recognition result set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310624084.5A CN116563641A (en) | 2023-05-30 | 2023-05-30 | Surface defect identification method and system based on small target detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310624084.5A CN116563641A (en) | 2023-05-30 | 2023-05-30 | Surface defect identification method and system based on small target detection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116563641A true CN116563641A (en) | 2023-08-08 |
Family
ID=87492963
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310624084.5A Withdrawn CN116563641A (en) | 2023-05-30 | 2023-05-30 | Surface defect identification method and system based on small target detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116563641A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116990229A (en) * | 2023-09-26 | 2023-11-03 | 南通赛可特电子有限公司 | Defect detection method and system for copper plating layer surface of product |
| CN117078689A (en) * | 2023-10-17 | 2023-11-17 | 沈阳宏远电磁线股份有限公司 | Cable defect identification method and system based on machine vision |
| CN117115148A (en) * | 2023-10-19 | 2023-11-24 | 苏州弘皓光电科技有限公司 | Chip surface defect intelligent identification method based on 5G technology |
| CN117151551A (en) * | 2023-10-31 | 2023-12-01 | 江苏富松模具科技有限公司 | Stamping quality management method and system for stator and rotor silicon steel sheets |
| CN117274239A (en) * | 2023-11-13 | 2023-12-22 | 江苏永鼎股份有限公司 | Method for rapidly detecting defects of chip packaging technology |
| CN117351001A (en) * | 2023-11-16 | 2024-01-05 | 肇庆市大正铝业有限公司 | Surface defect identification method for regenerated aluminum alloy template |
| CN117654907A (en) * | 2023-11-29 | 2024-03-08 | 嘉兴嘉视自动化科技有限公司 | Automatic eliminating method and system for strip detector |
| CN118334015A (en) * | 2024-06-12 | 2024-07-12 | 数智汇能(大连)科技发展有限公司 | Defect identification method, system, equipment and medium based on visual image |
-
2023
- 2023-05-30 CN CN202310624084.5A patent/CN116563641A/en not_active Withdrawn
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116990229B (en) * | 2023-09-26 | 2023-12-08 | 南通赛可特电子有限公司 | Defect detection method and system for copper plating layer surface of product |
| CN116990229A (en) * | 2023-09-26 | 2023-11-03 | 南通赛可特电子有限公司 | Defect detection method and system for copper plating layer surface of product |
| CN117078689A (en) * | 2023-10-17 | 2023-11-17 | 沈阳宏远电磁线股份有限公司 | Cable defect identification method and system based on machine vision |
| CN117078689B (en) * | 2023-10-17 | 2024-01-30 | 沈阳宏远电磁线股份有限公司 | Cable defect identification method and system based on machine vision |
| CN117115148A (en) * | 2023-10-19 | 2023-11-24 | 苏州弘皓光电科技有限公司 | Chip surface defect intelligent identification method based on 5G technology |
| CN117115148B (en) * | 2023-10-19 | 2024-05-14 | 苏州弘皓光电科技有限公司 | Chip surface defect intelligent identification method based on 5G technology |
| CN117151551B (en) * | 2023-10-31 | 2024-02-06 | 江苏富松模具科技有限公司 | Stamping quality management method and system for stator and rotor silicon steel sheets |
| CN117151551A (en) * | 2023-10-31 | 2023-12-01 | 江苏富松模具科技有限公司 | Stamping quality management method and system for stator and rotor silicon steel sheets |
| CN117274239B (en) * | 2023-11-13 | 2024-02-20 | 江苏永鼎股份有限公司 | Method for rapidly detecting defects of chip packaging technology |
| CN117274239A (en) * | 2023-11-13 | 2023-12-22 | 江苏永鼎股份有限公司 | Method for rapidly detecting defects of chip packaging technology |
| CN117351001A (en) * | 2023-11-16 | 2024-01-05 | 肇庆市大正铝业有限公司 | Surface defect identification method for regenerated aluminum alloy template |
| CN117351001B (en) * | 2023-11-16 | 2024-05-28 | 肇庆市大正铝业有限公司 | Surface defect identification method for regenerated aluminum alloy template |
| CN117654907A (en) * | 2023-11-29 | 2024-03-08 | 嘉兴嘉视自动化科技有限公司 | Automatic eliminating method and system for strip detector |
| CN118334015A (en) * | 2024-06-12 | 2024-07-12 | 数智汇能(大连)科技发展有限公司 | Defect identification method, system, equipment and medium based on visual image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116563641A (en) | Surface defect identification method and system based on small target detection | |
| CN106875381B (en) | Mobile phone shell defect detection method based on deep learning | |
| CN109583489B (en) | Defect classification identification method and device, computer equipment and storage medium | |
| CN111681240B (en) | Bridge surface crack detection method based on YOLO v3 and attention mechanism | |
| CN109033950B (en) | Vehicle illegal parking detection method based on multi-feature fusion cascade depth model | |
| CN110033431B (en) | Non-contact detection device and detection method for detecting corrosion area on surface of steel bridge | |
| CN111652098B (en) | Product surface defect detection method and device | |
| CN111539330B (en) | Transformer substation digital display instrument identification method based on double-SVM multi-classifier | |
| CN105447532A (en) | Identity authentication method and device | |
| CN117974601B (en) | Method and system for detecting surface defects of silicon wafer based on template matching | |
| CN115995056A (en) | Automatic bridge disease identification method based on deep learning | |
| CN111724376B (en) | Paper disease detection method based on texture feature analysis | |
| CN115147418B (en) | Compression training method and device for defect detection model | |
| CN111652846B (en) | Semiconductor defect identification method based on characteristic pyramid convolution neural network | |
| CN107610119A (en) | The accurate detection method of steel strip surface defect decomposed based on histogram | |
| CN111461010B (en) | Power equipment identification efficiency optimization method based on template tracking | |
| CN114004858A (en) | Method and device for identifying aviation cable surface code based on machine vision | |
| CN117455917B (en) | Establishment of false alarm library of etched lead frame and false alarm on-line judging and screening method | |
| CN113421223B (en) | Industrial product surface defect detection method based on deep learning and Gaussian mixture | |
| CN117475327A (en) | Multi-target detection positioning method and system based on remote sensing image in city | |
| CN112784494A (en) | Training method of false positive recognition model, target recognition method and device | |
| CN115082449A (en) | Electronic component defect detection method | |
| CN116128800A (en) | ViG-based mobile phone microphone part defect detection and segmentation method | |
| CN114708457A (en) | Hyperspectral deep learning identification method for purple fringing resistance identification | |
| CN111046876B (en) | License plate character rapid recognition method and system based on texture detection technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20230808 |
|
| WW01 | Invention patent application withdrawn after publication |