CN116309526A - Paper cup defect detection method and device - Google Patents

Paper cup defect detection method and device Download PDF

Info

Publication number
CN116309526A
CN116309526A CN202310377132.5A CN202310377132A CN116309526A CN 116309526 A CN116309526 A CN 116309526A CN 202310377132 A CN202310377132 A CN 202310377132A CN 116309526 A CN116309526 A CN 116309526A
Authority
CN
China
Prior art keywords
feature
paper cup
module
scale
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310377132.5A
Other languages
Chinese (zh)
Inventor
蒋亚军
曹昭辉
文煜超
张闯
付丹丹
胡志刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Polytechnic University
Original Assignee
Wuhan Polytechnic University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Polytechnic University filed Critical Wuhan Polytechnic University
Priority to CN202310377132.5A priority Critical patent/CN116309526A/en
Publication of CN116309526A publication Critical patent/CN116309526A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30124—Fabrics; Textile; Paper
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a paper cup defect detection method and device, which belong to the field of product detection, and the method comprises the following steps: acquiring a paper cup image to be detected; inputting the paper cup image into a trained detection model, and outputting a defect detection result of the paper cup; the detection model is a YOLO v5 network model, a CBAM module is added after a last layer of C3 module of a back bone part, a weighted bidirectional feature pyramid network is utilized to perform feature fusion in a Neck part, a fourth-scale detection layer is added to an output layer for construction, and the detection model is obtained after training samples according to paper cup images marked with defective labels; the scale of the fourth scale detection layer is larger than that of the original other three detection layers. The method improves the expression capability of defect characteristics in a complex environment, strengthens the fusion capability of the characteristics, has more accurate detection results of small targets such as paper cup defects, and effectively avoids the phenomena of missing detection and false detection.

Description

Paper cup defect detection method and device
Technical Field
The invention relates to the field of product detection, in particular to a paper cup defect detection method and device.
Background
Paper cups have been used as food containers for decades, and in recent years, paper cups have also been used in large quantities for food packaging. The paper cup has the characteristics of cleanness, sanitation, relatively simple production process and flexible and changeable style, is easy to degrade in natural environment, belongs to green packaging, and is favored by wide manufacturers and consumers. Paper cups are also used as products and are also required to be subjected to defect detection during the manufacturing process.
At present, paper cup defect detection tasks are currently mainly finished by manual screening and detection methods based on image processing technology. Wherein, manual screening has the defects of omission, low efficiency, large subjective randomness and the like, and can not meet the modern production requirements. The detection method based on the image processing technology needs a series of pretreatment operations such as graying, binarization and the like in the process of extracting the characteristics, and the process is complicated. Meanwhile, the detection method is influenced by the detection principle, has high requirements on illumination conditions of detection environments, has poor robustness, is easy to generate the phenomena of missing detection and false detection on paper cup defects with small size and unobvious characteristics, and is difficult to meet actual production requirements.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention provides a paper cup defect detection method and device.
The invention provides a paper cup defect detection method, which comprises the following steps: acquiring a paper cup image to be detected; inputting the paper cup image into a trained detection model, and outputting a defect detection result of the paper cup; the detection model is a YOLO v5 network model, a CBAM attention mechanism module is added after a last layer of C3 module of a back box part, a weighted bidirectional feature pyramid network is utilized for feature fusion in a Neck part, a fourth-scale detection layer is added on an output layer for construction, and the detection model is obtained after training according to a paper cup image training sample marked with a defective label; the scale of the fourth scale detection layer is larger than that of the original other three detection layers.
According to the paper cup defect detection method provided by the invention, before the paper cup image is input into the trained detection model, the method further comprises the following steps: obtaining unqualified paper cup images of various defect types, and constructing a data set; performing data enhancement operation on the data set, and adding the paper cup image in the data set into a label of a corresponding defect result to obtain a training sample; training the constructed YOLO v5 network model by using the training sample to obtain the trained detection model; wherein the data enhancement operations include rotation, clipping, and brightness enhancement.
According to the paper cup defect detection method provided by the invention, the last layer of the detection model back is an SPPF module, and a CBAM attention mechanism module is added between the last layer of C3 module and the SPPF module.
According to the paper cup defect detection method provided by the invention, the paper cup image is input into a trained detection model, and the paper cup defect detection method comprises the following steps:
inputting the paper cup image to be detected into a backbox part, and sequentially obtaining a first characteristic, a second characteristic, a third characteristic and a fourth characteristic of the paper cup image processed by the first C3 module, the second C3 module, the third C3 module and the SPPF module according to the initial sequence of the input direction; tensor splicing is carried out according to the third feature and the fourth feature, and a result is input into a fifth C3 module to obtain a fifth feature; tensor splicing is carried out according to the fifth feature and the second feature, and a result is input into a sixth C3 module to obtain a sixth feature; tensor splicing is carried out according to the sixth feature and the first feature, a result is input into a seventh C3 module to obtain a seventh feature, and output of a fourth scale is determined according to the seventh feature; tensor splicing is carried out according to the sixth feature and the seventh feature, a result is input into an eighth C3 module to obtain an eighth feature, and output of a third scale is determined according to the eighth feature; according to the third feature, the fifth feature and the eighth feature, feature fusion is carried out based on a weighted bidirectional feature pyramid network, a result is input into a ninth C3 module to obtain a ninth feature, and output of a second scale is determined according to the ninth feature; tensor splicing is carried out according to the ninth feature and the fourth feature, a result is input into a tenth C3 module to obtain a tenth feature, and output of a first scale is determined according to the tenth feature; wherein, the first scale to the fourth scale are arranged from small to large in sequence.
According to the paper cup defect detection method provided by the invention, the paper cup image is input into a trained detection model, and the paper cup defect detection method comprises the following steps: and up-sampling the features before the third-scale output layer, performing feature fusion with the features before the fourth-scale output layer, and determining the fourth-scale output result according to the fused features.
The invention also provides a paper cup defect detection device, which comprises: the acquisition module is used for acquiring a paper cup image to be detected; the processing module is used for inputting the paper cup image into the trained detection model and outputting a defect detection result of the paper cup;
the detection model is a YOLO v5 network model, a CBAM attention mechanism module is added after a last layer of C3 module of a back box part, a weighted bidirectional feature pyramid network is utilized for feature fusion in a Neck part, a fourth-scale detection layer is added on an output layer for construction, and the detection model is obtained after training according to a paper cup image training sample marked with a defective label; the scale of the fourth scale detection layer is larger than that of the original other three detection layers.
According to the paper cup defect detection device provided by the invention, the processing module is specifically used for:
inputting the paper cup image to be detected into a backbox part, and sequentially obtaining a first characteristic, a second characteristic, a third characteristic and a fourth characteristic of the paper cup image processed by the first C3 module, the second C3 module, the third C3 module and the SPPF module according to the initial sequence of the input direction; tensor splicing is carried out according to the third feature and the fourth feature, and a result is input into a fifth C3 module to obtain a fifth feature; tensor splicing is carried out according to the fifth feature and the second feature, and a result is input into a sixth C3 module to obtain a sixth feature; tensor splicing is carried out according to the sixth feature and the first feature, a result is input into a seventh C3 module to obtain a seventh feature, and output of a fourth scale is determined according to the seventh feature; tensor splicing is carried out according to the sixth feature and the seventh feature, a result is input into an eighth C3 module to obtain an eighth feature, and output of a third scale is determined according to the eighth feature; according to the third feature, the fifth feature and the eighth feature, feature fusion is carried out based on a weighted bidirectional feature pyramid network, a result is input into a ninth C3 module to obtain a ninth feature, and output of a second scale is determined according to the ninth feature; tensor splicing is carried out according to the ninth feature and the fourth feature, a result is input into a tenth C3 module to obtain a tenth feature, and output of a first scale is determined according to the tenth feature; wherein, the first scale to the fourth scale are arranged from small to large in sequence.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the paper cup defect detection method is realized by the processor when the program is executed.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a paper cup defect detection method as described in any of the above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a paper cup defect detection method as described in any of the above.
According to the paper cup defect detection method and device, the CBAM attention mechanism module is added, the feature extraction capacity of the model is improved, and the paper cup defect detection method and device are added after the last layer of C3 module, unlike the conventional application of the CBAM attention mechanism, the output result of the CBAM module after the last layer of C3 module is sequentially subjected to tensor splicing and other operations with the feature processing of other network layers of a backstone part, so that the feature extraction capacity of intermediate features is obviously improved, and the expression capacity of defect features in a complex environment is improved. And combining with the fusion of weighted bidirectional feature pyramid networks of the Neck part, the feature fusion capability is enhanced, so that the extracted intermediate features can more remarkably reflect the feature attribute of the detection target, and the detection result of the small target such as paper cup defect is more accurate. On the basis, the intermediate features with higher feature extraction capability and the fused features are combined, a fourth-scale detection layer larger than the original three detection layers is added to the output layer, the phenomena of missing detection and false detection caused by the fact that detailed feature information is greatly lost along with the deepening of the model depth are further reduced, the detection capability of the model on small and unobvious defects of the features is improved, and the detection accuracy of paper cup defects is further improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a paper cup defect detection method provided by the invention;
FIG. 2 is a diagram of the improved Yolov5 network architecture provided by the present invention;
FIG. 3 is a graph showing comparison of mAP curves of average precision means before and after improvement provided by the invention;
FIG. 4 is a graph comparing precision curves before and after improvement provided by the invention;
FIG. 5 is a graph comparing recall curves before and after improvement provided by the present invention;
FIG. 6a is a graph showing the comparison of the actual effect detection before and after improvement provided by the present invention;
FIG. 6b is a second comparison chart of the actual effect detection before and after improvement provided by the present invention;
FIG. 6c is a third comparison chart of the actual effect detection before and after improvement provided by the present invention;
fig. 7 is a schematic structural diagram of a paper cup defect detecting device provided by the invention;
fig. 8 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The paper cup defect detection method and apparatus of the present invention are described below with reference to fig. 1-8. Fig. 1 is a schematic flow chart of a paper cup defect detection method provided by the present invention, as shown in fig. 1, the present invention provides a paper cup defect detection method, including:
101. acquiring a paper cup image to be detected;
the images of paper cups on the assembly line, including partially defective paper cups, can be obtained by means of a camera or the like.
102. Inputting the paper cup image into a trained detection model, and outputting a defect detection result of the paper cup.
According to the invention, the paper cup image is input into the improved detection model based on the YOLO v5 network model, and a final defect detection result of the paper cup is output, for example, a detection frame corresponding to a defect part is output, and the part can refer to the training and detection process of the existing YOLO v5 network model. In addition, the detection model is obtained by training paper cup images marked with defect conditions in advance.
Specifically, the detection model is a YOLO v5 network model, a CBAM attention mechanism module is added after a last layer of C3 module of a back box part, a weighted bidirectional feature pyramid network is utilized to perform feature fusion in a Neck part, a fourth-scale detection layer is added on an output layer for construction, and the detection model is obtained after training according to a paper cup image training sample marked with a defective label; the scale of the fourth scale detection layer is larger than that of the original other three detection layers.
In the present invention, the improvement of the improved YOLO v5 paper cup defect detection model includes 3 aspects: introducing a CBAM attention mechanism module behind a last layer C3 module of a Backbone part in the YOLO v5 original network model, and improving the feature extraction capability of the model; and adding a shallow detection layer with a larger scale on the output part of the model, and changing the three-scale detection into four-scale detection. For example, the original three scales are 20×20, 40×40 and 80×80, and a shallow detection layer with a scale of 160×160 can be added in the invention, so that the detection capability of the model on small targets and targets with unobvious characteristics is improved. And a weighted bidirectional feature pyramid network BiFPN is used in the Neck part of the model to partially improve the PANet in the model, so that the feature fusion capability of the model is enhanced. The last layer of C3 modules is the C3 modules from the input direction to the last layer in sequence.
The CBAM attention mechanism module comprises a channel attention module and a space attention module, wherein the channel attention module focuses on the content information of the detection target, and the space attention module focuses on the position information of the detection target. The combination of the two can focus the output characteristic information on the key characteristic information. According to the invention, the CBAM attention mechanism module is placed behind the last layer of C3 module in the feature extraction network (backstene), so that the problem that detailed feature information is gradually lost along with the increase of the depth of a model in the feature extraction network can be solved, and the extraction capacity of the feature extraction network of an original model on the defect feature of the paper cup and the expression capacity of the defect feature of the paper cup under the condition of insufficient light are improved.
The Neck part uses BiFPN, introduces a weighting strategy and a cross-scale connection method, can fuse the characteristic information in the characteristic extraction network with the characteristic information in the transmission route from shallow to deep and regulate the contribution degree of the input characteristic information in the output characteristic diagram with emphasis, thereby realizing better characteristic fusion. .
Considering that part of the pictures of the paper cup defect data set contain different types of defect forms and are similar in characteristic expression, if the type of pictures are input into a model completely replaced by BiFPN for characteristic fusion, the characteristic expression of the similar different types of defects can reduce the characteristic fusion effect of the model under the BiFPN weighting operation of each scale. Therefore, the invention selects and partially improves the BiFPN based on the original feature fusion mode by referring to the weighting strategy and the trans-scale connection method, thereby enhancing the feature fusion capability of the model
According to the paper cup defect detection method, the CBAM attention mechanism module is added, the feature extraction capacity of the model is improved, and the paper cup defect detection method is added after the last layer of C3 module, unlike the conventional application of the CBAM attention mechanism, the output result of the CBAM module after the last layer of C3 module is sequentially subjected to tensor splicing and other operations with the feature processing of other network layers of a backstone part, so that the feature extraction capacity of intermediate features is obviously improved, and the expression capacity of defect features in a complex environment is improved. And combining with the fusion of weighted bidirectional feature pyramid networks of the Neck part, the feature fusion capability is enhanced, so that the extracted intermediate features can more remarkably reflect the feature attribute of the detection target, and the detection result of the small target such as paper cup defect is more accurate. On the basis, the intermediate features with higher feature extraction capability and the fused features are combined, a fourth-scale detection layer larger than the original three detection layers is added to the output layer, the phenomena of missing detection and false detection caused by the fact that detailed feature information is greatly lost along with the deepening of the model depth are further reduced, the detection capability of the model on small and unobvious defects of the features is improved, and the detection accuracy of paper cup defects is further improved.
In some embodiments, before the inputting the paper cup image into the trained detection model, the method further comprises: obtaining unqualified paper cup images of various defect types, and constructing a data set; performing data enhancement operation on the data set, and adding the paper cup image in the data set into a corresponding defect state label to obtain a training sample; training the constructed YOLO v5 network model by using the training sample to obtain the trained detection model; wherein the data enhancement operations include rotation, clipping, and brightness enhancement.
Specifically, the model is trained prior to application of the model. And a plurality of unqualified paper cup images to be detected on the transmission belt can be acquired through a camera, so that a data set is constructed. And then, performing operations such as rotation, cutting, brightness enhancement and the like on the collected unqualified paper cup image data set to enhance the data, so that the diversity of the data set is increased and the generalization capability of model training is improved. And then, the enhanced data set is subjected to standard marking by marking software, such as LabelImg software, the data set is subjected to VOC (volatile organic compound) format marking, and then, the data set in the YOLO format is converted by using a Python script, wherein the dividing ratio of the training data set to the test data set can be 8:2.
And finally, training the constructed YOLO v5 network model based on a training data set comprising unqualified paper cup images and corresponding labels to obtain a YOLO v5 network model meeting the precision requirement, namely the trained detection model in 102. In addition, the method also comprises a process of testing the trained model by using a test data set, wherein the test does not meet the precision requirement, and the training can be performed from a newly constructed data set.
In some embodiments, the last layer of the detection model backhaul part is an SPPF module, and a CBAM attention mechanism module is added between the last layer C3 module and the SPPF module.
Specifically, the invention realizes the fusion of the local feature and the global feature map level through the SPPF module on the basis of adding the CBAM attention mechanism after the last layer of C3 module.
In some embodiments, the inputting the paper cup image into a trained detection model and training the constructed YOLO v5 network model using the training sample comprises:
inputting the paper cup image to be detected into a backbox part, and sequentially obtaining a first characteristic, a second characteristic, a third characteristic and a fourth characteristic of the paper cup image processed by the first C3 module, the second C3 module, the third C3 module and the SPPF module according to the initial sequence of the input direction; tensor splicing is carried out according to the third feature and the fourth feature, and a result is input into a fifth C3 module to obtain a fifth feature; tensor splicing is carried out according to the fifth feature and the second feature, and a result is input into a sixth C3 module to obtain a sixth feature; tensor splicing is carried out according to the sixth feature and the first feature, a result is input into a seventh C3 module to obtain a seventh feature, and output of a fourth scale is determined according to the seventh feature; tensor splicing is carried out according to the sixth feature and the seventh feature, a result is input into an eighth C3 module to obtain an eighth feature, and output of a third scale is determined according to the eighth feature; according to the third feature, the fifth feature and the eighth feature, feature fusion is carried out based on a weighted bidirectional feature pyramid network, a result is input into a ninth C3 module to obtain a ninth feature, and output of a second scale is determined according to the ninth feature; tensor splicing is carried out according to the ninth feature and the fourth feature, a result is input into a tenth C3 module to obtain a tenth feature, and output of a first scale is determined according to the tenth feature; wherein, the first scale to the fourth scale are arranged from small to large in sequence.
As shown in fig. 2, the backhaul part of the initial network includes module parts 0-8 and 10 in order from the input, and the present invention adds a CBAM attention mechanism module between the last layer C3 module and SPPF module of the backhaul part, i.e., between the 8 th and 10 th modules. At this time, the features processed by the 2 nd, 4 th, 6 th and 10 th modules are respectively noted as a first feature, a second feature, a third feature and a fourth feature.
It should be noted that, before performing tensor stitching according to the third feature and the fourth feature, a person skilled in the art may perform processing operations before stitching, such as processing of feature dimensions, including operations of convolution and upsampling in fig. 2. The tensor concatenation of other features mentioned later also includes corresponding processing procedures, which are not described herein, and reference is made to the example in fig. 2. In addition, the training process of the model is also the same as the above steps.
The negk part also includes several C3 modules, as shown in fig. 2, and in order to distinguish four C3 modules of the Backbone, the fifth C3 module and the fifth C3 module are respectively marked as a tenth C3 module. The detection head (detector) of the network model is mainly responsible for multi-scale target detection of the feature map extracted by the backbone network.
According to the paper cup defect detection method, the fourth feature processed by the CBAM module and the SPPF module can be directly or indirectly spliced with the first to third features to obtain the novel seventh to tenth features, and the output of different scales is determined based on the features, so that the feature expression capability of the paper cup which is difficult to detect defects in a complex environment can be effectively improved. For the first and second small-scale detection, the ninth and tenth features of the feature expression capability are effectively improved, and the combination of multiple feature fusion of the weighted bidirectional feature pyramid network is used for further reducing the phenomena of missing detection and false detection caused by the fact that detail feature information is greatly lost along with the deepening of the depth of the model, improving the detection capability of the model on small and feature unobvious defects, and comprehensively improving the detection accuracy of paper cup defects by combining the fourth-scale output determined by the seventh feature.
The method integrates the above-mentioned partial key features by referring to the weighting strategy and the trans-scale connection method of BiFPN on the basis of PANet instead of replacing the whole PANet feature fusion network of the Neck part with the weighted bi-directional feature pyramid network BiFPN, only introduces the sixth layer of original feature information in the back into the deep part of the feature fusion network (BiFPN in fig. 2), and gives corresponding weight to the feature information input into the BiFPN, thereby effectively solving the defects that the deep part of the feature fusion network of the original model lacks of the original feature information participation and the original feature fusion network does not weight the feature information of different feature layers, and further improving the feature fusion efficiency of the original model. And the feature information weighting is given, so that the difference of contribution degrees of the feature information of different feature layers to the fused output feature information is considered.
In addition, the paper cup defect forms contain different types of defect forms and are similar in characteristic expression, if the type of pictures are input into a conventional model replaced by BiFPN for characteristic fusion, the different types of defects with similar characteristic expression can reduce the characteristic fusion effect of the model under the BiFPN weighting operation of each scale, and the problem can be effectively avoided by only referencing the weighting strategy and the trans-scale connection method of the BiFPN to the PANet characteristic fusion network.
Fig. 3 is a mAP of mean precision mAP before and after improvement provided by the present invention, fig. 4 is a mAP of precision mAP before and after improvement provided by the present invention, and fig. 5 is a mAP of recall ratio mAP before and after improvement provided by the present invention. As shown in each figure, it was verified through one experiment that the total number of iterations was set to 200, the iteration batch size was set to 16, and the picture size was set to 640×640. The improved model detection precision P is 89.1%, the recall rate R is 90.4%, the average precision average mAP is 89.5%, compared with the original model, the precision P is improved by 1.5%, the recall rate R is improved by 1.3%, and the average precision average mAP is improved by 1.2%, so that the invention can effectively improve the detection capability and the robustness of the model, has better target resolution capability, and particularly has obvious improvement on the detection effect of small-size and unobvious paper cup defects.
Fig. 6a to 6c are graphs for comparing actual effects before and after improvement provided by the present invention, and it can be seen that defects which cannot be detected by the conventional model (left side) can be detected based on the improved model (right side) of the present invention.
In some embodiments, the inputting the paper cup image into a trained detection model comprises: and up-sampling the features before the third-scale output layer, performing feature fusion with the features before the fourth-scale output layer, and determining the fourth-scale output result according to the fused features.
Specifically, in the embodiment of the present invention, before the output layer of the third scale (such as the 80×80 detection layer), the features thereof are up-sampled, and then feature fusion is performed with the features of the output layer of the fourth scale (such as the 160×160 detection layer). If the output layer of the fourth scale is 160×160, 2 times up-sampling is performed on the output layer of the third scale, and then fusion is performed. Based on this, the ability of the model to detect small objects and objects with insignificant features can be improved.
The paper cup defect detection device provided by the invention is described below, and the paper cup defect detection device described below and the paper cup defect detection method described above can be referred to correspondingly.
Fig. 7 is a schematic structural diagram of a paper cup defect detecting device according to the present invention, as shown in fig. 7, the paper cup defect detecting device includes: an acquisition module 701 and a processing module 702. The acquisition module 701 is used for acquiring a paper cup image to be detected; the processing module 702 is configured to input the paper cup image into the trained detection model, and output a defect detection result of the paper cup.
The detection model is a YOLO v5 network model, a CBAM attention mechanism module is added after a last layer of C3 module of a back box part, a weighted bidirectional feature pyramid network is utilized for feature fusion in a Neck part, a fourth-scale detection layer is added on an output layer for construction, and the detection model is obtained after training according to a paper cup image training sample marked with a defective label; the scale of the fourth scale detection layer is larger than that of the original other three detection layers.
The embodiment of the device provided by the embodiment of the present invention is for implementing the above embodiments of the method, and specific flow and details refer to the above embodiments of the method, which are not repeated herein.
The paper cup defect detection device provided by the embodiment of the invention has the same implementation principle and technical effects as those of the paper cup defect detection method embodiment, and for brief description, reference can be made to the corresponding content in the paper cup defect detection method embodiment for the non-mention part of the paper cup defect detection device embodiment.
Fig. 8 is a schematic structural diagram of an electronic device according to the present invention, as shown in fig. 8, the electronic device may include: a processor 801, a communication interface (Communications Interface) 802, a memory 803, and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804. Processor 801 may invoke logic instructions in memory 803 to perform a paper cup defect detection method comprising: acquiring a paper cup image to be detected; inputting the paper cup image into a trained detection model, and outputting a defect detection result of the paper cup; the detection model is a YOLO v5 network model, a CBAM attention mechanism module is added after a last layer of C3 module of a back box part, a weighted bidirectional feature pyramid network is utilized for feature fusion in a Neck part, a fourth-scale detection layer is added on an output layer for construction, and the detection model is obtained after training according to a paper cup image training sample marked with a defective label; the scale of the fourth scale detection layer is larger than that of the original other three detection layers.
Further, the logic instructions in the memory 803 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can execute the paper cup defect detection method provided by the above methods, and the method includes: acquiring a paper cup image to be detected; inputting the paper cup image into a trained detection model, and outputting a defect detection result of the paper cup; the detection model is a YOLO v5 network model, a CBAM attention mechanism module is added after a last layer of C3 module of a back box part, a weighted bidirectional feature pyramid network is utilized for feature fusion in a Neck part, a fourth-scale detection layer is added on an output layer for construction, and the detection model is obtained after training according to a paper cup image training sample marked with a defective label; the scale of the fourth scale detection layer is larger than that of the original other three detection layers.
In yet another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the paper cup defect detection method provided by the above methods, the method comprising: acquiring a paper cup image to be detected; inputting the paper cup image into a trained detection model, and outputting a defect detection result of the paper cup; the detection model is a YOLO v5 network model, a CBAM attention mechanism module is added after a last layer of C3 module of a back box part, a weighted bidirectional feature pyramid network is utilized for feature fusion in a Neck part, a fourth-scale detection layer is added on an output layer for construction, and the detection model is obtained after training according to a paper cup image training sample marked with a defective label; the scale of the fourth scale detection layer is larger than that of the original other three detection layers.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method for detecting defects in paper cups, comprising:
acquiring a paper cup image to be detected;
inputting the paper cup image into a trained detection model, and outputting a defect detection result of the paper cup;
the detection model is a YOLO v5 network model, a CBAM attention mechanism module is added after a last layer of C3 module of a back box part, a weighted bidirectional feature pyramid network is utilized for feature fusion in a Neck part, a fourth-scale detection layer is added on an output layer for construction, and the detection model is obtained after training according to a paper cup image training sample marked with a defective label; the scale of the fourth scale detection layer is larger than that of the original other three detection layers.
2. The paper cup defect detection method of claim 1, wherein before inputting the paper cup image into the trained detection model, further comprising:
obtaining unqualified paper cup images of various defect types, and constructing a data set;
performing data enhancement operation on the data set, and adding the paper cup image in the data set into a label of a corresponding defect result to obtain a training sample;
training the constructed YOLO v5 network model by using the training sample to obtain the trained detection model;
wherein the data enhancement operations include rotation, clipping, and brightness enhancement.
3. The method of claim 2, wherein the last layer of the inspection model backup part is SPPF module, and a CBAM attention mechanism module is added between the last layer C3 module and SPPF module.
4. The paper cup defect detection method of claim 3, wherein said inputting said paper cup image into a trained detection model comprises:
inputting the paper cup image to be detected into a backbox part, and sequentially obtaining a first characteristic, a second characteristic, a third characteristic and a fourth characteristic of the paper cup image processed by the first C3 module, the second C3 module, the third C3 module and the SPPF module according to the initial sequence of the input direction;
tensor splicing is carried out according to the third feature and the fourth feature, and a result is input into a fifth C3 module to obtain a fifth feature;
tensor splicing is carried out according to the fifth feature and the second feature, and a result is input into a sixth C3 module to obtain a sixth feature;
tensor splicing is carried out according to the sixth feature and the first feature, a result is input into a seventh C3 module to obtain a seventh feature, and output of a fourth scale is determined according to the seventh feature;
tensor splicing is carried out according to the sixth feature and the seventh feature, a result is input into an eighth C3 module to obtain an eighth feature, and output of a third scale is determined according to the eighth feature;
according to the third feature, the fifth feature and the eighth feature, feature fusion is carried out based on a weighted bidirectional feature pyramid network, a result is input into a ninth C3 module to obtain a ninth feature, and output of a second scale is determined according to the ninth feature;
tensor splicing is carried out according to the ninth feature and the fourth feature, a result is input into a tenth C3 module to obtain a tenth feature, and output of a first scale is determined according to the tenth feature;
wherein, the first scale to the fourth scale are arranged from small to large in sequence.
5. The paper cup defect detection method of claim 1, wherein said inputting the paper cup image into a trained detection model comprises:
and up-sampling the features before the third-scale output layer, performing feature fusion with the features before the fourth-scale output layer, and determining the fourth-scale output result according to the fused features.
6. A paper cup defect detection device, comprising:
the acquisition module is used for acquiring a paper cup image to be detected;
the processing module is used for inputting the paper cup image into the trained detection model and outputting a defect detection result of the paper cup;
the detection model is a YOLO v5 network model, a CBAM attention mechanism module is added after a last layer of C3 module of a back box part, a weighted bidirectional feature pyramid network is utilized for feature fusion in a Neck part, a fourth-scale detection layer is added on an output layer for construction, and the detection model is obtained after training according to a paper cup image training sample marked with a defective label; the scale of the fourth scale detection layer is larger than that of the original other three detection layers.
7. The paper cup defect detection device of claim 6, wherein the processing module is specifically configured to:
inputting the paper cup image to be detected into a backbox part, and sequentially obtaining a first characteristic, a second characteristic, a third characteristic and a fourth characteristic of the paper cup image processed by the first C3 module, the second C3 module, the third C3 module and the SPPF module according to the initial sequence of the input direction;
tensor splicing is carried out according to the third feature and the fourth feature, and a result is input into a fifth C3 module to obtain a fifth feature;
tensor splicing is carried out according to the fifth feature and the second feature, and a result is input into a sixth C3 module to obtain a sixth feature;
tensor splicing is carried out according to the sixth feature and the first feature, a result is input into a seventh C3 module to obtain a seventh feature, and output of a fourth scale is determined according to the seventh feature;
tensor splicing is carried out according to the sixth feature and the seventh feature, a result is input into an eighth C3 module to obtain an eighth feature, and output of a third scale is determined according to the eighth feature;
according to the third feature, the fifth feature and the eighth feature, feature fusion is carried out based on a weighted bidirectional feature pyramid network, a result is input into a ninth C3 module to obtain a ninth feature, and output of a second scale is determined according to the ninth feature;
tensor splicing is carried out according to the ninth feature and the fourth feature, a result is input into a tenth C3 module to obtain a tenth feature, and output of a first scale is determined according to the tenth feature;
wherein, the first scale to the fourth scale are arranged from small to large in sequence.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the paper cup defect detection method of any of claims 1 to 6 when the program is executed by the processor.
9. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the paper cup defect detection method of any of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the paper cup defect detection method according to any of claims 1 to 6.
CN202310377132.5A 2023-04-10 2023-04-10 Paper cup defect detection method and device Pending CN116309526A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310377132.5A CN116309526A (en) 2023-04-10 2023-04-10 Paper cup defect detection method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310377132.5A CN116309526A (en) 2023-04-10 2023-04-10 Paper cup defect detection method and device

Publications (1)

Publication Number Publication Date
CN116309526A true CN116309526A (en) 2023-06-23

Family

ID=86836130

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310377132.5A Pending CN116309526A (en) 2023-04-10 2023-04-10 Paper cup defect detection method and device

Country Status (1)

Country Link
CN (1) CN116309526A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117314868A (en) * 2023-10-10 2023-12-29 山东未来网络研究院(紫金山实验室工业互联网创新应用基地) YOLOv 5-based steel coil end face defect detection method, device and medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117314868A (en) * 2023-10-10 2023-12-29 山东未来网络研究院(紫金山实验室工业互联网创新应用基地) YOLOv 5-based steel coil end face defect detection method, device and medium
CN117314868B (en) * 2023-10-10 2024-03-19 山东未来网络研究院(紫金山实验室工业互联网创新应用基地) YOLOv 5-based steel coil end face defect detection method, device and medium

Similar Documents

Publication Publication Date Title
CN112651978B (en) Sublingual microcirculation image segmentation method and device, electronic equipment and storage medium
CN111161311A (en) Visual multi-target tracking method and device based on deep learning
CN110991560B (en) Target detection method and system combining context information
US20230360390A1 (en) Transmission line defect identification method based on saliency map and semantic-embedded feature pyramid
CN114972191A (en) Method and device for detecting farmland change
CN111612789A (en) Defect detection method based on improved U-net network
CN112651989B (en) SEM image molecular sieve particle size statistical method and system based on Mask RCNN example segmentation
CN116309526A (en) Paper cup defect detection method and device
CN110619619A (en) Defect detection method and device and electronic equipment
CN110599455A (en) Display screen defect detection network model, method and device, electronic equipment and storage medium
CN114742799A (en) Industrial scene unknown type defect segmentation method based on self-supervision heterogeneous network
CN114821350B (en) Building change detection method for high-resolution remote sensing image through multi-stage information fusion
CN116152209A (en) Earphone cover defect detection method, device, equipment and storage medium
CN116703885A (en) Swin transducer-based surface defect detection method and system
CN114332084A (en) PCB surface defect detection method based on deep learning
CN117437227A (en) Image generation and defect detection method, device, medium, equipment and product
Han et al. Remote sensing sea ice image classification based on multilevel feature fusion and residual network
CN116523897A (en) Semi-supervised enteromorpha detection method and system based on transconductance learning
CN116977249A (en) Defect detection method, model training method and device
CN115656952A (en) Underground target multi-classification detection method based on U-shaped multi-dimensional multi-scale fusion neural network
US20220198258A1 (en) Saliency Prioritization for Image Processing
CN113763384A (en) Defect detection method and defect detection device in industrial quality inspection
CN105335935B (en) Image processing apparatus and method
CN113034432A (en) Product defect detection method, system, device and storage medium
CN118115835A (en) Light guide plate defect small sample data expansion method, system, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination