CN113256623A - FPC defect detection method based on improved MASK RCNN - Google Patents
FPC defect detection method based on improved MASK RCNN Download PDFInfo
- Publication number
- CN113256623A CN113256623A CN202110723382.0A CN202110723382A CN113256623A CN 113256623 A CN113256623 A CN 113256623A CN 202110723382 A CN202110723382 A CN 202110723382A CN 113256623 A CN113256623 A CN 113256623A
- Authority
- CN
- China
- Prior art keywords
- fpc
- defect
- image
- model
- defect detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
Abstract
The invention discloses an FPC defect detection method based on improved MASK RCNN, which comprises the following steps: s1, ROI processing and image cutting, wherein a target detection image is collected, FPC original image data is preprocessed, and the small image suitable for network input is cut; s2, performing data enhancement processing on the preprocessed image to enlarge a data set of model training; s3, manual labeling; s4, determining the transverse stacking times N of the feature map blocks, and determining the transverse stacking times N of the feature map blocks according to the type and the quantity of FPC defect detection; s5, model training, namely, sending the marked image data set into an improved MAKS RCNN network model for training; s6, performing performance evaluation on the trained FPC defect detection model; and S7, optimizing and fine-tuning parameters, and further optimizing the model by combining the evaluation result of S6. The invention realizes the automatic detection of FPC defect, and the defect size can be obtained by dividing the image defect through the mask, thereby facilitating the reevaluation of the working personnel and greatly reducing the detection cost of enterprises.
Description
Technical Field
The invention relates to the technical field of flexible circuit board defect detection and image segmentation, in particular to an FPC defect detection method based on improved MASK RCNN.
Background
The flexible Printed Circuit board is called as fpc (flexible Printed Circuit board) for short, and has the characteristics of thin thickness, light weight, and free bending and folding. Compared with the traditional circuit board, the FPC occupies less space, can greatly reduce the packaging size and weight so as to meet the requirements of high integration and mobility of electronic products, can realize the three-dimensional space wiring of the circuit, enhances the reliability of the products and reduces the assembly cost. Because the FPC material is special, has high integration level and complex process, the manufacturing process is easily affected by factors such as equipment, personnel, environment and the like to generate defects.
The MASK RCNN network is an example segmentation model based on RCNN network improvement, and can accurately segment pixels corresponding to an object while detecting the object. The feature extraction of the MASK RCNN mainly comprises a residual error network and a feature pyramid network, the extracted feature graph is sent to an RPN network to screen candidate frames, and finally target detection and segmentation are finished by a classification branch, a detection branch and a MASK branch; however, compared with the PCB, the FPC has the defect characteristics of more image types, more quantity, random positions, less characteristic information, small defects and the like; the traditional image processing method is difficult to realize accurate detection and segmentation of such FPC defect images, and the MASK RCNN network model is also difficult to accurately detect, classify and segment FPC defects, so that an improved MASK RCNN-based FPC defect detection method is urgently needed to solve the problems.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides an FPC defect detection method based on an improved MASK RCNN, which realizes automatic detection, classification and segmentation of main defects of an FPC, saves the labor cost of enterprises, and greatly improves the defect detection efficiency.
The technical scheme of the invention is as follows:
the method comprises the following steps:
s1, ROI processing and image cutting, wherein a field target detection image is collected, and the data of an FPC original image is preprocessed, wherein the preprocessing comprises ROI processing and image cutting;
s2, performing data enhancement processing on the preprocessed image;
s3, manual marking, namely performing manual marking on the image data after the enhancement processing, and then dividing a data set;
s4, determining the transverse stacking times N of the feature map blocks, wherein the transverse stacking times N of the feature map blocks are determined according to the types and the quantity of FPC defects to be identified;
s5, model training, namely sending the marked image data into an improved MASK RCNN network model for training to obtain an FPC defect detection model;
s6, performing performance evaluation on the trained FPC defect detection model;
and S7, optimizing and fine-tuning parameters, and further optimizing and fine-tuning the FPC defect detection model by combining the evaluation result of S6.
Further, S5 specifically includes the following steps:
s51: setting the pixel area of the RPN network anchor to 82，162，322，642，2562The aspect ratio of the anchors is set to be {3:4, 1:1, 3:1}, and the step size of the anchors is set to be 4;
s52: extracting network extraction features by using EfficientNet as a backbone feature;
s53: combining the bottom layer characteristic graph and the high layer characteristic graph semantic information to create a characteristic graph block from left to right from bottom to top;
s54: the threshold value for judging defects by the model is set to G, and if the model output score is smaller than the threshold value G, the defect is judged to be not present, and if the score is larger than the threshold value G, the defect is judged to be present.
Furthermore, the specific method of ROI processing is to cut out a portion to be detected, i.e., an ROI (region of interest) image, from the original high-resolution FPC image, and cut out the ROI image into K pieces, where the K pieces of cut-out image have a maximum resolution of 1024 × 1024 and a minimum resolution of 256 × 256, and K selects a value from the natural number set {2, 3, 4} according to the original high-resolution FPC image.
Further, the data enhancement processing method adopts image rotation, image mirroring, image contrast adjustment, image translation and random deletion to increase the training data set.
Furthermore, the manual labeling adopts a Labelme labeling tool, the real position of the defect is labeled on the data set, and the labeled data set is divided into a training set and a verification set according to the proportion of 4 to 1.
Furthermore, the number of times of transverse stacking of the feature map blocks is N times, and N selects a numerical value in the natural number set {3, 4, 5, 6} according to the FPC defect type.
Further, the specific implementation steps of the feature map block are as follows: performing 1 × 1 convolution operation on 5 feature maps C1, C2, C3, C4 and C5 generated by the trunk feature extraction network to generate C1_0, C2_0, C3_0, C4_0 and C5_0, performing upsampling on the last four feature maps C2_0, C3_0, C4_0 and C5_0 to generate C2_1, C3_1, C4_1 and C5_1, performing element-by-element addition and fusion with the previous feature map to generate C1_2, C2_2, C3_2 and C4_2, and performing element-by-element addition on the C2_2, C3_2 and C4_2 generated by the trunk feature extraction network and the C2, C3 and C4 feature maps generated by the trunk feature extraction network, wherein the steps are stacking feature maps N times; finally, performing 3-by-3 convolution on C1_2 from bottom to top to generate a valid feature map P1; after the maximum pooling operation of P1, adding C2_2 element by element, and performing 3-by-3 convolution to generate an effective characteristic map P2; performing 3 × 3 convolution and maximum pooling on the effective feature map P2, adding the effective feature map P2 to C3_2 element by element, and performing 3 × 3 convolution to generate an effective feature map P3; performing 3 × 3 convolution and maximum pooling on the effective feature map P3, adding the effective feature map P3 to C4_2 element by element, and performing 3 × 3 convolution to generate an effective feature map P4; c5_2 is convolved 3 x3 with P4 and maximally pooled, and then added pixel by pixel, and further convolved 3 x3 to generate the active signature P5.
Further, when the number of times N of transverse stacking and the number N of FPC defect types is more than or equal to 1 and less than or equal to 9, taking N as 3; when the number N of the FPC defect types is more than or equal to 10 and less than or equal to 14, N is 4; when the number N of the FPC defect types is more than or equal to 15 and less than or equal to 20, taking N as 5; when the FPC defect type N exceeds 20, taking N as 6;
further, according to the characteristics of FPC defect, the pixel area of RPN network anchor is set to be 82，162，322，642，2562The aspect ratio of the anchors is set to be {3:4, 1:1, 3:1}, and the step size of the anchors is set to be 4;
further, the performance evaluation of the trained FPC defect detection model is mainly divided into three categories, namely defect positioning, defect classification and defect segmentation; for defect positioning and defect classification, calculating the average precision mAP as an evaluation index; for defect segmentation, calculating the ratio of a defect segmentation pixel to a defect actual pixel as an evaluation index; and if the mAP of the model is more than or equal to 90% and the ratio of correctly dividing the pixels is more than or equal to 60%, determining that the model can accurately detect and divide the defects of the FPC, otherwise, optimizing and finely adjusting the model.
Further, further optimization and fine tuning of the FPC defect detection model are mainly achieved by increasing the number of training samples, adjusting the transverse stacking times N of the feature map blocks, and modifying the learning rate and the iteration times.
Compared with the prior art, the invention has the following remarkable advantages:
(1) the invention enhances the detection capability of the network model for small targets and targets with single characteristics;
(2) the invention realizes the automatic detection, classification and segmentation of FPC defects;
(3) the invention solves the problem that the traditional algorithm can only detect specific defects, and saves labor cost;
(4) the segmented defect image is covered by a mask with random colors, so that field workers can conveniently judge the defect image, and the number of the defect pixels can be obtained through the number of the mask pixels.
Drawings
FIG. 1 is a flowchart illustrating an FPC defect detection method based on modified MASK RCNN according to the present embodiment;
fig. 2 is a block network structure of a feature map of the present example.
Detailed Description
The objects, technical solutions and advantages of the present invention will be further described with reference to the accompanying drawings. It should be understood that the embodiments described herein are not intended to be construed as merely illustrative of the present invention and not limitative of the scope thereof.
As shown in fig. 1, a FPC defect detection method based on improved MASK RCNN includes the following steps:
step 1, ROI processing and image cutting, wherein a target detection image of an industrial field is acquired, and FPC original image data is preprocessed, wherein the preprocessing comprises the ROI processing and the image cutting.
And 2, performing data enhancement processing on the preprocessed image, wherein the data enhancement processing adopts a traditional image processing method: image rotation, image mirroring, image contrast adjustment, image translation and random deletion, and the data set is enlarged through data enhancement processing, so that the robustness of the model is improved.
And 3, manually labeling the FPC image data after the enhancement processing, wherein Labelme software is used for labeling the FPC image data set in the embodiment, and recording the real pixel point coordinates of the defects in the data set. Dividing a training set and a verification set into data sets according to the proportion of 4 to 1, wherein image data of the training set is used for training a model, image data of the verification set is used for verifying the detection capability of the model, and the model continuously adjusts network parameters according to the verification effect through a large amount of training.
Step 4, determining the transverse stacking times N of feature block, please refer to fig. 2, where fig. 2 is a network structure of the feature block of the present example, where a plus sign (+) in the figure indicates element-by-element addition, a dashed box indicates the feature block, and for the types and number of FPC defects to be identified, determining the transverse stacking times N of the feature block, where N is equal to or greater than 1 and equal to or less than 9, and where N is 3; when the number N of the defect types is more than or equal to 10 and less than or equal to 14, taking N as 4; when the number N of the defect types is more than or equal to 15 and less than or equal to 20, taking N as 5; when the number N of defect types exceeds 20, taking N as 6; the specific implementation steps of the feature map block are as follows: performing 1 × 1 convolution operation on 5 feature maps C1, C2, C3, C4 and C5 generated by the trunk feature extraction network to generate C1_0, C2_0, C3_0, C4_0 and C5_0, performing upsampling on the last four feature maps C2_0, C3_0, C4_0 and C5_0 to generate C2_1, C3_1, C4_1 and C5_1, performing element-by-element addition and fusion with the previous feature map to generate C1_2, C2_2, C3_2 and C4_2, and performing element-by-element addition on the C2_2, C3_2 and C4_2 generated by the trunk feature extraction network and the C2, C3 and C4 feature maps generated by the trunk feature extraction network, wherein the steps are stacking feature maps N times; finally, performing 3-by-3 convolution on C1_2 from bottom to top to generate a valid feature map P1; after the maximum pooling operation of P1, adding C2_2 element by element, and performing 3-by-3 convolution to generate an effective characteristic map P2; performing 3 × 3 convolution and maximum pooling on the effective feature map P2, adding the effective feature map P2 to C3_2 element by element, and performing 3 × 3 convolution to generate an effective feature map P3; performing 3 × 3 convolution and maximum pooling on the effective feature map P3, adding the effective feature map P3 to C4_2 element by element, and performing 3 × 3 convolution to generate an effective feature map P4; c5_2 is convolved 3 x3 with P4 and maximally pooled, and then added pixel by pixel, and further convolved 3 x3 to generate the active signature P5.
Step 5, model training, namely setting the pixel area of the RPN network anchor as 82，162，322，642，2562Setting the width-to-height ratio of the anchors as {3:4, 1:1, 3:1}, setting the step length of the anchors as 4, setting the threshold value of the defects judged by the model as G, judging whether the defects exist if the output score of the model is smaller than the threshold value G, judging whether the defects exist if the output score of the model is larger than the threshold value G, sending the marked FPC image data into an improved MASK RCNN network model for training, and training an FPC defect detection model.
Step 6, performing performance evaluation on the trained FPC defect detection model, wherein the model evaluation is mainly divided into three categories of defect positioning, defect classification and defect segmentation; for defect positioning and defect classification, calculating the average precision mAP as an evaluation index; for defect segmentation, calculating the ratio of a defect segmentation pixel to a defect actual pixel as an evaluation index; and testing the FPC image by using the trained FPC detection model, if the mAP of the model is more than or equal to 90% and the ratio of correctly dividing pixels is more than or equal to 60%, determining that the model can accurately detect and divide the defects of the FPC, and otherwise, optimizing and finely adjusting the model.
And 7, optimizing and fine-tuning parameters, further optimizing and fine-tuning the FPC defect detection model by combining the evaluation result of the step 6, and mainly realizing the optimization and fine-tuning by increasing the number of training samples, adjusting the transverse stacking times N of the feature diagram blocks, modifying the learning rate and the iteration times.
Examples
In the embodiment, the method for detecting the FPC defect based on the improved MASK RCNN comprises the steps of firstly preprocessing an acquired original FPC image, then performing data enhancement on the preprocessed image to enlarge the data volume of the FPC image, then manually marking the enhanced image data, dividing a marked data set into a training set and a verification set according to the proportion of 4: 1, then sending the divided data set into a model for training, finally evaluating the trained model, and judging whether the model meets the FPC detection requirement. The method comprises the following specific steps:
step 1: and ROI processing and image cutting, namely acquiring a target detection image of an industrial field, and preprocessing the data of the FPC original image, including ROI processing and image cutting.
Step 2: and performing data enhancement processing on the preprocessed image, wherein the data enhancement processing adopts a traditional image processing method: image rotation, image mirroring, image contrast adjustment, image translation and random deletion, and the data set is expanded to 3 times of the original data set through data enhancement processing.
And step 3: and manual labeling, namely performing manual labeling on the FPC image data after the enhancement processing, and labeling the FPC image data set by using Labelme software in the embodiment and recording the coordinates of the real pixel points of the defects in the data set. Dividing a training set and a verification set into data sets according to the proportion of 4 to 1, wherein image data of the training set is used for training a model, image data of the verification set is used for verifying the detection capability of the model, and the model continuously adjusts network parameters according to the verification effect through a large amount of training.
And 4, step 4: determining the transverse stacking times N of the feature map block, and determining the transverse stacking times N of the feature map block aiming at the types and the number of FPC (flexible printed circuit) defects to be identified, wherein the number N of the types of the FPC data defects collected in the embodiment is 3, the value of N is within the range that N is not less than 1 and not more than 9, and the transverse stacking times N are 3;
and 5: model training, the pixel area of the RPN network anchor is set as 82，162，322，642，2562The aspect ratio of the anchors is set to be {3:4, 1:1, 3:1}, and the step size of the anchors is set to be 4; and setting a threshold value of the model for judging the defects to be 0.7, judging that the defects do not exist if the output score of the model is less than the threshold value of 0.7, judging that the defects exist if the output score of the model is greater than the threshold value of 0.7, and sending the marked FPC image data into an improved MASK RCNN network model for training to obtain an FPC defect detection model.
Step 6: performing performance evaluation on the trained FPC defect detection model, wherein the model evaluation mainly comprises three types of defect positioning, defect classification and defect segmentation; for defect positioning and defect classification, calculating the average precision mAP as an evaluation index; for defect segmentation, calculating the ratio of a defect segmentation pixel to a defect actual pixel as an evaluation index; and testing the FPC image by using the trained FPC detection model, if the mAP of the model is more than or equal to 90% and the ratio of correctly dividing pixels is more than or equal to 60%, determining that the model can accurately detect and divide the defects of the FPC, and otherwise, optimizing and finely adjusting the model.
And 7: and (4) optimizing and fine-tuning parameters, further optimizing and fine-tuning the FPC defect detection model by combining the evaluation result of the step (6), and mainly realizing the optimization and fine-tuning by increasing the number of training samples, adjusting the transverse stacking times N of the feature diagram block, modifying the learning rate and the iteration times.
This example uses a pytorech deep learning framework with experimental computer hardware configured to: AMD Ryzen 93900X 12 core 24 thread processor, GeForce RTX3090 graphics card, 32G operation memory, 500G NVME solid state disk; the experimental software environment is 64-bit Windows 10 professional edition, Pycharm integrated development environment, CUDA version is 11.1, CUDNN version is 8.0.4.30, and the final result is shown in Table 1.
TABLE 1 results of the experiments
As can be seen from table 1, after training, the average accuracy maps of the defect detection of the 2 types of FPC images ((the first type of FPC image (FPC 1) and the second type of FPC image (FPC 2)) are all over 90%, the correct segmented pixel ratio of the FPC1 is 72.3%, the FPC2 data set is small, the correct segmented pixel ratio is 63.6%, and the average detection time of a single image is over 23 milliseconds.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (9)
1. An FPC defect detection method based on improved MASK RCNN is characterized by comprising the following steps:
s1, ROI processing and image cutting, wherein a field target detection image is collected, and the data of an FPC original image is preprocessed, wherein the preprocessing comprises ROI processing and image cutting;
s2, performing data enhancement processing on the preprocessed image;
s3, manual marking, namely performing manual marking on the image data after the enhancement processing, and then dividing a data set;
s4, determining the transverse stacking times N of the feature map blocks, wherein the transverse stacking times N of the feature map blocks are determined according to the types and the quantity of FPC defects to be identified;
s5, model training, namely sending the marked image data into an improved MASK RCNN network model for training to obtain an FPC defect detection model;
s6, performing performance evaluation on the trained FPC defect detection model;
and S7, optimizing and fine-tuning parameters, and further optimizing and fine-tuning the FPC defect detection model by combining the evaluation result of S6.
2. The method for FPC defect detection based on modified MASK RCNN according to claim 1, wherein: the specific method of ROI processing is that a part needing to be detected, namely an ROI image, is cut out of an original high-resolution FPC image, then the ROI image is cut out into K parts, the maximum resolution of the K parts of cut images is 1024 x 1024, the minimum resolution is 256 x 256, and K selects a numerical value in a natural number set {2, 3, 4} according to the original high-resolution FPC image.
3. The method for FPC defect detection based on modified MASK RCNN according to claim 1, wherein: the data enhancement processing method adopts image rotation, image mirroring, image contrast adjustment, image translation and random deletion to increase a training data set.
4. The method for FPC defect detection based on modified MASK RCNN according to claim 1, wherein: and the manual marking adopts a Labelme marking tool, the real position of the defect is marked on the data set, and the marked data set is divided into a training set and a verification set according to the proportion of 4 to 1.
5. The method for FPC defect detection based on modified MASK RCNN according to claim 1, wherein: the number of times of transverse stacking of the feature graphs is N times, N selects a numerical value in a natural number set {3, 4, 5, 6} according to the FPC defect types, and when the number N of the FPC defect types is more than or equal to 1 and less than or equal to 9, N is 3; when the number N of the defect types is more than or equal to 10 and less than or equal to 14, taking N as 4; when the number N of the defect types is more than or equal to 15 and less than or equal to 20, taking N as 5; when the number N of defect types exceeds 20, N is 6.
6. The method according to claim 1, wherein the step S5 specifically comprises the following steps:
s51: setting the pixel area of the RPN network anchor to 82，162，322，642，2562The aspect ratio of the anchors is set to be {3:4, 1:1, 3:1}, and the step size of the anchors is set to be 4;
s52: extracting network extraction features by using EfficientNet as a backbone feature;
s53: combining the bottom layer characteristic graph and the high layer characteristic graph semantic information to create a characteristic graph block from left to right from bottom to top;
s54: the threshold value for judging defects by the model is set to G, and if the model output score is smaller than the threshold value G, the defect is judged to be not present, and if the score is larger than the threshold value G, the defect is judged to be present.
7. The method according to claim 6, wherein the method comprises the following steps: the specific method of S53 is as follows: performing 1 × 1 convolution operation on 5 feature maps C1, C2, C3, C4 and C5 generated by the trunk feature extraction network to generate C1_0, C2_0, C3_0, C4_0 and C5_0, performing upsampling on the last four feature maps C2_0, C3_0, C4_0 and C5_0 to generate C2_1, C3_1, C4_1 and C5_1, performing element-by-element addition and fusion with the previous feature map to generate C1_2, C2_2, C3_2 and C4_2, and performing element-by-element addition on the C2_2, C3_2 and C4_2 generated by the trunk feature extraction network and the C2, C3 and C4 feature maps generated by the trunk feature extraction network, wherein the steps are stacking feature maps N times; finally, performing 3-by-3 convolution on C1_2 from bottom to top to generate a valid feature map P1; after the maximum pooling operation of P1, adding C2_2 element by element, and performing 3-by-3 convolution to generate an effective characteristic map P2; performing 3 × 3 convolution and maximum pooling on the effective feature map P2, adding the effective feature map P2 to C3_2 element by element, and performing 3 × 3 convolution to generate an effective feature map P3; performing 3 × 3 convolution and maximum pooling on the effective feature map P3, adding the effective feature map P3 to C4_2 element by element, and performing 3 × 3 convolution to generate an effective feature map P4; c5_2 is convolved 3 x3 with P4 and maximally pooled, and then added pixel by pixel, and further convolved 3 x3 to generate the active signature P5.
8. The method for FPC defect detection based on modified MASK RCNN according to claim 1, wherein: the performance evaluation of the trained FPC defect detection model is mainly divided into three categories of defect positioning, defect classification and defect segmentation; for defect positioning and defect classification, calculating the average precision mAP as an evaluation index; for defect segmentation, calculating the ratio of a defect segmentation pixel to a defect actual pixel as an evaluation index; and if the mAP of the model is more than or equal to 90% and the ratio of correctly dividing the pixels is more than or equal to 60%, determining that the model can accurately detect and divide the defects of the FPC, otherwise, optimizing and finely adjusting the model.
9. The method for FPC defect detection based on modified MASK RCNN according to claim 1, wherein: the further optimization and fine tuning of the FPC defect detection model are mainly realized by increasing the number of training samples, adjusting the transverse stacking times N of block of the feature diagram, and modifying the learning rate and the iteration times.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110723382.0A CN113256623B (en) | 2021-06-29 | 2021-06-29 | FPC defect detection method based on improved MASK RCNN |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110723382.0A CN113256623B (en) | 2021-06-29 | 2021-06-29 | FPC defect detection method based on improved MASK RCNN |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113256623A true CN113256623A (en) | 2021-08-13 |
| CN113256623B CN113256623B (en) | 2021-10-26 |
Family
ID=77189989
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110723382.0A Active CN113256623B (en) | 2021-06-29 | 2021-06-29 | FPC defect detection method based on improved MASK RCNN |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113256623B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113793343A (en) * | 2021-08-23 | 2021-12-14 | 南昌航空大学 | Defect positioning and segmentation method, system, terminal and storage medium based on image |
| CN114419081A (en) * | 2022-03-28 | 2022-04-29 | 南昌工程学院 | Image semantic segmentation method and system and readable storage medium |
| CN114462558A (en) * | 2022-04-13 | 2022-05-10 | 南昌工程学院 | Data-augmented supervised learning image defect classification method and system |
| CN117218121A (en) * | 2023-11-08 | 2023-12-12 | 北京航空航天大学江西研究院 | Casting DR image defect identification method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110798A (en) * | 2019-05-13 | 2019-08-09 | 广东工业大学 | A kind of weld joint recognition method based on Mask-RCNN network |
| CN110163207A (en) * | 2019-05-20 | 2019-08-23 | 福建船政交通职业学院 | One kind is based on Mask-RCNN ship target localization method and storage equipment |
| WO2020007118A1 (en) * | 2018-07-02 | 2020-01-09 | 北京百度网讯科技有限公司 | Display screen peripheral circuit detection method and device, electronic equipment and storage medium |
| CN110853015A (en) * | 2019-11-12 | 2020-02-28 | 中国计量大学 | Aluminum profile defect detection method based on improved Faster-RCNN |
| CN110992318A (en) * | 2019-11-19 | 2020-04-10 | 上海交通大学 | Special metal flaw detection system based on deep learning |
| CN111862025A (en) * | 2020-07-14 | 2020-10-30 | 中国船舶重工集团公司第七一六研究所 | PCB defect automatic detection method based on deep learning |
| US20200394413A1 (en) * | 2019-06-17 | 2020-12-17 | The Regents of the University of California, Oakland, CA | Athlete style recognition system and method |
| CN112991267A (en) * | 2021-02-08 | 2021-06-18 | 上海工程技术大学 | Strip steel surface defect detection method based on improved efficientNet-RCNN |
-
2021
- 2021-06-29 CN CN202110723382.0A patent/CN113256623B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020007118A1 (en) * | 2018-07-02 | 2020-01-09 | 北京百度网讯科技有限公司 | Display screen peripheral circuit detection method and device, electronic equipment and storage medium |
| CN110110798A (en) * | 2019-05-13 | 2019-08-09 | 广东工业大学 | A kind of weld joint recognition method based on Mask-RCNN network |
| CN110163207A (en) * | 2019-05-20 | 2019-08-23 | 福建船政交通职业学院 | One kind is based on Mask-RCNN ship target localization method and storage equipment |
| US20200394413A1 (en) * | 2019-06-17 | 2020-12-17 | The Regents of the University of California, Oakland, CA | Athlete style recognition system and method |
| CN110853015A (en) * | 2019-11-12 | 2020-02-28 | 中国计量大学 | Aluminum profile defect detection method based on improved Faster-RCNN |
| CN110992318A (en) * | 2019-11-19 | 2020-04-10 | 上海交通大学 | Special metal flaw detection system based on deep learning |
| CN111862025A (en) * | 2020-07-14 | 2020-10-30 | 中国船舶重工集团公司第七一六研究所 | PCB defect automatic detection method based on deep learning |
| CN112991267A (en) * | 2021-02-08 | 2021-06-18 | 上海工程技术大学 | Strip steel surface defect detection method based on improved efficientNet-RCNN |
Non-Patent Citations (3)
| Title |
|---|
| JIE SHI ET AL.: ""Target Detection Based on Improved Mask Rcnn in Service Robot"", 《PROCEEDINGS OF THE 38TH CHINESE CONTROL CONFERENCE》 * |
| 林宗沛: ""基于改进Faster--RCNN的高密度柔性电路基板缺陷检测研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 浦东 等: ""基于轻量级Mask RCNN的均压环歪斜检测"", 《电力信息与通信技术》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113793343A (en) * | 2021-08-23 | 2021-12-14 | 南昌航空大学 | Defect positioning and segmentation method, system, terminal and storage medium based on image |
| CN113793343B (en) * | 2021-08-23 | 2023-06-06 | 南昌航空大学 | Image-based defect positioning and segmentation method, system, terminal and storage medium |
| CN114419081A (en) * | 2022-03-28 | 2022-04-29 | 南昌工程学院 | Image semantic segmentation method and system and readable storage medium |
| CN114462558A (en) * | 2022-04-13 | 2022-05-10 | 南昌工程学院 | Data-augmented supervised learning image defect classification method and system |
| CN117218121A (en) * | 2023-11-08 | 2023-12-12 | 北京航空航天大学江西研究院 | Casting DR image defect identification method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113256623B (en) | 2021-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113256623B (en) | FPC defect detection method based on improved MASK RCNN | |
| CN109583483B (en) | Target detection method and system based on convolutional neural network | |
| CN111753828B (en) | Natural scene horizontal character detection method based on deep convolutional neural network | |
| CN109191424B (en) | Breast mass detection and classification system and computer-readable storage medium | |
| CN111898406A (en) | Face detection method based on focus loss and multitask cascade | |
| CN110751606B (en) | Foam image processing method and system based on neural network algorithm | |
| CN111461101A (en) | Method, device and equipment for identifying work clothes mark and storage medium | |
| CN110598698A (en) | Natural scene text detection method and system based on adaptive regional suggestion network | |
| CN111325728B (en) | Product defect detection method, device, equipment and storage medium | |
| CN111652117A (en) | Method and medium for segmenting multi-document image | |
| CN115205626A (en) | Data enhancement method applied to field of coating defect detection | |
| CN111754502A (en) | Method for detecting surface defects of magnetic core based on fast-RCNN algorithm of multi-scale feature fusion | |
| CN113205511B (en) | Electronic component batch information detection method and system based on deep neural network | |
| CN112434730A (en) | GoogleNet-based video image quality abnormity classification method | |
| CN115861922B (en) | Sparse smoke detection method and device, computer equipment and storage medium | |
| CN110766001A (en) | Bank card number positioning and end-to-end identification method based on CNN and RNN | |
| CN115797314A (en) | Part surface defect detection method, system, equipment and storage medium | |
| CN113435358B (en) | Sample generation method, device, equipment and program product for training model | |
| CN115908356A (en) | PCB defect detection method based on LDLFModel | |
| CN115345895A (en) | Image segmentation method and device for visual detection, computer equipment and medium | |
| CN112633327B (en) | Staged metal surface defect detection method, system, medium, equipment and application | |
| CN115147643A (en) | FPC defect classification method based on CNN and TRANSFORMER | |
| CN112966757A (en) | Method and device for expanding training sample, storage medium and equipment | |
| CN111652012B (en) | Curved surface QR code positioning method based on SSD network model | |
| CN112396582B (en) | Mask RCNN-based equalizing ring skew detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |