CN113469299B - Defect detection method and defect detection device in industrial detection - Google Patents
Defect detection method and defect detection device in industrial detection Download PDFInfo
- Publication number
- CN113469299B CN113469299B CN202111035503.9A CN202111035503A CN113469299B CN 113469299 B CN113469299 B CN 113469299B CN 202111035503 A CN202111035503 A CN 202111035503A CN 113469299 B CN113469299 B CN 113469299B
- Authority
- CN
- China
- Prior art keywords
- workpiece
- picture
- model
- map
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention provides a defect detection method and a defect detection device in industrial detection, wherein the method comprises the following steps: acquiring a picture of a workpiece to be detected; inputting a workpiece picture into a trained defect detection model to obtain a defect prediction result, wherein the defect detection model comprises a MASK-RCNN model and a FPN model, and the defect prediction result comprises a preselected frame; carrying out image registration on the workpiece picture to obtain a front and back background segmentation picture after registration; screening a preselected frame of a defect prediction result according to the registered front and rear background segmentation maps; and obtaining a defect detection result according to the screened preselection frame. According to the method, the image registration and the FPN model are utilized, the area generated by the pre-selection frame is limited, so that the model can be concentrated on the detection of the optical surface area, the detection speed of the model is improved, and the omission ratio is reduced; the image registration is used for screening the pre-selection frame, so that the defects of the non-optical surface can be screened out, and the overdetection rate of the model is reduced.
Description
Technical Field
The invention relates to the technical field of detection, in particular to a defect detection method and a defect detection device in industrial detection.
Background
In the field of industrial quality inspection, the speed, the overdetection rate and the omission factor of a detection model are three important indexes for evaluating the detection model.
In the related art, during the detection process, the detection model can detect all areas in the picture without distinction, which wastes computation power on non-optical surfaces (areas not required to be detected, such as background), and simultaneously causes the reduction of the detection speed, and the detection of objects on the non-optical surfaces also causes the reduction of the accuracy of the model and the increase of the overdetection rate.
Disclosure of Invention
In order to solve the above technical problems, a first object of the present invention is to provide a defect detection method in industrial detection, which utilizes image registration and an FPN (Feature Pyramid Network) model to limit a region generated by a preselected frame, so that the model can be focused on detection of an optical surface region, thereby increasing the detection speed of the model and reducing the omission ratio; the image registration is used for screening the pre-selection frame, so that the defects of the non-optical surface can be screened out, and the overdetection rate of the model is reduced.
A second object of the present invention is to provide a defect detecting apparatus in industrial inspection.
The technical scheme adopted by the invention is as follows:
the embodiment of the first aspect of the invention provides a defect detection method in industrial detection, which comprises the following steps: acquiring a picture of a workpiece to be detected; inputting the workpiece picture into a trained defect detection model to obtain a defect prediction result, wherein the defect detection model comprises a MASK-RCNN (MASK-Region conditional Neural Networks) model and an FPN (FPN) model, and the defect prediction result comprises a preselected frame; carrying out image registration on the workpiece picture to obtain a front background segmentation picture and a rear background segmentation picture after registration; screening a preselected frame of the defect prediction result according to the registered front and rear background segmentation maps; and obtaining a defect detection result according to the screened preselection frame.
The defect detection method in industrial detection provided by the invention can also have the following additional technical characteristics:
according to one embodiment of the invention, the defect detection model is trained in the following way: inputting the training set of the workpiece picture into a main network RESNET-50 of the MASK-RCNN for feature extraction to obtain a feature map; sending the feature map into the FPN model to obtain a segmentation feature map; acquiring a first front and rear background segmentation map according to the segmentation feature map; carrying out image registration on the training set to obtain a registered front and back background segmentation map of the training set; acquiring an intersection of the first front and rear background segmentation map and the registered front and rear background segmentation map to generate a second background segmentation map; inputting the feature map into an RPN (Region delivery Network) Network in the MASK-RCNN to generate a first preselected box; screening out a preselected frame of which the center point falls in a background area of a second background segmentation chart in the first preselected frame to obtain a second preselected frame; further screening the second preselected frame by using a Non-Maximum Suppression (NMS) algorithm to obtain a third preselected frame; acquiring a target feature segmentation map according to the third pre-selection frame and the feature map; performing concat (merging array) operation fusion on the segmentation feature map and the target feature segmentation map to obtain a fusion feature map; acquiring the defect prediction result according to the fusion feature map; and acquiring a training set label, and iterating the model weight according to the training set label and the defect prediction result to perform supervised learning.
According to one embodiment of the invention, the registered segmentation maps of the front and back backgrounds are obtained by the following steps: after the workpiece picture and the standard positioning picture are subjected to scaling and Gaussian filtering, sending the workpiece picture and the standard positioning picture into an image registration model homograph (an image registration model) to obtain a homography transformation matrix of the workpiece picture and the standard positioning picture; acquiring a standard front and back background segmentation chart of the standard positioning picture; and performing homography according to the standard front and rear background segmentation map and the homography transformation matrix to obtain the registered front and rear background segmentation map.
According to an embodiment of the invention, acquiring the defect detection result according to the screened pre-selection frame specifically comprises: judging whether the workpiece has defects according to the screened preselection frame; if the workpiece has defects, sending the workpiece into an NG (No Good) material box; and if the workpiece has no defects, sending the workpiece into a GOOD (GOOD product) material box.
An embodiment of the second aspect of the present invention provides a defect detecting apparatus in industrial inspection, including: the acquisition module is used for acquiring a workpiece picture to be detected; the prediction module is used for inputting the workpiece picture into a trained defect detection model to obtain a defect prediction result, wherein the defect detection model comprises a MASK-RCNN model and a FPN model, and the defect prediction result comprises a preselected frame; the registration module is used for carrying out image registration on the workpiece picture so as to obtain a front and back background segmentation picture after registration; the screening module is used for screening a preselected frame of the defect prediction result according to the registered front and rear background segmentation maps; and the detection module is used for acquiring a defect detection result according to the screened preselection frame.
The defect detection device in industrial detection provided by the invention can also have the following additional technical characteristics:
according to an embodiment of the invention, the prediction module is further configured to: inputting the training set of the workpiece picture into a main network RESNET-50 of the MASK-RCNN for feature extraction to obtain a feature map; sending the feature map into the FPN model to obtain a segmentation feature map; acquiring a first front and rear background segmentation map according to the segmentation feature map; carrying out image registration on the training set to obtain a registered front and back background segmentation map of the training set; acquiring an intersection of the first front and rear background segmentation map and the registered front and rear background segmentation map to generate a second background segmentation map; inputting the feature map into an RPN network in the MASK-RCNN to generate a first preselected box; screening out a preselected frame of which the center point falls in a background area of a second background segmentation chart in the first preselected frame to obtain a second preselected frame; further screening the second preselected frame by using an NMS algorithm to obtain a third preselected frame; acquiring a target feature segmentation map according to the third pre-selection frame and the feature map; performing concat operation fusion on the segmentation feature map and the target feature segmentation map to obtain a fusion feature map; acquiring the defect prediction result according to the fusion feature map; and acquiring a training set label, and iterating the model weight according to the training set label and the defect prediction result to perform supervised learning.
According to an embodiment of the invention, the registration module is specifically configured to: after the workpiece picture and the standard positioning picture are subjected to scaling and Gaussian filtering, sending the workpiece picture and the standard positioning picture into an image registration model homograph to obtain a homography transformation matrix of the workpiece picture and the standard positioning picture; acquiring a standard front and back background segmentation chart of the standard positioning picture; and performing homography according to the standard front and rear background segmentation map and the homography transformation matrix to obtain the registered front and rear background segmentation map.
According to an embodiment of the present invention, the detection module is specifically configured to: judging whether the workpiece has defects according to the screened preselection frame; if the workpiece has defects, sending the workpiece into an NG material box; and if the workpiece has no defects, feeding the workpiece into a GOOD material box.
The invention has the beneficial effects that:
according to the method, the image registration and the FPN model are utilized, the area generated by the pre-selection frame is limited, so that the model can be concentrated on the detection of the optical surface area, the detection speed of the model is improved, and the omission ratio is reduced; the image registration is used for screening the pre-selection frame, so that the defects of the non-optical surface can be screened out, and the overdetection rate of the model is reduced.
Drawings
FIG. 1 is a flow diagram of a method of defect detection in industrial inspection according to one embodiment of the present invention;
FIG. 2 is a flow diagram of training of a defect detection model according to one embodiment of the invention;
FIG. 3 is a training schematic of a defect detection model according to one embodiment of the present invention;
FIG. 4 is a flowchart of post-registration pre-and post-background segmentation map acquisition, according to one embodiment of the present invention;
FIG. 5 is a block diagram of a defect detection apparatus in industrial inspection according to one embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
FIG. 1 is a flow chart of a method of defect detection in industrial inspection according to one embodiment of the present invention. As shown in fig. 1, the method comprises the steps of:
and S1, acquiring the picture of the workpiece to be detected.
And S2, inputting the workpiece picture into the trained defect detection model to obtain a defect prediction result, wherein the defect detection model comprises a MASK-RCNN model and a FPN model, and the defect prediction result comprises a preselected frame.
And S3, carrying out image registration on the workpiece picture to acquire a front and back background segmentation map after registration.
And S4, screening a preselected frame of the defect prediction result according to the registered front and rear background segmentation maps.
And S5, acquiring a defect detection result according to the screened preselection frame.
Specifically, a workpiece picture to be detected is obtained by an optical camera on a production line, and is sent to a trained defect detection model for detection to obtain a defect prediction result, the defect detection model can utilize an FPN model to perform front and back background segmentation on the workpiece picture to obtain a front and back background segmentation map, MASK-RCNN can perform defect prediction on the workpiece picture after the front and back background segmentation, MASK-RCNN can concentrate on detection of an optical surface area (foreground) according to the front and back background segmentation map, no calculation is wasted on a non-optical surface area (background), the detection speed of the model is improved, defects (dirt on a loading platform and the like) of the non-optical surface can be prevented from mixing with real defect characteristics, the detection capability of the model is improved, and the omission ratio is reduced. After the defect prediction result is obtained, image registration is carried out on the workpiece picture, a front and rear registered background segmentation image is obtained, then a preselection frame of the defect prediction result is screened according to the front and rear registered background segmentation image, for example, the preselection frame of the background area of the front and rear registered background segmentation image with the central point in the defect prediction result is screened out, and the preselection frame of the foreground area of the front and rear registered background segmentation image with the central point in the defect prediction result is reserved, so that the defect of the non-optical surface can be screened out, the over-inspection rate of the model is reduced, and the accuracy of the model is improved.
Therefore, the region generated by the pre-selection frame is limited by utilizing the image registration and the FPN model, so that the model can be concentrated on the detection of the optical surface region, the detection speed of the model is improved, and the omission ratio is reduced; the image registration is used for screening the pre-selection frame, so that the defects of the non-optical surface can be screened out, and the overdetection rate of the model is reduced.
In the present invention, in a sample, the foreground refers to a region (optical surface) to be detected, and the background refers to a region (non-optical surface) not to be detected.
In an embodiment of the present invention, as shown in fig. 2, the defect detection model is trained in the following manner:
s201, inputting the training set of the workpiece picture into a main network RESNET-50 of MASK-RCNN for feature extraction to obtain a feature map.
S202, the feature map is sent into the FPN model to obtain a segmentation feature map.
The invention can adopt FPN simplified version, and the final output pixel types are only two types: foreground and background.
And S203, acquiring a first front and rear background segmentation map according to the segmentation feature map.
And S204, carrying out image registration on the training set to obtain a registered front and back background segmentation map of the training set.
And S205, acquiring the intersection of the first front and rear background segmentation map and the registered front and rear background segmentation map to generate a second background segmentation map.
S206, inputting the characteristic diagram into an RPN network in MASK-RCNN to generate a first pre-selection box.
And S207, screening out the preselected frame of which the central point falls in the background area of the second background segmentation map in the first preselected frame to obtain a second preselected frame.
And S208, further screening the second preselected frame by using the NMS algorithm to obtain a third preselected frame.
Specifically, the idea of NMS is to search for local maxima and suppress non-maximum elements, and the specific implementation steps are as follows: setting a confidence threshold of a preselected frame, wherein a common threshold is about 0.5; arranging the candidate frame list according to the confidence degree in a descending order; selecting a box A with the highest confidence coefficient to be added into an output list, and deleting the box A from the candidate box list; calculating the intersection ratio of the A and all frames in the candidate frame list, and deleting the preselected frames larger than the threshold; and repeating the process until the candidate box list is empty, and returning to the output list.
Suppose step S206 generates N0The first pre-selection frame comprising N0A 1, utilizing N0The first pre-selection frames are used as a screening basis, pre-selection frames with the central points falling in the background area (non-optical surface area) of the second background segmentation chart in S205 are screened out, false positive defects of the non-optical surface are screened out, the pre-selection frames with the central points falling in the foreground area of the second background segmentation chart are reserved, and N remains1The second pre-selection frame comprising N1N is1≤N0. Then, N will remain by the NMS algorithm1And carrying out secondary screening on the second pre-selection frames to obtain third pre-selection frames.
Compared with the mode of directly screening by NMS, the method for carrying out preselection frame secondary screening of the invention has the advantages that the model training speed and the subsequent prediction speed are improved by K times, wherein K is expressed as follows:
And S209, acquiring a target feature segmentation map according to the third pre-selection frame and the feature map.
S210, performing concat operation fusion on the segmentation feature map and the target feature segmentation map to obtain a fusion feature map.
And S211, acquiring a defect prediction result according to the fusion feature map.
S212, obtaining the training set label, and iterating the model weight according to the training set label and the defect prediction result to perform supervised learning.
Therefore, through the steps, the training set is used for model training of the FPN and the MASK-RCNN, and the trained models of the FPN and the MASK-RCNN are obtained through multiple rounds of iteration.
For a person skilled in the art to understand the present invention more clearly, the principle of steps S201-S212 can be seen in fig. 3, and the present invention is not described in detail.
According to an embodiment of the present invention, as shown in fig. 4, the registered segmentation maps of the front and back backgrounds are obtained by the following steps:
s301, after the workpiece picture and the standard positioning picture are subjected to scaling and Gaussian filtering, the workpiece picture and the standard positioning picture are sent to an image registration model homograph to obtain a homography transformation matrix of the workpiece picture and the standard positioning picture.
Specifically, the zoom factor may be 1/2 (i.e. 2 times smaller) of the original image size, where too small a factor may reduce the speed of the subsequent model, and too large a factor may reduce the accuracy of the subsequent registration. In addition, aiming at different projects, the scaling factor can be adjusted, and the optimal value can be found according to corresponding experimental verification.
And (4) carrying out one layer of Gaussian filtering processing on the zoomed workpiece picture, wherein the purpose is to smooth the image so as to remove Gaussian noise in the image, thereby improving the accuracy of the model. The formula of gaussian filtering is:
wherein, (x, y) is the point coordinate of the workpiece picture.
And taking the workpiece picture and the standard positioning picture after the scaling and Gaussian filtering processing as input and sending the input into an image registration model homograph, wherein the image registration model homograph can automatically generate a homography transformation matrix of the two pictures.
The standard positioning picture is a standard workpiece picture, namely a registration template of all workpiece pictures, and the image registration is to convert the workpiece picture (original picture, before registration) into a new workpiece picture (after registration) close to the standard picture through affine transformation.
S302, obtaining a standard front and back background segmentation chart of the standard positioning picture.
And S303, performing homography transformation according to the standard front and rear background segmentation maps and the homography transformation matrix to obtain the front and rear background segmentation maps after registration.
Specifically, the homography transformation matrix is applied to the standard front and rear background segmentation maps, so that the registered front and rear background segmentation maps can be obtained, wherein the homography transformation process is as follows:
wherein R is a homography transformation matrix, typically a 3 x 3 matrix;
the coordinate position of any point in the standard front and back background segmentation graph is obtained;
and the corresponding coordinate positions in the front and back background segmentation images after single registration.
According to an embodiment of the invention, acquiring the defect detection result according to the screened pre-selection frame specifically comprises: judging whether the workpiece has defects according to the screened preselection frame; if the workpiece has defects, sending the workpiece into an NG material box; and if the workpiece has no defects, feeding the workpiece into a GOOD material box.
That is, if the prediction result is defective, the corresponding workpiece is sent into an NG material box and discarded as waste; and if the prediction result has no defects, sending the corresponding workpiece into a GOOD material box, and detecting the workpiece as a GOOD product.
In summary, according to the defect detection method in industrial inspection of the embodiment of the present invention, a workpiece picture to be detected is obtained, the workpiece picture is input into a trained defect detection model to obtain a defect prediction result, wherein the defect detection model includes a MASK-RCNN model and a FPN model, the defect prediction result includes a preselected frame, the workpiece picture is subjected to image registration to obtain a pre-registered front and rear background segmentation map, the preselected frame of the defect prediction result is screened according to the pre-registered front and rear background segmentation map, and the defect detection result is obtained according to the screened preselected frame. Therefore, the method limits the area generated by the pre-selection frame by utilizing image registration and the FPN model, so that the model can be concentrated on the detection of the optical surface area, the detection speed of the model is improved, and the omission ratio is reduced; the image registration is used for screening the pre-selection frame, so that the defects of the non-optical surface can be screened out, and the overdetection rate of the model is reduced.
Corresponding to the defect detection method in industrial detection, the invention also provides a defect detection device in industrial detection. Since the device embodiment of the present invention corresponds to the method embodiment described above, details that are not disclosed in the device embodiment may refer to the method embodiment described above, and are not described again in the present invention.
FIG. 5 is a block diagram of a defect detection apparatus in industrial inspection according to one embodiment of the present invention. As shown in fig. 5, the apparatus includes: the device comprises an acquisition module 1, a prediction module 2, a registration module 3, a screening module 4 and a detection module 5.
The acquisition module 1 is used for acquiring a workpiece picture to be detected; the prediction module 2 is used for inputting a workpiece picture into the trained defect detection model to obtain a defect prediction result, wherein the defect detection model comprises a MASK-RCNN model and a FPN model, and the defect prediction result comprises a preselected frame; the registration module 3 is used for carrying out image registration on the workpiece picture so as to obtain a front and rear background segmentation image after registration; the screening module 4 is used for screening a preselected frame of the defect prediction result according to the registered front and rear background segmentation maps; and the detection module 5 is used for acquiring a defect detection result according to the screened preselection frame.
According to an embodiment of the invention, the prediction module 2 is further configured to: inputting the training set of the workpiece picture into a main network RESNET-50 of MASK-RCNN for feature extraction to obtain a feature map; sending the feature map into an FPN model to obtain a segmentation feature map; acquiring a first front and rear background segmentation map according to the segmentation feature map; carrying out image registration on the training set to obtain a registered front and back background segmentation graph of the training set; acquiring an intersection of the first front and rear background segmentation maps and the registered front and rear background segmentation maps to generate a second background segmentation map; inputting the feature map into an RPN network in MASK-RCNN to generate a first preselected box; screening out a preselected frame of which the center point falls in the background area of the second background segmentation chart in the first preselected frame to obtain a second preselected frame; further screening the second pre-selection frame by using an NMS algorithm to obtain a third pre-selection frame; acquiring a target feature segmentation chart according to the third pre-selection frame and the feature chart; performing concat operation fusion on the segmentation feature map and the target feature segmentation map to obtain a fusion feature map; acquiring a defect prediction result according to the fusion characteristic diagram; and acquiring a training set label, and iterating the model weight according to the training set label and the defect prediction result to perform supervised learning.
According to one embodiment of the invention, the registration module 3 is specifically configured to: after the workpiece picture and the standard positioning picture are subjected to scaling and Gaussian filtering, the workpiece picture and the standard positioning picture are sent to an image registration model homograph to obtain a homography transformation matrix of the workpiece picture and the standard positioning picture; acquiring a standard front and back background segmentation chart of a standard positioning picture; and performing homography according to the standard front and back background segmentation maps and the homography transformation matrix to obtain the front and back background segmentation maps after registration.
According to an embodiment of the present invention, the detection module is specifically configured to: judging whether the workpiece has defects according to the screened preselection frame; if the workpiece has defects, sending the workpiece into an NG material box; and if the workpiece has no defects, feeding the workpiece into a GOOD material box.
In summary, according to the defect detection apparatus in industrial inspection of the embodiment of the present invention, the acquisition module acquires a workpiece picture to be detected, and the prediction module inputs the workpiece picture into the trained defect detection model to acquire a defect prediction result, where the defect detection model includes a MASK-RCNN model and a FPN model, and the defect prediction result includes a pre-selection frame. The registration module performs image registration on the workpiece picture to obtain a registered front and rear background segmentation image, the screening module screens a preselected frame of a defect prediction result according to the registered front and rear background segmentation image, and the detection module obtains a defect detection result according to the screened preselected frame. Therefore, the device limits the area generated by the pre-selection frame by utilizing image registration and the FPN model, so that the model can be concentrated on the detection of the optical surface area, the detection speed of the model is improved, and the omission ratio is reduced; the image registration is used for screening the pre-selection frame, so that the defects of the non-optical surface can be screened out, and the overdetection rate of the model is reduced.
In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The meaning of "plurality" is two or more unless specifically limited otherwise.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (6)
1. A defect detection method in industrial detection is characterized by comprising the following steps:
acquiring a picture of a workpiece to be detected;
inputting the workpiece picture into a trained defect detection model to obtain a defect prediction result, wherein the defect detection model comprises a MASK-RCNN model and a FPN model, and the defect prediction result comprises a preselected frame;
carrying out image registration on the workpiece picture to obtain a front background segmentation picture and a rear background segmentation picture after registration;
screening a preselected frame of the defect prediction result according to the registered front and rear background segmentation maps;
acquiring a defect detection result according to the screened preselection frame;
wherein the defect detection model is trained in the following manner:
inputting the training set of the workpiece picture into a main network RESNET-50 of the MASK-RCNN for feature extraction to obtain a feature map;
sending the feature map into the FPN model to obtain a segmentation feature map;
acquiring a first front and rear background segmentation map according to the segmentation feature map;
carrying out image registration on the training set to obtain a registered front and back background segmentation map of the training set;
acquiring an intersection of the first front and rear background segmentation map and the registered front and rear background segmentation map to generate a second background segmentation map;
inputting the feature map into an RPN network in the MASK-RCNN to generate a first preselected box;
screening out a preselected frame of which the center point falls in a background area of a second background segmentation chart in the first preselected frame to obtain a second preselected frame;
further screening the second preselected frame by using an NMS algorithm to obtain a third preselected frame;
acquiring a target feature segmentation map according to the third pre-selection frame and the feature map;
performing concat operation fusion on the segmentation feature map and the target feature segmentation map to obtain a fusion feature map;
acquiring the defect prediction result according to the fusion feature map;
and acquiring a training set label, and iterating the model weight according to the training set label and the defect prediction result to perform supervised learning.
2. The method of claim 1, wherein the pre-background segmentation map and the post-registration background segmentation map are obtained by the following steps:
after the workpiece picture and the standard positioning picture are subjected to scaling and Gaussian filtering, sending the workpiece picture and the standard positioning picture into an image registration model homograph to obtain a homography transformation matrix of the workpiece picture and the standard positioning picture;
acquiring a standard front and back background segmentation chart of the standard positioning picture;
and performing homography according to the standard front and rear background segmentation map and the homography transformation matrix to obtain the registered front and rear background segmentation map.
3. The method according to claim 1, wherein the obtaining of the defect detection result according to the screened pre-selected frame specifically comprises:
judging whether the workpiece has defects according to the screened preselection frame;
if the workpiece has defects, sending the workpiece into an NG material box;
and if the workpiece has no defects, feeding the workpiece into a GOOD material box.
4. A defect detecting apparatus in industrial inspection, comprising:
the acquisition module is used for acquiring a workpiece picture to be detected;
the prediction module is used for inputting the workpiece picture into a trained defect detection model to obtain a defect prediction result, wherein the defect detection model comprises a MASK-RCNN model and a FPN model, and the defect prediction result comprises a preselected frame;
the registration module is used for carrying out image registration on the workpiece picture so as to obtain a front and back background segmentation picture after registration;
the screening module is used for screening a preselected frame of the defect prediction result according to the registered front and rear background segmentation maps;
the detection module is used for acquiring a defect detection result according to the screened preselection frame;
the prediction module is further to:
inputting the training set of the workpiece picture into a main network RESNET-50 of the MASK-RCNN for feature extraction to obtain a feature map;
sending the feature map into the FPN model to obtain a segmentation feature map;
acquiring a first front and rear background segmentation map according to the segmentation feature map;
carrying out image registration on the training set to obtain a registered front and back background segmentation map of the training set;
acquiring an intersection of the first front and rear background segmentation map and the registered front and rear background segmentation map to generate a second background segmentation map;
inputting the feature map into an RPN network in the MASK-RCNN to generate a first preselected box;
screening out a preselected frame of which the center point falls in a background area of a second background segmentation chart in the first preselected frame to obtain a second preselected frame;
further screening the second preselected frame by using an NMS algorithm to obtain a third preselected frame;
acquiring a target feature segmentation map according to the third pre-selection frame and the feature map;
performing concat operation fusion on the segmentation feature map and the target feature segmentation map to obtain a fusion feature map;
acquiring the defect prediction result according to the fusion feature map;
and acquiring a training set label, and iterating the model weight according to the training set label and the defect prediction result to perform supervised learning.
5. The apparatus for defect detection in industrial inspection according to claim 4, wherein the registration module is specifically configured to:
after the workpiece picture and the standard positioning picture are subjected to scaling and Gaussian filtering, sending the workpiece picture and the standard positioning picture into an image registration model homograph to obtain a homography transformation matrix of the workpiece picture and the standard positioning picture;
acquiring a standard front and back background segmentation chart of the standard positioning picture;
and performing homography according to the standard front and rear background segmentation map and the homography transformation matrix to obtain the registered front and rear background segmentation map.
6. The apparatus of claim 4, wherein the detection module is specifically configured to:
judging whether the workpiece has defects according to the screened preselection frame;
if the workpiece has defects, sending the workpiece into an NG material box;
and if the workpiece has no defects, feeding the workpiece into a GOOD material box.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111035503.9A CN113469299B (en) | 2021-09-06 | 2021-09-06 | Defect detection method and defect detection device in industrial detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111035503.9A CN113469299B (en) | 2021-09-06 | 2021-09-06 | Defect detection method and defect detection device in industrial detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113469299A CN113469299A (en) | 2021-10-01 |
| CN113469299B true CN113469299B (en) | 2021-12-07 |
Family
ID=77867491
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111035503.9A Active CN113469299B (en) | 2021-09-06 | 2021-09-06 | Defect detection method and defect detection device in industrial detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113469299B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110473197A (en) * | 2019-08-13 | 2019-11-19 | 树根互联技术有限公司 | Material surface defect detection method, device, equipment and storage medium |
| CN110517224A (en) * | 2019-07-12 | 2019-11-29 | 上海大学 | A kind of photovoltaic panel defect inspection method based on deep neural network |
| CN111369550A (en) * | 2020-03-11 | 2020-07-03 | 创新奇智(成都)科技有限公司 | Image registration and defect detection method, model, training method, device and equipment |
| CN113066088A (en) * | 2021-06-03 | 2021-07-02 | 常州微亿智造科技有限公司 | Detection method, detection device and storage medium in industrial detection |
-
2021
- 2021-09-06 CN CN202111035503.9A patent/CN113469299B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110517224A (en) * | 2019-07-12 | 2019-11-29 | 上海大学 | A kind of photovoltaic panel defect inspection method based on deep neural network |
| CN110473197A (en) * | 2019-08-13 | 2019-11-19 | 树根互联技术有限公司 | Material surface defect detection method, device, equipment and storage medium |
| CN111369550A (en) * | 2020-03-11 | 2020-07-03 | 创新奇智(成都)科技有限公司 | Image registration and defect detection method, model, training method, device and equipment |
| CN113066088A (en) * | 2021-06-03 | 2021-07-02 | 常州微亿智造科技有限公司 | Detection method, detection device and storage medium in industrial detection |
Non-Patent Citations (1)
| Title |
|---|
| "3D Semantic VSLAM of Indoor Environment Based on Mask Scoring RCNN";Chongben Tao等;《Discrete Dynamics in Nature and Society》;20201020;第1-14页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113469299A (en) | 2021-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109255352B (en) | Target detection method, device and system | |
| CN108268869B (en) | Target detection method, device and system | |
| CN110414507B (en) | License plate recognition method and device, computer equipment and storage medium | |
| CN111507958B (en) | Target detection method, training method of detection model and electronic equipment | |
| CN109035292B (en) | Moving target detection method and device based on deep learning | |
| CN111583097A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN110598512B (en) | Parking space detection method and device | |
| CN110378837B (en) | Target detection method and device based on fish-eye camera and storage medium | |
| CN110796649B (en) | Target detection method and device, electronic equipment and storage medium | |
| CN111310746B (en) | Text line detection method, model training method, device, server and medium | |
| CN113160161B (en) | Method and device for detecting defects at edge of target | |
| CN113066088A (en) | Detection method, detection device and storage medium in industrial detection | |
| JP4427052B2 (en) | Image processing apparatus and area tracking program | |
| CN111967345A (en) | Method for judging shielding state of camera in real time | |
| CN113888438A (en) | Image processing method, device and storage medium | |
| CN113469299B (en) | Defect detection method and defect detection device in industrial detection | |
| CN116189038A (en) | Picture abnormality judging method, device, equipment and storage medium | |
| CN110796117B (en) | Blood cell automatic analysis method, system, blood cell analyzer and storage medium | |
| CN109727193B (en) | Image blurring method and device and electronic equipment | |
| CN114550062A (en) | Method and device for determining moving object in image, electronic equipment and storage medium | |
| CN112418244A (en) | Target detection method, device and electronic system | |
| Pooyoi et al. | Snow scene segmentation using cnn-based approach with transfer learning | |
| CN113382134B (en) | Focusing debugging method of linear array industrial camera | |
| CN115118948B (en) | Repairing method and device for irregular shielding in panoramic video | |
| CN114120070B (en) | Image detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |