CN111681220B - Method, device, system and storage medium for constructing defect detection model - Google Patents
Method, device, system and storage medium for constructing defect detection model Download PDFInfo
- Publication number
- CN111681220B CN111681220B CN202010502442.1A CN202010502442A CN111681220B CN 111681220 B CN111681220 B CN 111681220B CN 202010502442 A CN202010502442 A CN 202010502442A CN 111681220 B CN111681220 B CN 111681220B
- Authority
- CN
- China
- Prior art keywords
- image
- defect
- sample
- detection model
- product
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 313
- 238000001514 detection method Methods 0.000 title claims abstract description 207
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000010191 image analysis Methods 0.000 claims abstract description 16
- 230000002950 deficient Effects 0.000 claims description 10
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000005498 polishing Methods 0.000 claims description 5
- 239000000047 product Substances 0.000 description 179
- 239000011521 glass Substances 0.000 description 15
- 238000013135 deep learning Methods 0.000 description 5
- 238000003672 processing method Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- GNFTZDOKVXKIBK-UHFFFAOYSA-N 3-(2-methoxyethoxy)benzohydrazide Chemical compound COCCOC1=CC=CC(C(=O)NN)=C1 GNFTZDOKVXKIBK-UHFFFAOYSA-N 0.000 description 1
- YTAHJIFKAKIKAV-XNMGPUDCSA-N [(1R)-3-morpholin-4-yl-1-phenylpropyl] N-[(3S)-2-oxo-5-phenyl-1,3-dihydro-1,4-benzodiazepin-3-yl]carbamate Chemical compound O=C1[C@H](N=C(C2=C(N1)C=CC=C2)C1=CC=CC=C1)NC(O[C@H](CCN1CCOCC1)C1=CC=CC=C1)=O YTAHJIFKAKIKAV-XNMGPUDCSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 229910052736 halogen Inorganic materials 0.000 description 1
- 150000002367 halogens Chemical class 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Evolutionary Biology (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device, a system and a computer readable storage medium for constructing a defect detection model, wherein the method comprises the following steps: the method comprises the steps of performing multi-angle acquisition on a sample product through image acquisition equipment to obtain a sample product image with a preset number of parts, and performing image analysis on the sample product image to obtain a sample defect image; and obtaining a defect image training set according to the relevance of the image labels in the sample defect image, and performing image training on the defect image training set to obtain a defect detection model. According to the invention, sample products are acquired at multiple angles, sample product images are analyzed to obtain sample defect images, and the associated sample defect images are subjected to image training according to the relevance of image labels in the sample defect images to obtain a defect detection model, so that the detection capability and the detection stability of product defect detection are improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a system, and a storage medium for constructing a defect detection model.
Background
The existing product defect detection method comprises a traditional image analysis processing method and a single-image AI detection method. The traditional image analysis processing method is that after the product defect is mapped, the traditional image analysis processing technology algorithm is utilized to carry out single-map threshold segmentation, filtering, BLOB (connected region) analysis, template matching, edge extraction and other quantitative calculation analysis on the product defect image, and then analysis and judgment are carried out by summarizing the result. The traditional image analysis processing method takes the quantitative value of the defective image of the product as the basis of analysis processing, so that the requirements on the consistency of conditions such as the brightness of a light source, the interference degree of ambient light, the drawing position and the gesture of the defective image of the product are higher, and because the position of the defective image of the product is not fixed, the optical property is very close to the foreground or the background, the depth or the thickness difference and different defects are simultaneously appeared or mixed together, even if the defects are polished or drawn through multiple angles, the traditional image analysis processing method of the defects can not detect the defects, and further, the traditional image analysis processing method needs a large number of codes and has poor product compatibility. The single-image AI detection method is often insufficient in characteristics of the single-image optical representation for stable judgment of defects due to the complexity of the product surface, and more missed detection or over detection can occur. From this, the existing product defect detection method has poor detection capability and low detection stability.
Disclosure of Invention
The invention mainly aims to provide a method, a device, a system and a computer readable storage medium for constructing a defect detection model, and aims to solve the technical problems of poor detection capability and low detection stability of the existing product defect detection method.
In order to achieve the above object, the present invention provides a method for constructing a defect detection model, the method for constructing a defect detection model comprising the steps of:
the method comprises the steps of performing multi-angle acquisition on a sample product through image acquisition equipment to obtain a sample product image with a preset number of parts, and performing image analysis on the sample product image to obtain a sample defect image;
and obtaining a defect image training set according to the relevance of the image labels in the sample defect image, and performing image training on the defect image training set to obtain a defect detection model.
Optionally, the obtaining the defect image training set according to the relevance of the image labels in the sample defect image includes:
acquiring image tags in the sample defect image, and determining the same number of the image tags in the sample defect image according to the relevance of the image tags;
obtaining the similarity degree of the image labels in the sample defect image according to the same number;
and classifying the sample defect images according to the similarity degree to obtain defect image training sets corresponding to various sample defect images.
Optionally, the step of acquiring the sample product at multiple angles by the image acquisition device to obtain a sample product image with a preset number of parts, and performing image analysis on the sample product image to obtain a sample defect image includes:
performing multi-angle polishing on the sample product through image acquisition equipment, and performing multi-angle acquisition on the polished sample product to obtain the sample product images with the preset number of parts;
image decision is carried out on the sample product image through a decision network model, so that a decision value of the sample product image is obtained;
and determining a sample defect image corresponding to the sample product image according to the decision value.
Optionally, the step of determining a sample defect image corresponding to the sample product image according to the decision value includes:
judging whether the decision value of the sample product image is larger than a preset decision value or not;
if the decision value of the sample product image is larger than the preset decision value, positioning the defective pixels of the sample product image to obtain a positioning image, and dividing the sample product image according to the positioning image to obtain a divided image comprising the positioning image;
and attaching an image tag corresponding to the sample product image to the segmented image to obtain the sample defect image.
Optionally, after the step of obtaining the defect detection model by obtaining a defect image training set according to the relevance of the image labels in the sample defect image and performing image training on the defect image training set, the method further includes:
acquiring a product image to be detected through the image acquisition equipment, and identifying an image tag in the product image to be detected;
matching an image detection model corresponding to the image label of the product image to be detected in the defect detection model, and carrying out image detection on the product image to be detected through the image detection model to obtain a detection result.
In addition, in order to achieve the above object, the present invention further provides a device for constructing a defect detection model, which is characterized in that the device for constructing a defect detection model includes:
the acquisition module is used for acquiring sample products at multiple angles through the image acquisition equipment to obtain sample product images with preset parts;
the analysis module is used for carrying out image analysis on the sample product image to obtain a sample defect image;
the determining module is used for obtaining a defect image training set according to the relevance of the image labels in the sample defect image;
and the training module is used for carrying out image training on the defect image training set to obtain a defect detection model.
In addition, in order to achieve the above object, the present invention also provides a system for constructing a defect detection model, where the system for constructing a defect detection model includes a memory, a processor, a GPU display card, and a program for constructing a defect detection model stored in the memory and running on the processor and the GPU display card, where the program for constructing a defect detection model implements the steps of the method for constructing a defect detection model described above when the program for constructing a defect detection model is completed by the processor and the GPU display card.
In addition, in order to achieve the above object, the present invention further provides a computer readable storage medium having stored thereon a program for constructing a defect detection model, which when completed by a processor and a GPU display card, implements the steps of the method for constructing a defect detection model as described above.
The method and the device realize that sample products are collected at multiple angles to obtain sample product images with preset parts, the sample product images are analyzed to obtain sample defect images, and the associated sample defect images are subjected to image training according to the relevance of image labels to obtain a defect detection model. Therefore, in the process of sample product image acquisition, the sample product is subjected to multi-angle acquisition, so that multi-angle images of all parts of the sample product are obtained, then the sample product image is analyzed to obtain sample defect images, then the multi-angle images of all defect parts are associated according to the association of image labels, so that complete defect image training sets of all defects are obtained, then the defect image training sets are subjected to image training, and therefore a defect detection model capable of detecting defects of different dimensions is obtained, and the detection capability and the detection stability of the defect detection model on product defects are improved.
Drawings
FIG. 1 is a flow chart of a first embodiment of a method of constructing a defect detection model according to the present invention;
FIG. 2 is a schematic diagram showing a construction apparatus of a defect detection model according to the present invention;
FIG. 3 is a schematic diagram of a hardware operating environment according to an embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of a first embodiment of a method for constructing a defect detection model according to the present invention.
The present invention provides an embodiment of a method for constructing a defect detection model, and it should be noted that although a logic sequence is shown in the flowchart, the steps shown or described may be performed in a different order than the sequence shown or described herein under certain data.
The method for constructing the defect detection model comprises the following steps:
step S10, multi-angle acquisition is carried out on a sample product through image acquisition equipment to obtain a sample product image with a preset number of parts, and image analysis is carried out on the sample product image to obtain a sample defect image.
The defect detection system collects images of all angles of a sample product by calling image collection equipment to obtain a sample product image with preset number of times, then detects whether the sample product image has image features including but not limited to projection, dark ring or speckled and the like, judges whether the sample product image is a sample defect image according to the image features, if the image features of projection, dark ring or speckled are detected to be absent on the sample product image, the defect detection system determines that the sample product image is a defect-free image, and if the image features of projection, dark ring or speckled are detected to be present on the sample product image, the defect detection system determines that the sample product image is a sample defect image.
The image capturing device may be a camera, a video camera, a camera hole, etc., and the embodiment does not limit the form of the image capturing device. The image acquisition device can be carried by the defect detection system, such as a camera in the defect detection system, or externally connected with the defect detection system, such as a mobile phone, a camera and the like, and the image acquisition device needs to be connected with the defect detection system through a network interface when the image acquisition device is externally connected with the defect detection system. The preset number of parts can be set according to the requirement, and the embodiment is not limited.
Further, the step S10 includes:
step a, polishing a sample product at multiple angles through image acquisition equipment, and acquiring polished sample products at multiple angles to obtain sample product images with preset parts;
b, performing image decision on the sample product image through a decision network model to obtain a decision value of the sample product image;
and c, determining a sample defect image corresponding to the sample product image according to the decision value.
Specifically, the defect detection system supplements different light sources and light source brightness for the sample product from each angle through a light source device in the image acquisition device, acquires sample product images from each angle through the image acquisition device, and can acquire one or more sample product images from each angle to obtain sample product images after the preset number of the supplement light sources, and transmits the sample product images to a judgment network model of the defect detection system. And then the defect detection system converts the sample product image into a pixel expansion image through the judgment network model, detects whether a white pixel image or a fuzzy pixel image exists in the pixel expansion image, and determines that no defect exists in the sample product image if the white pixel image or the fuzzy pixel image does not exist. If the white pixel image or the fuzzy pixel image exists, the defect detection system calculates the area size of the white pixel image, calculates the proportion of the area size to the whole pixel expansion image area, and outputs a proportion value, wherein the proportion value is the decision value of the sample product image. The defect detection system compares the decision value with a preset decision value in the defect detection system to obtain a comparison result, and whether the sample product image is a sample defect image or not is determined according to the comparison result.
Among them, a light source apparatus is an apparatus that provides a light source and whose brightness is adjustable, and the light source apparatus includes, but is not limited to, LED (light emitting diode) lamps, halogen lamps, incandescent lamps, strobe light sources, strobe lamps, adjustable light-compensating lamps, strobe light-compensating lamps, and the like. The number of sample product images acquired at each angle is not limited and is set according to the type and nature of the sample product. The judgment network model is an image model which is built in advance by the defect detection system, and the judgment network model is an image model which is built based on the loss function model. The pixel expansion image is an image of a sample product after the image is amplified by a preset multiple, wherein the preset multiple can be set according to requirements and can be 10 times, 100 times, 1000 times and the like. The preset decision value is preset in the defect detection system, and can be set according to the requirement, and the embodiment is not limited. The comparison result comprises that the decision value is smaller than or equal to a preset decision value, and the decision value is larger than the preset decision value.
Further, it should be noted that, when the defect detection system collects the sample product images through the image collecting device, one or more sample product images may be directly collected from multiple angles without supplementing the light source.
Further, the step c includes:
step d, judging whether the decision value of the sample product image is larger than a preset decision value;
step e, if the decision value of the sample product image is larger than the preset decision value, positioning the defective pixels of the sample product image to obtain a positioning image, and dividing the sample product image according to the positioning image to obtain a divided image comprising the positioning image;
and f, attaching an image tag corresponding to the sample product image to the segmented image to obtain the sample defect image.
Specifically, after the decision value of the sample product image is obtained, the defect detection system judges whether the decision value of the sample product image is larger than a preset decision value, if the decision value is smaller than or equal to the preset decision value, which indicates that defects in the sample product image can be ignored, and the defect detection system marks corresponding marks on the sample product image. If the decision value is larger than the preset decision value, the defect detecting system indicates that the sample product image has an inadvisable defect, according to the pixel characteristics of the sample product image, wherein the pixel characteristics can be pixel resolution, defective pixels in the sample product image are determined according to the pixel resolution, an image area corresponding to the defective pixels with low pixel resolution is framed, a positioning image is obtained, the sample product image is divided into two parts of sample product local images according to a positioning mark, one part of sample product local images carrying the positioning image is obtained, the other part of sample product local images not carrying the positioning image is obtained, the sample product local images not carrying the positioning image are deleted, the sample product local images carrying the positioning image are reserved, the sample product local images carrying the positioning image are divided images of the sample product image, and finally, an image tag corresponding to the sample product image is attached to the divided images and is marked in the divided images, so that the sample defect image is obtained.
The pixel resolution is a parameter for judging whether the image pixel is normal or not. The registration mark is a specific mark set for the image area by the defect detection system when determining the registration image. The image tag is identification information of an image and is obtained when the defect detection system calls the image acquisition equipment to acquire an image of a sample product. The image tags include a base tag including, but not limited to, the name of the sample product and the name of the sample product acquisition area, and an additional tag including, but not limited to, acquisition angle, light source type, decision value, and light source brightness value.
It should be noted that, the pixel resolution of the area image is obtained by performing pixel feature matching between the pixel resolution of the area image and the pixel resolution of the defect area image and the pixel resolution of the complete area image, and the higher the pixel resolution matching degree of the defect area image, that is, the higher the similarity, the more serious the pixel distortion of the area image, the more the defect pixels exist, and the higher the pixel resolution matching degree of the defect area image, that is, the higher the similarity, the more complete the pixels of the area image. The basic label can be manually input or set in the defect detection system when the image acquisition device acquires the sample product image, and the additional label is automatically generated by the image acquisition device according to the set data when the image acquisition device acquires the sample product image.
In this embodiment, for example, the name of the sample product is glass, the name of the sample product collecting area is cup, the preset decision value is 1/10000, the decision value of the sample product image obtained by the defect detection system is 2/10000, the defect detection system determines that an inadvisable defect exists in the sample product image, then the defect is positioned in the sample product image, then the defect image is segmented from the sample product image, and finally the image label corresponding to the sample product image is added to the segmentation graph, so as to obtain the sample defect image carrying the image label of 2/10000, cup and glass.
Step S20, obtaining a defect image training set according to the relevance of the image labels in the sample defect image, and performing image training on the defect image training set to obtain a defect detection model.
The defect detection system detects image labels in the sample defect images, judges the number of the same labels in the image labels, determines the relevance between the sample defect images according to the number of the same labels in the image labels, and classifies the sample defect images according to the relevance degree to obtain a sample defect image training set. And after the sample defect image training set is obtained, the defect detection system carries out image training on the sample defect image training set based on deep learning to obtain a defect detection model. Here, deep Learning refers to artificial intelligence (Artificial Intelligence) neural networks.
The more the number of the same labels in the image labels, the greater the correlation between the sample defect images, and the closer the relationship between the sample defect images. The degree of association is the number of identical labels in the image label.
Further, the step S20 includes:
step g, obtaining image labels in the sample defect image, and determining the same number of the image labels in the sample defect image according to the relevance of the image labels;
step h, obtaining the similarity degree of the image labels in the sample defect image according to the same number;
and i, classifying the sample defect images according to the similarity degree to obtain defect image training sets corresponding to various sample defect images.
Specifically, the defect detection system acquires image tags attached to sample defect images, then compares the image tags in the sample defect images one to obtain the same number of image tags, calculates the proportion of the same number of image tags to the total number of image tags in the corresponding sample defect images, and obtains a tag occupation ratio, wherein the occupation ratio is the similarity degree of the image tags. The defect detection system acquires a preset priority label, compares the similarity degree of the image label with the preset similarity degree in the defect detection system to obtain a comparison result, classifies corresponding sample defect images into different types of images if the similarity degree of the image label is smaller than the preset similarity degree, and classifies the corresponding sample defect images into the similar types of images if the similarity degree of the image label is greater than or equal to the preset similarity degree to obtain a defect image training set corresponding to each similar type of defect image.
The preset similarity degree is set according to the requirement, and the embodiment is not limited. The preset priority label is a classification label with priority, and is set in advance by the defect detection system according to the requirement, which is not limited in this embodiment. For example, the preset priority label may be set as a sample product name label, may be a sample product defect name, or may be a light source type.
In this embodiment, for example, a preset priority label is a sample product name label, a sample defect image 1 is a sample product name label, a sample product defect name label is a glass, a sample product defect name is a glass, a light source type is white light, a sample defect image 2 is a sample product name label is a plastic cup, a sample product defect name is a glass defect, a light source type is white light, a sample defect image 3 is a sample product name label is a plastic cup, a sample product defect name is a glass defect, a light source type is a gray light, a sample defect image 4 is a sample product name label is a glass, a sample product defect name is a glass, a light source type is a gray light, a defect detection system classifies the sample defect image 1 and the sample defect image 4 as similar images, a defect image training set of the sample product name label is a glass, the sample product defect name is a glass is obtained, the sample defect image 3 is a similar image, and a defect image training set of the sample product name label is a plastic cup is obtained.
According to the embodiment, the sample product is collected at multiple angles, the sample product images with the preset number of times are obtained, the sample product images are analyzed to obtain sample defect images, and the associated sample defect images are subjected to image training according to the relevance of the image labels to obtain the defect detection model. Therefore, in the process of sample product image acquisition, the sample product is subjected to multi-angle acquisition, so that multi-angle images of all parts of the sample product are obtained, then the sample product image is analyzed to obtain sample defect images, then the multi-angle images of all defect parts are associated according to the association of image labels, so that complete defect image training sets of all defects are obtained, then the defect image training sets are subjected to image training, and therefore defect detection models capable of detecting defects of different dimensions are obtained, and the detection capability and the detection stability of the defect detection models on the defects of the product are improved.
Further, a second embodiment of the method for constructing a defect detection model of the present invention is presented.
The second embodiment of the method for constructing a defect detection model is different from the first embodiment of the method for constructing a defect detection model in that the method for constructing a defect detection model further includes:
step j, acquiring a product image to be detected through the image acquisition equipment, and identifying an image tag in the product image to be detected;
and k, matching an image detection model corresponding to the image label of the product image to be detected in the defect detection model, and carrying out image detection on the product image to be detected through the image detection model to obtain a detection result.
Specifically, the defect detection system calls an image acquisition device to acquire a product to be detected in the same acquisition mode as that of the sample product image in the first embodiment, and obtains the image of the product to be detected. After the product image to be detected is obtained, the defect detection system identifies an image label in the product image to be detected, pairs an image detection model corresponding to the image label in a defect detection model obtained by the defect detection system based on deep learning training according to the image label, and finally compares and detects the product image to be detected with a sample defect image in the image detection model to obtain a detection result.
The defect detection model is composed of image detection models corresponding to various image labels.
In this embodiment, for example, the defect detection system acquires an image of a product to be detected, recognizes that an image tag corresponding to the image tag is a glass, then pairs a glass image detection model in the defect detection model, and detects the glass image detection model to obtain a detection result that a glass handle part in the glass is defective.
According to the method, the image acquisition equipment is used for acquiring the image of the product to be detected, the image of the product to be detected is input into a defect detection model obtained based on deep learning training, the image label in the image of the product to be detected is identified, and the image detection is carried out on the image of the product to be detected by utilizing a deep learning algorithm according to the image label in the image of the product to be detected, so that a detection result is obtained. Therefore, after the image of the product to be detected is acquired, the defect detection system acquires the image label in the image of the product to be detected, and then the image label of the image of the product to be detected is matched with a corresponding detection model in the defect detection model for detection, so that a detection result is obtained, and the accuracy of product detection is improved.
Further, a third embodiment of the method for constructing a defect detection model of the present invention is proposed.
The third embodiment of the method for constructing a defect detection model is different from the first embodiment of the method for constructing a defect detection model and the second embodiment of the method for constructing a defect detection model in that the method for constructing a defect detection model further includes:
and step l, storing the defect detection model in a defect detection system, inputting the product image to be detected into the defect detection model for detection after other product images to be detected are acquired, and outputting a detection result.
Specifically, after the defect detection system builds the defect detection model, the defect detection model is stored in the defect detection system, and after the defect detection system acquires the product image to be detected later, the product image to be detected is directly input into the defect detection model for detection, and then the detection result is directly output.
In this embodiment, the defect detection model is stored in the defect detection system, and after other product images to be detected are acquired, the product images to be detected are input into the defect detection model for detection, and the detection result is output. Therefore, after the image of the product to be detected is acquired, the defect detection model is not required to be built, the image of the product to be detected is directly input into the defect detection model for detection, and then the detection result is directly output, so that the efficiency of detecting the defects of the product is improved.
In addition, the present invention also provides a device for constructing a defect detection model, referring to fig. 2, the device for constructing a defect detection model includes:
the acquisition module 10 is used for acquiring sample products at multiple angles through the image acquisition equipment to obtain a sample product image with a preset number of parts;
an analysis module 20, configured to perform image analysis on the sample product image to obtain a sample defect image;
a determining module 30, configured to obtain a defect image training set according to the relevance of the image labels in the sample defect image;
and the training module 40 is configured to perform image training on the defect image training set to obtain a defect detection model.
Further, the determining module 30 includes:
an acquisition unit, configured to acquire an image tag in the sample defect image;
the first determining unit is used for determining the same number of image labels in the sample defect image according to the relevance of the image labels;
the second determining unit is used for obtaining the similarity degree of the image labels in the sample defect images according to the same number;
and the third determining unit is used for classifying the sample defect images according to the similarity degree to obtain defect image training sets corresponding to various sample defect images.
Further, the collecting module 10 is further configured to perform multi-angle polishing on the sample product through the image collecting device, and perform multi-angle collection on the polished sample product, so as to obtain the sample product image with the preset number of copies.
Further, the analysis module 20 includes:
and the decision unit is used for carrying out image decision on the sample product image through a decision network model to obtain a decision value of the sample product image.
Further, the determining module 30 is further configured to determine a sample defect image corresponding to the sample product image according to the decision value.
Further, the determining module 30 includes:
the judging unit is used for judging whether the decision value of the sample product image is larger than a preset decision value or not;
the positioning unit is used for positioning the defective pixels of the sample product image to obtain a positioning image if the decision value of the sample product image is larger than the preset decision value;
the segmentation unit is used for segmenting the sample product image according to the positioning image to obtain a segmented image comprising the positioning image;
and the attaching unit is used for attaching the image label corresponding to the sample product image to the segmented image to obtain the sample defect image.
Further, the acquisition module 10 is further configured to acquire an image of the product to be detected through the image acquisition device.
Further, the device for constructing the defect detection model further comprises:
the identification module is used for identifying the image tag in the image of the product to be detected;
the pairing module is used for pairing the image detection model corresponding to the image label of the product image to be detected in the defect detection model;
and the detection module is used for carrying out image detection on the product image to be detected through the image detection model to obtain a detection result.
The specific implementation manner of the device for constructing the defect detection model is basically the same as that of each embodiment of the method for constructing the defect detection model, and is not repeated here.
In addition, the invention also provides a system for constructing the defect detection model. As shown in fig. 3, fig. 3 is a schematic structural diagram of a hardware running environment according to an embodiment of the present invention.
It should be noted that fig. 3 is a schematic structural diagram of a hardware running environment of the system for constructing the defect detection model.
As shown in fig. 3, the system for constructing the defect detection model may include: a processor 1001 such as a CPU (Central Processing Unit ), a memory 1005, a user interface 1003, a network interface 1004, a communication bus 1002, and a GPU graphics card. The GPU graphics card and processor 1001 has functions of image analysis, model construction, image positioning, image detection, etc., and the communication bus 1002 is used for realizing connection communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a keyboard (board), and the optional user interface 1003 may further include a standard wired interface (e.g., USB (Universal Serial Bus, universal serial bus) interface), a wireless interface (e.g., bluetooth interface). The network interface 1004 may optionally include a standard wired interface, a Wireless interface, such as a WI-FI (Wireless-Fidelity) interface. The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above. The memory 1005 is mainly used for storing a sample defect image and a defect detection mode.
Optionally, the system for constructing the defect detection model may further include an RF (Radio Frequency) circuit, a sensor, a WiFi module, and the like.
It will be appreciated by those skilled in the art that the construction system architecture of the defect detection model shown in fig. 3 does not constitute a limitation of the construction system of the defect detection model, and may include more or fewer components than shown, or may combine certain components, or may be a different arrangement of components.
As shown in fig. 3, a memory 1005, which is a computer storage medium, may include an operating system, a network communication module, a user interface module, and a program for constructing a defect detection model. The operating system is a program for managing and controlling system hardware and software resources for constructing the defect detection model, and supports the construction program of the defect detection model and the running of other software or programs.
In the system for constructing a defect detection model shown in the figure, the user interface 1003 is mainly used for an image acquisition device to acquire a sample product image and a product image to be detected; the network interface 1004 is mainly used for a defect detection system and image transmission with the image acquisition equipment; the processor 1001 may be configured to call a build program of the defect detection model stored in the memory 1005 and complete the steps of the control method of the build system of the defect detection model as described above.
The specific implementation manner of the system for constructing the defect detection model is basically the same as that of each embodiment of the method for constructing the defect detection model, and is not repeated here.
In addition, an embodiment of the present invention also proposes a computer-readable storage medium having stored thereon a program for constructing a defect detection model, which when completed by a processor, implements the steps of the method for constructing a defect detection model as described above.
The specific implementation manner of the computer readable storage medium of the present invention is basically the same as the above-mentioned embodiments of the method for constructing the defect detection model, and will not be described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above embodiment method may be implemented by means of software plus necessary general hardware platform, or of course by means of hardware, but the former is a preferred embodiment under many data. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of software goods stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a system for constructing a defect detection model to perform the method according to the embodiments of the present invention.
Claims (5)
1. The method for constructing the defect detection model is characterized by comprising the following steps of:
the method comprises the steps of performing multi-angle acquisition on a sample product through image acquisition equipment to obtain a sample product image with a preset number of parts, and performing image analysis on the sample product image to obtain a sample defect image;
the method for obtaining the sample defect image comprises the steps of performing multi-angle collection on a sample product through image collection equipment to obtain a sample product image with a preset number of parts, and performing image analysis on the sample product image to obtain the sample defect image, wherein the steps of obtaining the sample defect image comprise:
performing multi-angle polishing on the sample product through image acquisition equipment, and performing multi-angle acquisition on the polished sample product to obtain the sample product images with the preset number of parts;
image decision is carried out on the sample product image through a decision network model, so that a decision value of the sample product image is obtained;
determining a sample defect image corresponding to the sample product image according to the decision value;
obtaining a defect image training set according to the relevance of image labels in the sample defect image, and performing image training on the defect image training set to obtain a defect detection model;
the obtaining the defect image training set according to the relevance of the image labels in the sample defect image comprises the following steps:
acquiring image tags in the sample defect image, and determining the same number of the image tags in the sample defect image according to the relevance of the image tags; obtaining the similarity degree of the image labels in the sample defect image according to the same number; classifying the sample defect images according to the similarity degree to obtain defect image training sets corresponding to various sample defect images;
obtaining a defect image training set according to the relevance of the image labels in the sample defect image, and performing image training on the defect image training set to obtain a defect detection model, wherein the method further comprises the following steps:
acquiring a product image to be detected through the image acquisition equipment, and identifying an image tag in the product image to be detected; matching an image detection model corresponding to the image label of the product image to be detected in the defect detection model, and carrying out image detection on the product image to be detected through the image detection model to obtain a detection result.
2. The method for constructing a defect detection model according to claim 1, wherein the step of determining a sample defect image corresponding to the sample product image according to the decision value comprises:
judging whether the decision value of the sample product image is larger than a preset decision value or not;
if the decision value of the sample product image is larger than the preset decision value, positioning the defective pixels of the sample product image to obtain a positioning image, and dividing the sample product image according to the positioning image to obtain a divided image comprising the positioning image;
and attaching an image tag corresponding to the sample product image to the segmented image to obtain the sample defect image.
3. The device for constructing the defect detection model is characterized by comprising the following components:
the acquisition module is used for acquiring sample products at multiple angles through the image acquisition equipment to obtain sample product images with preset parts;
the acquisition module is also used for acquiring the data of the sample,
performing multi-angle polishing on the sample product through image acquisition equipment, and performing multi-angle acquisition on the polished sample product to obtain the sample product images with the preset number of parts; image decision is carried out on the sample product image through a decision network model, so that a decision value of the sample product image is obtained; determining a sample defect image corresponding to the sample product image according to the decision value;
the analysis module is used for carrying out image analysis on the sample product image to obtain a sample defect image;
the determining module is used for obtaining a defect image training set according to the relevance of the image labels in the sample defect image;
the training module is used for carrying out image training on the defect image training set to obtain a defect detection model;
the training module is further used for acquiring image tags in the sample defect image, and determining the same number of the image tags in the sample defect image according to the relevance of the image tags; obtaining the similarity degree of the image labels in the sample defect image according to the same number; classifying the sample defect images according to the similarity degree to obtain defect image training sets corresponding to various sample defect images;
the device for constructing the defect detection model is also used for,
acquiring a product image to be detected through the image acquisition equipment, and identifying an image tag in the product image to be detected; matching an image detection model corresponding to the image label of the product image to be detected in the defect detection model, and carrying out image detection on the product image to be detected through the image detection model to obtain a detection result.
4. A system for constructing a defect detection model, wherein the system for constructing a defect detection model comprises a memory, a processor, a GPU display card and a program for constructing a defect detection model stored in the memory and running on the processor and the GPU display card, and the program for constructing a defect detection model realizes the steps of the method for constructing a defect detection model according to any one of claims 1 to 2 when the program for constructing a defect detection model is completed by the processor and the GPU display card.
5. A computer-readable storage medium, wherein a program for constructing a defect detection model is stored on the computer-readable storage medium, and the program for constructing a defect detection model, when completed by a processor and a GPU display card, implements the steps of the method for constructing a defect detection model according to any one of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010502442.1A CN111681220B (en) | 2020-06-04 | 2020-06-04 | Method, device, system and storage medium for constructing defect detection model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010502442.1A CN111681220B (en) | 2020-06-04 | 2020-06-04 | Method, device, system and storage medium for constructing defect detection model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111681220A CN111681220A (en) | 2020-09-18 |
| CN111681220B true CN111681220B (en) | 2024-02-13 |
Family
ID=72453961
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010502442.1A Active CN111681220B (en) | 2020-06-04 | 2020-06-04 | Method, device, system and storage medium for constructing defect detection model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111681220B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112365491A (en) * | 2020-11-27 | 2021-02-12 | 上海市计算技术研究所 | Method for detecting welding seam of container, electronic equipment and storage medium |
| CN114113119A (en) * | 2021-12-03 | 2022-03-01 | 浙江伟星实业发展股份有限公司 | Product visual quality inspection method, system, equipment and medium based on defect grade |
| CN115661155A (en) * | 2022-12-28 | 2023-01-31 | 北京阿丘机器人科技有限公司 | Defect detection model construction method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109978870A (en) * | 2019-03-29 | 2019-07-05 | 北京百度网讯科技有限公司 | Method and apparatus for output information |
| CN110148130A (en) * | 2019-05-27 | 2019-08-20 | 北京百度网讯科技有限公司 | Method and apparatus for detecting part defect |
| CN110599453A (en) * | 2019-08-08 | 2019-12-20 | 武汉精立电子技术有限公司 | Panel defect detection method and device based on image fusion and equipment terminal |
| CN110619619A (en) * | 2018-06-04 | 2019-12-27 | 杭州海康威视数字技术股份有限公司 | Defect detection method and device and electronic equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108776808A (en) * | 2018-05-25 | 2018-11-09 | 北京百度网讯科技有限公司 | A kind of method and apparatus for detecting ladle corrosion defect |
-
2020
- 2020-06-04 CN CN202010502442.1A patent/CN111681220B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110619619A (en) * | 2018-06-04 | 2019-12-27 | 杭州海康威视数字技术股份有限公司 | Defect detection method and device and electronic equipment |
| CN109978870A (en) * | 2019-03-29 | 2019-07-05 | 北京百度网讯科技有限公司 | Method and apparatus for output information |
| CN110148130A (en) * | 2019-05-27 | 2019-08-20 | 北京百度网讯科技有限公司 | Method and apparatus for detecting part defect |
| CN110599453A (en) * | 2019-08-08 | 2019-12-20 | 武汉精立电子技术有限公司 | Panel defect detection method and device based on image fusion and equipment terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111681220A (en) | 2020-09-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111681220B (en) | Method, device, system and storage medium for constructing defect detection model | |
| KR101880004B1 (en) | Method and apparatus for identifying television channel information | |
| JP6208383B2 (en) | Image capturing parameter adjustment in preview mode | |
| US8126264B2 (en) | Device and method for identification of objects using color coding | |
| WO2019169772A1 (en) | Picture processing method, electronic apparatus, and storage medium | |
| CN104616021B (en) | Traffic sign image processing method and device | |
| CN112136139A (en) | Method and system for improving quality inspection | |
| US9171224B2 (en) | Method of improving contrast for text extraction and recognition applications | |
| CN112136154A (en) | Method and system for improving quality inspection | |
| CN107292307B (en) | Automatic identification method and system for inverted Chinese character verification code | |
| AU2012321562A1 (en) | Device and method for automatically identifying a QR code | |
| KR101659091B1 (en) | Device and method for creating text collage message | |
| CN110197238B (en) | Font type identification method, system and terminal equipment | |
| CN111507324A (en) | Card frame identification method, device, equipment and computer storage medium | |
| CN116071294A (en) | Optical fiber surface defect detection method and device | |
| CN110596118A (en) | Print pattern detection method and print pattern detection device | |
| CN114332058A (en) | Serum quality identification method, device, equipment and medium based on neural network | |
| CN113159029A (en) | Method and system for accurately capturing local information in picture | |
| CN112446421A (en) | Silkworm cocoon counting and identifying method based on machine vision | |
| CN110210401B (en) | Intelligent target detection method under weak light | |
| JP2010186246A (en) | Image processing apparatus, method, and program | |
| CN115760854A (en) | Deep learning-based power equipment defect detection method and device and electronic equipment | |
| CN111428796B (en) | General object detection method and system based on deep learning | |
| CN111008676B (en) | Security inspection method and security inspection system | |
| CN112884740A (en) | Image detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |