CN111428374A - Part defect detection method, device, equipment and storage medium - Google Patents
Part defect detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111428374A CN111428374A CN202010234961.4A CN202010234961A CN111428374A CN 111428374 A CN111428374 A CN 111428374A CN 202010234961 A CN202010234961 A CN 202010234961A CN 111428374 A CN111428374 A CN 111428374A
- Authority
- CN
- China
- Prior art keywords
- model
- simulation
- defect
- defect detection
- simulated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 269
- 238000001514 detection method Methods 0.000 title claims abstract description 193
- 238000003860 storage Methods 0.000 title claims abstract description 35
- 238000004088 simulation Methods 0.000 claims abstract description 189
- 238000012549 training Methods 0.000 claims abstract description 159
- 238000000034 method Methods 0.000 claims abstract description 44
- 238000012545 processing Methods 0.000 claims abstract description 22
- 230000002950 deficient Effects 0.000 claims description 17
- 238000009826 distribution Methods 0.000 claims description 13
- 238000007689 inspection Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 3
- 238000013461 design Methods 0.000 description 8
- 238000009877 rendering Methods 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 3
- 238000012938 design process Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000010923 batch production Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012372 quality testing Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8806—Specially adapted optical and illumination features
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The embodiment of the invention discloses a method, a device, equipment and a storage medium for detecting part defects. The method comprises the following steps: determining a simulation part sample model; processing the simulated part sample model based on a simulated light source model to determine a defect detection simulation training sample set; and performing model training according to the defect detection simulation training sample set, and determining a part defect detection model for detecting the defects of the part real object. The embodiment of the application can solve the problem that when the part defect is detected, enough and high-quality training data are difficult to obtain, and the performance of the assembly quality detection model obtained by training is poor, so that the high-precision training data can be obtained to timely obtain the high-performance quality detection model before the part defect is detected, and the effect of the part defect detection accuracy is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of computer vision, in particular to a method, a device, equipment and a storage medium for detecting part defects.
Background
Computer vision technology based on deep learning is widely applied to intelligent manufacturing, a large amount of training data is still needed in the deep learning training process at present, and the quality and the quantity of training samples determine the quality of a training model.
In some application scenarios, the problem of insufficient training data is particularly prominent, namely the "small sample" problem. For example, some fields are in a limited space, for example, when detecting a part defect, shooting a large amount of training data presents an implementation-level obstacle, resulting in a small amount of training data being obtained. At present, in order to solve the above problems, there is a technology for automatically generating training samples, but the automatically generated training pictures are often inconsistent with the environment of the actual application scene, so that when the training models obtained by using these samples are detected in the actual application, a deviation in the detection result occurs.
In addition, the training time required for training the model by adopting algorithms such as deep learning is relatively long, training data needs to be acquired on site and then subjected to preprocessing, and the period of the whole process is long. For a scene of small-batch production, after the deep learning training process is completely finished, the whole production process may be finished and terminated, so that the formation of a model cannot meet the speed requirement of actual part defect detection.
Disclosure of Invention
The embodiment of the invention provides a method, a device, equipment and a storage medium for detecting part defects, which are used for solving the problem that the performance of a trained part defect detection model is poor due to the fact that enough and high-quality training data are difficult to obtain when the part defects are detected.
In a first aspect, an embodiment of the present invention provides a method for detecting a defect of a part, where the method includes:
determining a simulation part sample model;
processing the simulated part sample model based on a simulated light source model to determine a defect detection simulation training sample set;
and performing model training according to the defect detection simulation training sample set, and determining a part defect detection model for detecting the defects of the part real object.
In a second aspect, an embodiment of the present invention provides a part defect detecting apparatus, including:
the simulation part sample model determining module is used for determining a simulation part sample model;
the defect detection simulation training sample set determining module is used for processing the simulation part sample model based on a simulation light source model and determining a defect detection simulation training sample set;
and the part defect detection model determining module is used for performing model training according to the defect detection simulation training sample set, determining a part defect detection model and detecting the defects of the part real object.
In a third aspect, an embodiment of the present invention further provides an apparatus, where the apparatus includes:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a part defect detection method as in any one of the embodiments of the invention.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the part defect detection method according to any one of the embodiments of the present invention.
In the embodiment of the invention, a simulation part sample model is determined; based on a simulated light source model, processing the simulated part sample model, determining a defect detection simulated training sample set, performing model training according to the defect detection simulated training sample set after acquiring a high-quality, comprehensive and sufficient training sample image set, and determining a part defect detection model for detecting the defects of a part real object, thereby realizing the effect of acquiring high-precision training data to obtain a high-performance part defect detection model in time before detecting the part defects so as to improve the accuracy of the part defect detection.
Drawings
FIG. 1 is a flow chart of a method for detecting defects of a part according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for detecting defects of a part according to another embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a part defect detection apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a part defect detecting apparatus according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Fig. 1 is a flowchart of a method for detecting a defect of a part according to an embodiment of the present invention. The method for detecting the part defects provided by the embodiment can be suitable for the condition of detecting the part defects, and typically, the embodiment of the invention can be suitable for the condition of obtaining a training sample set through a simulation environment in the process of product assembly design or after the product assembly design, and obtaining a part defect detection model through model training, wherein the part defect detection model is used for detecting the defects of a part real object. The method may be specifically performed by a part defect detection apparatus, which may be implemented by software and/or hardware, which may be integrated in a part defect detection device. Referring to fig. 1, the method of the embodiment of the present invention specifically includes:
and S110, determining a simulation part sample model.
The simulation part sample model can comprise a simulation defective part model and/or a simulation non-defective part model, the simulation defective part model is constructed by defect data and the simulation part model, and the simulation non-defective part model can be a non-defective simulation part model. The defect data may be data obtained by measuring a part real object in which a defect exists.
In addition, a simulation light source model is required to be built on the three-dimensional simulation platform, the parameters of the simulation light source model are the same as the parameters of a real light source in the real assembly environment, the parameters comprise the light source specification parameters and the light source angle parameters of the simulation light source model, the illumination environment in the simulation assembly environment is ensured to be consistent with the illumination environment in the real assembly environment, the real assembly environment is restored, so that the image acquired in the simulation assembly environment can be consistent with the image acquired in the real environment, and the reduction degree and the accuracy of the training image are improved.
In the embodiment of the application, a simulation part model and a simulation light source model can be constructed according to parameters of a part real object and parameters of a light source in a real assembly environment, and a design part model and a design environment model constructed in the design process of a product assembly production line can also be obtained and used as the simulation part model and the simulation environment model.
In the embodiment of the application, the simulation part sample model is determined, so that a defect detection training sample set is obtained according to the simulation part sample model subsequently, and the model training is carried out to obtain the part defect detection model.
And S120, processing the simulated part sample model based on the simulated light source model, and determining a defect detection simulation training sample set.
Illustratively, determining a simulated part sample model includes: generating a simulation defect model according to the defect data; and embedding the simulation defect model into a simulation part model to obtain the simulation defect part model. Embedding the simulation defect model into a simulation part model to obtain a simulation defect part model, wherein the simulation defect part model comprises the following steps: aiming at different types of defects, acquiring a position distribution rule of the defects on a part real object; wherein the position distribution rule comprises a random distribution rule or a statistical rule; and determining the embedding position of the simulation defect model on the simulation part model according to the position distribution rule corresponding to each type of defect. Correspondingly, based on the simulated light source model, processing the simulated part sample model to determine a defect detection simulation training sample set, including: determining label information corresponding to a simulation defect part model according to the defect data, wherein the label information is used as label information of a negative training sample image in the defect detection training sample; wherein the tag information includes a type of the defect and/or a location of the defect.
For example, the defect data of the part real object can be obtained by measuring the part real object with the defect before the model is built, and the simulation defect part model can be determined through the defect data and the simulation part model. Determining label information corresponding to a simulation defect part model according to the defect data, wherein the label information is used as the label information of a training image in the defect detection training sample; wherein the tag information includes a type of the defect and/or a location of the defect. The types of defects may include: scratches, cracks, splashes, etc.
Specifically, the simulated part model may be directly operated according to the defect data, for example, if the defect data is a hemispherical protrusion with a diameter of 1mm, the hemispherical protrusion with a diameter of 1mm is directly arranged on the simulated part model, so as to obtain the simulated defective part model. For example, for a scratch having a complicated structure, a scratch model may be generated, and the simulated defect model may be embedded in the simulated part model to obtain the simulated defect part model.
In the embodiment of the application, the embedded position of the simulated defect model on the simulated part model can be realized in various specific ways. For example, random position embedding is adopted, and the method is suitable for the condition that the position of the surface defect of the actual part is irregular. For the condition that the surface defect generating position accords with the statistical rule, the position where the surface defect actually generates can be counted to obtain a proper embedding position. For example, the splash-like defects are irregularly and randomly generated and may appear at any position on the surface of the part real object, so that when the simulated defect model is embedded into the simulated part model, the position can be randomly selected for embedding. For the crack defects, the occurrence of the crack defects is regular, for example, cracks are easy to generate at certain parts of the part, therefore, the positions of the surface of the part where the cracks are easy to generate are determined according to experience and statistical data, and the simulated defect model is embedded into the simulated part model according to the statistical positions. And rendering the simulated part defect model in the light source environment of the simulated light source model to obtain a defect detection negative training sample image. For each simulation defect part model, because the defect type and/or position of the simulation defect part model are known, the label information corresponding to the simulation defect part model can be accurately and quickly determined and used as the label information of the negative training sample image obtained through the simulation defect part model. The specific representation form may be that a gray image with an equal size is generated, the gray value represents the defect type (for example, a gray value of 0 represents a normal region, a gray value of 1 represents a scratch, and a gray value of 2 represents a crack, etc.), and regions with different gray values represent different defect regions.
In the embodiment of the application, the simulated flawless part model can be rendered under the light source environment of the simulated light source model to obtain the defect detection training sample image set. For a simulated flawless part model, since it is known to be flawless, label information of a positive training sample image obtained by the simulated flawless part model can be directly obtained.
In the embodiment of the application, the defect detection simulation training sample set may be a defect detection positive training sample image set and label information, may also be a defect detection negative training sample image set and label information, and may also include both the defect detection positive training sample image set and the label information, and also include the defect detection negative training sample image set and the label information.
In the embodiment of the application, the simulation defective part model is obtained by embedding the simulation part model in the simulation environment according to the known defect data, and the simulation non-defective part model is known to be non-defective, so that the label information of the defect detection training image obtained by simulating the defective part model can be timely, accurately and automatically generated according to the known defect data, and the label information and the training image can be generated simultaneously.
And S130, performing model training according to the defect detection simulation training sample set, and determining a part defect detection model for detecting the defects of the part real object.
Illustratively, based on a deep learning algorithm, model training is performed according to a defect detection simulation model training sample set, so that a part defect detection model is obtained, the part defect detection model is used for detecting defects of parts, and a detection result is obtained for technicians to check. The deep learning algorithm can be a Mask R-CNN, Unet or PSPnet algorithm model.
In the embodiment of the invention, a simulation part sample model is determined; based on a simulated light source model, processing the simulated part sample model, determining a defect detection simulation training sample set, performing model training according to the defect detection simulation training sample set after obtaining a high-quality, comprehensive and sufficient training sample image set, determining a part defect detection model for detecting the defects of a part real object, and in addition, because the training of the part defect detection model is completed in the process of production line design, the problems that the model training period is long and the model cannot be formed to meet the speed requirement of actual part defect detection are solved, so that the high-precision training data can be obtained to obtain the high-performance part defect detection model in time before the part defect detection is performed, and the effect of the accuracy of the part defect detection is improved.
Fig. 2 is a flowchart of a part defect detection method according to another embodiment of the present invention. For further optimization of the embodiments, details which are not described in detail in the embodiments are described in the embodiments. Referring to fig. 2, the method for detecting a defect of a part according to this embodiment may include:
s201, determining a simulation part sample model.
S202, processing a simulated part sample model through at least one angle based on a simulated light source model to obtain a defect detection training sample image set; and the at least one angle is consistent with the acquisition angle of the defect detection image of the part real object.
Illustratively, in a simulation environment, a simulation part sample model is rendered through at least one angle to obtain a defect detection training sample image set. The number of the training sample images in the defect detection training sample image set may be determined according to actual conditions, and may be 1000 sheets, for example. The defect detection training sample image set can be obtained by rendering one simulation part sample module through a plurality of angles, can also be obtained by rendering a plurality of simulation part sample modules through one angle, and can also be obtained by rendering a plurality of simulation part sample modules through a plurality of angles, and is not particularly limited herein, and can be rendered according to actual conditions.
In the embodiment of the application, at least one rendered angle is consistent with the acquisition angle of the quality detection image of the part real object, so that the condition of the part real object can be truly reflected by the defect detection training sample image set obtained by rendering, the defect detection training sample image set is consistent with the defect detection training sample image of the part real object acquired in a real environment, and the reliability and the scene reducibility are higher.
S203, determining a defect detection simulation training sample set according to the defect detection training sample image set and label information corresponding to the defect detection training sample image.
And S204, performing model training according to the defect detection simulation training sample set, and determining a part defect detection model for detecting the defects of the part real object.
And S205, assembling the simulation part model to obtain a simulation product model.
In the embodiment of the application, the quality of product assembly can be detected. Similarly, in the product assembly design process or after the product assembly design, before the product object assembly is implemented, the simulation part model is assembled through the three-dimensional simulation platform to obtain the simulation product model. Since the product assembly in the real assembly environment is actually a process of assembling the part real objects to obtain the product real objects, in the simulation environment, the assembly process in the real assembly environment is simulated, and the simulation part model is assembled into the simulation product model.
In the embodiment of the application, the simulation product model may be a positive sample model of a simulation product with correct assembly, may also be a negative sample model of a simulation product with incorrect assembly, and may also include both a positive sample model and a negative sample model of a simulation product.
S206, processing the simulation product model based on the simulation light source model, and determining a quality detection simulation training sample set.
Illustratively, assembling the simulation part model to obtain the simulation product model includes: and assembling the simulation part model based on a predicted wrong product assembling mode to obtain a simulation product negative sample model. Correspondingly, based on the simulated light source model, processing the simulated product model to determine a quality detection simulation training sample set, comprising: determining error assembly label information corresponding to a simulation product negative sample model according to a predicted error product assembly mode, wherein the error assembly label information is used as label information of a negative training sample image in the quality detection simulation training sample set; wherein the tag information comprises a type of mis-assembly and/or a location of the mis-assembly.
Specifically, the product assembly mode of assembly errors which can occur in the product assembly process is determined according to daily experience, such as multi-assembly of parts (such as multi-assembly of rivets), neglected assembly (such as missing of small parts like screws), misassembly (reverse assembly of automobile rearview mirrors) and the like, wherein the misassembly specifically means that the assembly position of the parts is wrong, and the position of the possible misassembly is determined. And assembling the simulation part model into a simulation product negative sample model according to a known error assembly mode, and rendering the simulation product negative sample model in the light source environment of the simulation light source model to obtain a quality detection negative training sample image. For each simulation product negative sample model, the corresponding misassembly mode is known, that is, the corresponding misassembly type and/or misassembly position are known, so that the label information corresponding to the simulation product negative sample model can be accurately and quickly determined and used as the label information of the negative training sample image obtained through the simulation product negative sample model. The expression form of the label information may be that a rectangular frame with different colors is used to mark an error area in the negative training sample image, and the type of the incorrect assembly is displayed in the upper left corner of the rectangular frame.
Illustratively, assembling the simulation part model to obtain the simulation product model further comprises: and assembling the simulation part model based on a predicted correct product assembling mode to obtain a simulation product positive sample model. Correspondingly, based on the simulated light source model, processing the simulated product model to determine a quality detection simulation training sample set, further comprising: and determining correct assembly label information corresponding to the simulation product positive sample model according to a correct product assembly mode, wherein the correct assembly label information is used as label information of the positive training sample image in the quality detection simulation training sample set.
Specifically, the quality detection simulation training sample set may further include a positive training sample. The positive training sample is a positive training sample image obtained by collecting a positive sample model image of a correctly assembled simulation product and corresponding label information. Similarly, the simulation part model is assembled according to a known correct product assembly mode to obtain a simulation product positive sample model, and the simulation product positive sample model is rendered under the light source environment of the simulation light source model to obtain a quality detection positive training sample image. Because the positive sample model of the simulation product is known to be the model with correct assembly, the label information corresponding to the positive sample model of the simulation product is also known and is used as the label information of the positive training sample image obtained by the positive sample model of the simulation product.
In the embodiment of the application, the simulation part model is assembled according to the known assembly mode in the simulation environment to obtain the simulation product model, so that the label information of the quality detection training sample image obtained through the simulation product model can be timely, accurately and automatically generated according to the known assembly mode, the label information and the training image can be generated simultaneously, compared with the existing scheme for manually determining the label information, the scheme in the embodiment of the application not only can realize the automation of label determination, but also can avoid the problem of low error accuracy rate of the label information caused by artificial subjective judgment in the scheme for manually determining the label information of the training image, and therefore the accuracy of the label information is improved.
And S207, performing model training according to the quality detection simulation training sample set, and determining a product assembly quality detection model for detecting the assembly quality of a product object.
Illustratively, based on a deep learning algorithm, model training is carried out according to a quality detection simulation model training sample set, so as to obtain a product quality detection model, which is used for detecting the assembly quality of a product real object, and obtaining a detection result for a technician to check.
And S208, acquiring a defect detection image of the part real object.
In the embodiment of the application, the defect detection image of the part real object can be acquired through at least one angle. And in the defect detection process, obtaining a defect detection image for detecting the defects of the parts. And the acquisition angle of the defect detection image is consistent with the acquisition angle of the defect detection training sample image set. In the product entity assembly environment, the quality detection image of the product entity is acquired through at least one angle. In the process of detecting the assembly quality of the product, a quality detection image is obtained, the quality detection image of the product real object is subjected to image recognition to obtain a defect detection image of the part real object,
s209, determining the detection result of the part defect according to the defect detection image based on the part defect detection model.
Illustratively, the defect detection image of the part real object is input into the part defect detection model to obtain the detection result of the part defect. When the defect detection training sample set is a positive training sample set, the detection result may include whether the part is defective; when the defect detection training sample set is a negative training sample set, the detection result may include whether the part is defective, and the type and/or location of the defect.
S210, acquiring a quality detection image of a product object; wherein, the product entity is obtained by assembling part entities.
In the product object assembly environment, acquiring a quality detection image of a product object through at least one angle, and acquiring the quality detection image for detecting the assembly quality of the product. And the acquisition angle of the quality detection image is consistent with the acquisition angle of the quality detection training sample image set.
S211, determining a detection result of the product assembly quality according to the quality detection image based on the product assembly quality detection model.
Illustratively, the quality detection image is input into the product assembly quality detection model to obtain a detection result of the product assembly quality. When the quality detection training sample set is a positive training sample set, the detection result may include whether the product assembly is correct; when the quality testing training sample set is a negative training sample set, the testing result may include whether the product assembly is wrong and the type and/or location of the mistake; when the quality test training sample set includes both a positive training sample set and a negative training sample set, the test results may include product assembly authenticity or error, and the type and/or location of the error.
It should be noted that, in the embodiment of the present application, the execution sequence of determining the product assembly quality inspection model and determining the part defect inspection model is not specifically limited, the quality inspection model may be determined first, or the part defect inspection model may be determined, and similarly, the execution process of the product assembly quality inspection and the part defect inspection is not specifically limited, and the part defect inspection may be executed after the product assembly quality inspection, or may be executed before the product assembly quality inspection. In general, considering that the surface of the part may be damaged during the product assembly process to cause the part to have defects, the part defect detection is arranged to be performed after the product assembly quality detection.
According to the technical scheme of the embodiment of the invention, at least one rendered angle is consistent with the acquisition angle of the defect detection image in the part real object detection environment, so that the defect detection training sample image set obtained by rendering can truly reflect the condition of the part real object, is consistent with the defect detection training sample image acquired in the real environment, and has higher reliability and scene reducibility. And the detection of the assembly quality of the product can be realized, and the product quality is further ensured.
Fig. 3 is a schematic structural diagram of a part defect detection apparatus according to an embodiment of the present invention. The device can be suitable for detecting the defects of the parts, typically, the embodiment of the invention can be suitable for detecting the defects of the parts, and typically, the embodiment of the invention can be suitable for obtaining a training sample set through a simulation environment in the process of product assembly design or after the product assembly design, and obtaining a part defect detection model through model training, wherein the part defect detection model is used for detecting the defects of a part real object. The apparatus may be implemented by software and/or hardware, and the apparatus may be integrated in a device. Referring to fig. 3, the apparatus specifically includes:
a simulation part sample model determining module 310, configured to determine a simulation part sample model;
a defect detection simulation training sample set determining module 320, configured to process the simulation part sample model based on a simulation light source model, and determine a defect detection simulation training sample set;
and the part defect detection model determining module 330 is configured to perform model training according to the defect detection simulation training sample set, determine a part defect detection model, and detect a defect of a part real object.
In the embodiment of the application, the simulation part sample model comprises a simulation defective part model and/or a simulation non-defective part model; and the parameters of the simulation light source model are the same as the parameters of the real light source in the part real object defect detection environment.
In an embodiment of the present application, the simulation part sample model determining module 310 includes:
the simulation defect model generating unit is used for generating a simulation defect model according to the defect data;
and the defect embedding unit is used for embedding the simulation defect model into a simulation part model to obtain the simulation defect part model.
In an embodiment of the present application, the defect embedding unit includes:
the distribution rule obtaining subunit is used for obtaining the position distribution rule of the defects on the part real object aiming at the defects of different types; wherein the position distribution rule comprises a random distribution rule or a statistical rule;
and the embedded position determining subunit is used for determining the embedded position of the simulation defect model on the simulation part model according to the position distribution rule corresponding to each type of defect.
In this embodiment of the application, the defect detection simulation training sample set determining module 320 is specifically configured to:
determining label information corresponding to a simulation defect part model according to the defect data, wherein the label information is used as label information of a negative training sample image in the defect detection training sample; wherein the tag information includes a type of the defect and/or a location of the defect.
In this embodiment of the application, the defect detection simulation training sample set determining module 320 includes:
the rendering unit is used for processing the simulation part sample model through at least one angle based on the simulation light source model to obtain a defect detection training sample image set; wherein the at least one angle is consistent with the acquisition angle of the defect detection image of the part real object;
and the sample set determining unit is used for determining the defect detection simulation training sample set according to the defect detection training sample image set and the label information corresponding to the defect detection training sample image.
In an embodiment of the present application, the apparatus further includes:
the defect detection image acquisition module is used for acquiring a defect detection image of a part real object;
and the defect detection module is used for determining the detection result of the part defects according to the defect detection image based on the part defect detection model.
In an embodiment of the present application, the apparatus further includes:
the assembly simulation module is used for assembling the simulation part model to obtain a simulation product model;
the quality detection simulation training sample set determining module is used for processing the simulation product model based on a simulation light source model to determine a quality detection simulation training sample set;
and the product assembly quality detection model determining module is used for performing model training according to the quality detection simulation training sample set, determining a product assembly quality detection model and detecting the assembly quality of a product object.
The part defect detection device provided by the embodiment of the application can execute the part defect detection method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method.
Fig. 4 is a schematic structural diagram of a part defect detecting apparatus according to an embodiment of the present invention. FIG. 4 illustrates a block diagram of an exemplary device 412 suitable for use in implementing embodiments of the present invention. The device 412 shown in fig. 4 is only an example and should not impose any limitation on the functionality or scope of use of embodiments of the present invention.
As shown in fig. 4, the device 412 may be fixed or head-mounted, including: one or more processors 416; the memory 428 is configured to store one or more programs, and when the one or more programs are executed by the one or more processors 416, the one or more processors 416 implement the method for detecting a defect of a part according to the embodiment of the present invention, including:
determining a simulation part sample model;
processing the simulated part sample model based on a simulated light source model to determine a defect detection simulation training sample set;
and performing model training according to the defect detection simulation training sample set, and determining a part defect detection model for detecting the defects of the part real object.
Is expressed in the form of general-purpose equipment. The components of device 412 may include, but are not limited to: one or more processors or processors 416, a device memory 428, and a bus 418 that couples the various device components including the device memory 428 and the processors 416.
The device memory 428 may include computer device readable storage media in the form of volatile memory, such as Random Access Memory (RAM)430 and/or cache memory 432. The device 412 may further include other removable/non-removable, volatile/nonvolatile computer device storage media. By way of example only, storage device 434 may be used to read from and write to non-removable, nonvolatile magnetic storage media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical storage medium) may be provided. In these cases, each drive may be connected to bus 418 by one or more data storage media interfaces. Memory 428 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 440 having a set (at least one) of program modules 442 may be stored, for instance, in memory 428, such program modules 462 including, but not limited to, an operating device, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 462 generally perform the functions and/or methodologies of the described embodiments of the invention.
The device 412 may also communicate with one or more external devices 414 (e.g., keyboard, pointing device, display 426, etc.), and may also communicate with one or more devices that enable a user to interact with the device 412, and/or with any devices (e.g., network card, modem, etc.) that enable the device 412 to communicate with one or more other computing devices.
The processor 416 performs various functional applications and data processing, such as implementing a method for part defect detection provided by embodiments of the present invention, by executing at least one of the other programs stored in the device memory 428.
One embodiment of the present invention provides a storage medium containing computer-executable instructions that, when executed by a computer processor, perform a method for part defect detection:
determining a simulation part sample model;
processing the simulated part sample model based on a simulated light source model to determine a defect detection simulation training sample set;
and performing model training according to the defect detection simulation training sample set, and determining a part defect detection model for detecting the defects of the part real object.
Computer storage media for embodiments of the present invention can take the form of any combination of one or more computer-readable storage media. The computer readable storage medium may be a computer readable signal storage medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the invention, the computer readable storage medium may be any tangible storage medium that can contain, or store a program for use by or in connection with an instruction execution apparatus, device, or apparatus.
A computer readable signal storage medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal storage medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution apparatus, device, or apparatus.
Program code embodied on a computer readable storage medium may be transmitted using any appropriate storage medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including AN object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (11)
1. A method of detecting a defect in a part, the method comprising:
determining a simulation part sample model;
processing the simulated part sample model based on a simulated light source model to determine a defect detection simulation training sample set;
and performing model training according to the defect detection simulation training sample set, and determining a part defect detection model for detecting the defects of the part real object.
2. The method of claim 1, wherein the simulated part sample model comprises a simulated defective part model and/or a simulated non-defective part model; and the parameters of the simulation light source model are the same as the parameters of the real light source in the part real object defect detection environment.
3. The method of claim 1, wherein determining a simulated part sample model comprises:
generating a simulation defect model according to the defect data;
and embedding the simulation defect model into a simulation part model to obtain the simulation defect part model.
4. The method of claim 3, wherein embedding the simulated defect model into a simulated part model to obtain a simulated defective part model comprises:
aiming at different types of defects, acquiring a position distribution rule of the defects on a part real object; wherein the position distribution rule comprises a random distribution rule or a statistical rule;
and determining the embedding position of the simulation defect model on the simulation part model according to the position distribution rule corresponding to each type of defect.
5. The method of claim 3 or 4, wherein processing the simulated part sample model based on a simulated light source model to determine a defect detection simulated training sample set comprises:
determining label information corresponding to a simulation defect part model according to the defect data, wherein the label information is used as label information of a negative training sample image in the defect detection training sample; wherein the tag information includes a type of the defect and/or a location of the defect.
6. The method of claim 1, wherein processing the simulated part sample model based on a simulated light source model to determine a defect detection simulated training sample set comprises:
processing the simulated part sample model through at least one angle based on the simulated light source model to obtain a defect detection training sample image set; wherein the at least one angle is consistent with the acquisition angle of the defect detection image of the part real object;
and determining a defect detection simulation training sample set according to the defect detection training sample image set and the label information corresponding to the defect detection training sample image.
7. The method of claim 1, wherein after determining the part defect inspection model, the method further comprises:
acquiring a defect detection image of a part real object;
and determining the detection result of the part defect according to the defect detection image based on the part defect detection model.
8. The method of claim 1, further comprising:
assembling the simulation part model to obtain a simulation product model;
processing the simulation product model based on a simulation light source model to determine a quality detection simulation training sample set;
and performing model training according to the quality detection simulation training sample set, and determining a product assembly quality detection model for detecting the assembly quality of a product object.
9. A part defect detection apparatus, the apparatus comprising:
the simulation part sample model determining module is used for determining a simulation part sample model;
the defect detection simulation training sample set determining module is used for processing the simulation part sample model based on a simulation light source model and determining a defect detection simulation training sample set;
and the part defect detection model determining module is used for performing model training according to the defect detection simulation training sample set, determining a part defect detection model and detecting the defects of the part real object.
10. A part defect inspection apparatus, comprising: one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a part defect detection method as claimed in any one of claims 1-8.
11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for defect detection of a part according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010234961.4A CN111428374A (en) | 2020-03-30 | 2020-03-30 | Part defect detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010234961.4A CN111428374A (en) | 2020-03-30 | 2020-03-30 | Part defect detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111428374A true CN111428374A (en) | 2020-07-17 |
Family
ID=71555537
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010234961.4A Pending CN111428374A (en) | 2020-03-30 | 2020-03-30 | Part defect detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111428374A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112465825A (en) * | 2021-02-02 | 2021-03-09 | 聚时科技(江苏)有限公司 | Method for acquiring spatial position information of part based on image processing |
| CN112485260A (en) * | 2020-11-26 | 2021-03-12 | 常州微亿智造科技有限公司 | Workpiece defect detection method and device |
| CN113034114A (en) * | 2021-04-01 | 2021-06-25 | 苏州惟信易量智能科技有限公司 | Flow control system and method based on wearable device |
| CN113052561A (en) * | 2021-04-01 | 2021-06-29 | 苏州惟信易量智能科技有限公司 | Flow control system and method based on wearable device |
| CN113610792A (en) * | 2021-07-30 | 2021-11-05 | 杭州申昊科技股份有限公司 | Track fastener detection method, device and readable storage medium |
| CN114035013A (en) * | 2021-10-19 | 2022-02-11 | 中国电子产品可靠性与环境试验研究所（（工业和信息化部电子第五研究所）（中国赛宝实验室）） | Defect diagnosis method and defect diagnosis device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030182251A1 (en) * | 2002-03-22 | 2003-09-25 | Donglok Kim | Accelerated learning in machine vision using artificially implanted defects |
| CN108291878A (en) * | 2015-11-17 | 2018-07-17 | 科磊股份有限公司 | Single image detects |
| CN109596638A (en) * | 2018-10-26 | 2019-04-09 | 中国科学院光电研究院 | There are the defect inspection method and device of figure wafer and mask |
| CN110458778A (en) * | 2019-08-08 | 2019-11-15 | 深圳市灵明光子科技有限公司 | A kind of depth image denoising method, device and storage medium |
-
2020
- 2020-03-30 CN CN202010234961.4A patent/CN111428374A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030182251A1 (en) * | 2002-03-22 | 2003-09-25 | Donglok Kim | Accelerated learning in machine vision using artificially implanted defects |
| CN108291878A (en) * | 2015-11-17 | 2018-07-17 | 科磊股份有限公司 | Single image detects |
| CN109596638A (en) * | 2018-10-26 | 2019-04-09 | 中国科学院光电研究院 | There are the defect inspection method and device of figure wafer and mask |
| CN110458778A (en) * | 2019-08-08 | 2019-11-15 | 深圳市灵明光子科技有限公司 | A kind of depth image denoising method, device and storage medium |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112485260A (en) * | 2020-11-26 | 2021-03-12 | 常州微亿智造科技有限公司 | Workpiece defect detection method and device |
| CN112485260B (en) * | 2020-11-26 | 2023-01-03 | 常州微亿智造科技有限公司 | Workpiece defect detection method and device |
| CN112465825A (en) * | 2021-02-02 | 2021-03-09 | 聚时科技(江苏)有限公司 | Method for acquiring spatial position information of part based on image processing |
| CN113034114A (en) * | 2021-04-01 | 2021-06-25 | 苏州惟信易量智能科技有限公司 | Flow control system and method based on wearable device |
| CN113052561A (en) * | 2021-04-01 | 2021-06-29 | 苏州惟信易量智能科技有限公司 | Flow control system and method based on wearable device |
| CN113610792A (en) * | 2021-07-30 | 2021-11-05 | 杭州申昊科技股份有限公司 | Track fastener detection method, device and readable storage medium |
| CN114035013A (en) * | 2021-10-19 | 2022-02-11 | 中国电子产品可靠性与环境试验研究所（（工业和信息化部电子第五研究所）（中国赛宝实验室）） | Defect diagnosis method and defect diagnosis device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111428373A (en) | Product assembly quality detection method, device, equipment and storage medium | |
| CN111428374A (en) | Part defect detection method, device, equipment and storage medium | |
| CN111078908B (en) | Method and device for detecting data annotation | |
| CN109871895B (en) | Method and device for detecting defects of circuit board | |
| CN109977191B (en) | Problem map detection method, device, electronic equipment and medium | |
| JP6868119B2 (en) | Holographic anti-counterfeit code inspection method and equipment | |
| CN108010118B (en) | Virtual object processing method, virtual object processing apparatus, medium, and computing device | |
| CN111986159B (en) | Electrode defect detection method and device for solar cell and storage medium | |
| CN113532882A (en) | Automobile instrument testing method, device and system and storage medium | |
| WO2024002187A1 (en) | Defect detection method, defect detection device, and storage medium | |
| CN115937147B (en) | Defect detection parameter determining method, device, equipment and storage medium | |
| CN112559341A (en) | Picture testing method, device, equipment and storage medium | |
| CN113989616A (en) | Target detection method, device, equipment and storage medium | |
| CN109840212B (en) | Function test method, device and equipment of application program and readable storage medium | |
| CN113936232A (en) | Screen fragmentation identification method, device, equipment and storage medium | |
| CN114359161A (en) | Defect detection method, device, equipment and storage medium | |
| CN113744252A (en) | Method, apparatus, storage medium and program product for marking and detecting defects | |
| CN116721104A (en) | Live three-dimensional model defect detection method and device, electronic equipment and storage medium | |
| CN116245808A (en) | Workpiece defect detection method and device, electronic equipment and storage medium | |
| CN111488846A (en) | Method and equipment for identifying water level | |
| CN113971650A (en) | Product flaw detection method, computer device and storage medium | |
| CN115908977A (en) | Image data labeling method and device, electronic equipment and storage medium | |
| CN114841255A (en) | Detection model training method, device, equipment, storage medium and program product | |
| CN114565780A (en) | Target identification method and device, electronic equipment and storage medium | |
| CN111124862B (en) | Intelligent device performance testing method and device and intelligent device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200717 |