CN110136130A - A kind of method and device of testing product defect - Google Patents
A kind of method and device of testing product defect Download PDFInfo
- Publication number
- CN110136130A CN110136130A CN201910431943.2A CN201910431943A CN110136130A CN 110136130 A CN110136130 A CN 110136130A CN 201910431943 A CN201910431943 A CN 201910431943A CN 110136130 A CN110136130 A CN 110136130A
- Authority
- CN
- China
- Prior art keywords
- gray level
- sample
- level image
- defect
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
This application provides a kind of method and devices of testing product defect, comprising: the depth image is converted to plane gray level image by the depth image for obtaining product to be detected；The plane gray level image is input to the deep learning network constructed in advance, obtains the defects detection result of the product to be detected.The detection efficiency of product defects can be promoted.
Description
Technical field
This application involves detection technique fields, in particular to a kind of method and device of testing product defect.
Background technique
In industrial processes, due to the external environments such as mechanical shock, sound and light and complexity production technology,
It may cause that the product appearance produced is bad, so that the product produced be made to become faulty goods, reduce production efficiency.For
It ensures that the appearance of product meets corresponding production requirement, needs to carry out a series of detections to the product in industrial processes.
At present in product testing, the vision-based detection of product defects is generally carried out using industrial camera, based on to product table
The two dimensional image that face is shot is analyzed.But the product defects detection method is needed for every a kind of defect, artificial to design
For the visual detection algorithm of the defects detection, the corresponding detective operators of visual detection algorithm, which need to adjust quantity of parameters, can just make
It detects stable, it is then desired to consume the plenty of time, and product defects that can be detected are more single, cannot detect to remove
Product defects except each visual detection algorithm of design, so that the detection efficiency of product defects is not high.
Summary of the invention
In view of this, a kind of method and device for being designed to provide testing product defect of the application, promotes product and lacks
Sunken detection efficiency.
In a first aspect, the embodiment of the present application provides a kind of method of testing product defect, comprising:
The depth image is converted to plane gray level image by the depth image for obtaining product to be detected；
The plane gray level image is input to the deep learning network constructed in advance, obtains lacking for the product to be detected
Fall into testing result, wherein construct the deep learning network as follows；
The sample depth image for obtaining sample, is converted to sample plane gray level image for the sample depth image；
According to the defect characteristic that the sample includes, the corresponding sample plane gray level image of the sample is labeled,
According to the sample plane gray level image of mark, the sample plane gray level image of the mark is divided into training set and test
Collection；
The defect area of the sample plane gray level image marked using in the training set trains network as deep learning
Input, the output of training network using the corresponding defect type of the defect area as the deep learning, to the deep learning
Training network is trained；
Deep learning training of the defect area of the sample plane gray level image marked using in the test set as training
The input of network obtains the output of deep learning training network of the training as a result, with the corresponding defect of the defect area
Type is compared, and obtains the neural network accuracy of the deep learning training network of the training；
If the neural network accuracy is less than pre-set precision threshold, according to the training set to the depth of the training
It practises training network to continue to train, until the neural network accuracy of the deep learning training network of the training is not less than the precision threshold
Value, obtains the deep learning network.
Optionally, the defect for including according to the sample, sample plane gray level image mark corresponding to the sample
Note, comprising:
The defect area that the sample includes is analyzed, the defect characteristic of the defect area is extracted, is lacked according to what is constructed in advance
The corresponding relationship for falling into type and defect characteristic, determines the corresponding defect type of each defect characteristic；
In the defect type mark that the defect area of the corresponding sample plane gray level image of the sample is determined.
Optionally, the sample plane gray level image according to mark draws the sample plane gray level image of the mark
It is divided into training set and test set, comprising:
For the sample plane gray level image of each mark, the defect type of the sample plane gray level image of the mark is counted
Quantity, if the defect type quantity of statistics is more than preset defect type number threshold value, by the sample plane gray level image of the mark
It is placed in training set, otherwise, the sample plane gray level image of the mark is placed in test set.
Optionally, also include flawless sample plane gray level image in the training set, also wrapped in the test set
Contain flawless sample plane gray level image.
Optionally, after the sample depth image is converted to sample plane gray level image, according to the sample packet
The defect contained, before being labeled to the corresponding sample plane gray level image of the sample, the method also includes:
The sample plane gray level image is pre-processed.
Optionally, the pretreatment includes: that noise suppressed pretreatment, morphological image pretreatment and rotation transformation are located in advance
Reason.
Optionally, after being labeled to the corresponding sample plane gray level image of the sample, according to the sample of mark
Plane gray level image, before the sample plane gray level image of the mark is divided into training set and test set, the method
Further include:
The all defect region area that the sample plane gray level image includes is counted, all defect area surface is calculated
The long-pending ratio with the area of the sample plane gray level image；
If the ratio is less than pre-set proportion threshold value, the sample plane gray level image is split, is deleted
The segmented image not comprising defect area of segmentation.
It optionally, include integer defect area in each point image divided.
Second aspect, the embodiment of the present application provide a kind of device of testing product defect, comprising:
Depth image obtains module and is converted to the depth image flat for obtaining the depth image of product to be detected
Face gray level image；
Defects detection module is obtained for the plane gray level image to be input to the deep learning network constructed in advance
The defects detection result of the product to be detected, wherein construct the deep learning network as follows；
The sample depth image for obtaining sample, is converted to sample plane gray level image for the sample depth image；
According to the defect characteristic that the sample includes, the corresponding sample plane gray level image of the sample is labeled,
According to the sample plane gray level image of mark, the sample plane gray level image of the mark is divided into training set and test
Collection；
The defect area of the sample plane gray level image marked using in the training set trains network as deep learning
Input, the output of training network using the corresponding defect type of the defect area as the deep learning, to the deep learning
Training network is trained；
Deep learning training of the defect area of the sample plane gray level image marked using in the test set as training
The input of network obtains the output of deep learning training network of the training as a result, with the corresponding defect of the defect area
Type is compared, and obtains the neural network accuracy of the deep learning training network of the training；
If the neural network accuracy is less than pre-set precision threshold, according to the training set to the depth of the training
It practises training network to continue to train, until the neural network accuracy of the deep learning training network of the training is not less than the precision threshold
Value, obtains the deep learning network.
Optionally, described device further include:
Display module, for showing the plane gray level image for being labeled with the defects detection result on a display screen.
The third aspect, the embodiment of the present application provide a kind of computer equipment, including memory, processor and are stored in institute
The computer program that can be run on memory and on the processor is stated, the processor executes real when the computer program
The step of existing above method.
Fourth aspect, the embodiment of the present application provide a kind of computer readable storage medium, the computer-readable storage
Computer program is stored on medium, the computer program executes above-mentioned method when being run by processor the step of.
The method and device of a kind of testing product defect provided by the embodiments of the present application, by the depth for obtaining product to be detected
Image is spent, the depth image is converted into plane gray level image；The plane gray level image is input to the depth constructed in advance
Learning network is spent, obtains the defects detection result of the product to be detected, wherein construct the deep learning as follows
Network；The sample depth image for obtaining sample, is converted to sample plane gray level image for the sample depth image；According to described in
The defect characteristic that sample includes is labeled the corresponding sample plane gray level image of the sample, flat according to the sample of mark
The sample plane gray level image of the mark is divided into training set and test set by face gray level image；In the training set
Input of the defect area of the sample plane gray level image of mark as deep learning training network, it is corresponding with the defect area
Output of the defect type as deep learning training network is trained deep learning training network；With described
Input of the defect area of the sample plane gray level image marked in test set as the deep learning training network of training, obtains
The deep learning of the training trains the output of network as a result, being compared with the corresponding defect type of the defect area, obtains
Take the neural network accuracy of the deep learning training network of the training；If the neural network accuracy is less than pre-set precision threshold,
Continue to train according to deep learning training network of the training set to the training, until the deep learning training of the training
The neural network accuracy of network is not less than the precision threshold, obtains the deep learning network.In this way, by by the depth map of product
As being converted to plane gray level image, defects detection is carried out to plane gray level image using trained deep learning network, it can be with
It effectively avoids carrying out the detection of product defects caused by defects detection according to two dimensional image and the visual detection algorithm artificially designed
Single technical problem promotes the detection efficiency of product defects.
To enable the above objects, features, and advantages of the application to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate
Appended attached drawing, is described in detail below.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only some embodiments of the application, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 is a kind of method flow schematic diagram of testing product defect provided by the embodiments of the present application；
Fig. 2 is the method flow schematic diagram of building deep learning network provided by the embodiments of the present application；
Fig. 3 is a kind of apparatus structure schematic diagram of testing product defect provided by the embodiments of the present application；
Fig. 4 is a kind of structural schematic diagram of computer equipment 400 provided by the embodiments of the present application.
Specific embodiment
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application
Middle attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only
It is some embodiments of the present application, instead of all the embodiments.The application being usually described and illustrated herein in the accompanying drawings is real
The component for applying example can be arranged and be designed with a variety of different configurations.Therefore, below to the application's provided in the accompanying drawings
The detailed description of embodiment is not intended to limit claimed scope of the present application, but is merely representative of the selected reality of the application
Apply example.Based on embodiments herein, those skilled in the art institute obtained without making creative work
There are other embodiments, shall fall in the protection scope of this application.
Fig. 1 is a kind of method flow schematic diagram of testing product defect provided by the embodiments of the present application.As shown in Figure 1, should
Method includes:
Step 101, the depth image is converted to plane gray level image by the depth image for obtaining product to be detected；
In the embodiment of the present application, as an alternative embodiment, acquires and obtain to be detected using 3D sensor or 3D camera
The depth image of product.
In the embodiment of the present application, what industrial camera obtained is the information such as product surface color, gray scale, brightness, but due to producing
Product are three-dimension objects, are not in the same plane, thus, by the color, gray scale, luminance information of product plane, to be difficult
The depth information of product is obtained, and then can not detect the defect that product is formed in difference in height direction.In the embodiment of the present application,
On the basis of industrial camera, 3D camera is formed by increasing laser and infrared sensor, utilizes the monochromaticjty combination triangle of laser
Measuring principle, the point cloud of available product, the i.e. depth image of product, and image procossing is carried out based on depth image, thus
It realizes to the defects detection on product depth direction, has opened up new answer for the detection of 3D defective vision and deep learning defects detection
With and expand, provide new solution for industrial defects detection.
In the embodiment of the present application, after the depth image (point cloud) for obtaining product, depth image (point cloud) is converted to two
The plane gray level image of dimension, which includes depth information, obtains product surface luminance information with industrial camera
Two dimensional image be different image.
It, can be by the depth information of the depth image of product according to pre- as an alternative embodiment in the embodiment of the present application
The mapping relations being first arranged are mapped as gray value, to obtain plane gray level image.It is also possible in the depth image by product
The corresponding azimuth of every bit cloud be mapped as the gray value of corresponding pixel points, to obtain plane gray level image.Wherein, orientation
The vector and current point cloud to upper right side of current point cloud position are arrived in position (viewpoint) when angle is 3D sensor or 3D camera scanning
The angle of the vector of neighbor point cloud.
Step 102, the plane gray level image is input to the deep learning network constructed in advance, is obtained described to be detected
The defects detection result of product.
In the embodiment of the present application, defects detection result includes: with defect and zero defect.For having defective detection
As a result, can also include: each specific defect type and the corresponding specific location of each defect type.In this way, convenient for related skill
There is the reason of defect according to specific defect type and the corresponding specific location analysis of each defect type and carry out in art personnel
It improves.
In the embodiment of the present application, as an alternative embodiment, this method can also include:
The plane gray level image for being labeled with the defects detection result is shown on a display screen.
Fig. 2 is the method flow schematic diagram of building deep learning network provided by the embodiments of the present application.As shown in Fig. 2, should
Method includes:
Step 201, the sample depth image is converted to sample plane grayscale image by the sample depth image for obtaining sample
Picture；
In the embodiment of the present application, sample includes: defective sample and zero defect sample.It is adopted using 3D sensor or 3D camera
Collect the depth image of sample.
Step 202, the defect characteristic for including according to the sample, to the corresponding sample plane gray level image of the sample into
Rower note, according to the sample plane gray level image of mark, by the sample plane gray level image of the mark be divided into training set with
And test set；
In the embodiment of the present application, sample plane gray level image is labeled and is divided.
It is corresponding to the sample according to the defect that the sample includes as an alternative embodiment in the embodiment of the present application
Sample plane gray level image mark, comprising:
A11 analyzes the defect area that the sample includes, and extracts the defect characteristic of the defect area, according to building in advance
Defect type and defect characteristic corresponding relationship, determine the corresponding defect type of each defect characteristic；
In the embodiment of the present application, a sample may include one or more defect areas, each defect area corresponding one
Defect characteristic.As an alternative embodiment, manual analysis is carried out to the defect area of sample, so that it is determined that going out the defect area
Defect type.Wherein, defect type, which can be, largely has defective product by analysis, is sorted out to defect.
In the embodiment of the present application, each sample depth image that can be will acquire is labeled, and marks sample depth figure
The defect area as present in.As an alternative embodiment, it is corresponding that defect area can also be marked out in sample depth image
Defect type.
In the embodiment of the present application, as an alternative embodiment, preset complementary annotation tool can be used, to defect area
The defect type of domain and defect area is marked.
A12, in the defect type mark that the defect area of the corresponding sample plane gray level image of the sample is determined.
In the embodiment of the present application, as an alternative embodiment, in each defect area of sample, corresponding defect class is carried out respectively
The mark of type.
In the embodiment of the present application, as an alternative embodiment, the sample plane gray level image of defect type mark will be carried out
Classification are as follows: training set and test set.It include to have defective sample plane gray level image and flawless sample in training set
Plane gray level image also includes to have defective sample plane gray level image and flawless sample plane ash in test set
Spend image.In this way, in training set and test set, comprising with defect and without the sample plane gray level image of defect, so that
When carrying out deep learning training, it can effectively inhibit deep learning training network over-fitting and lead to the reduction of defects detection effect,
Improve the defects detection accuracy of the deep learning network generated.
In the embodiment of the present application, as an alternative embodiment, the sample plane gray level image quantity in training set be can be
N times of sample plane gray level image quantity in test set, wherein N is greater than or equal to 3 integer.Training set and test set
In, defective sample plane gray level image quantity is identical as flawless sample plane gray level image quantitative proportion or connects
Closely.
In the embodiment of the present application, as an alternative embodiment, according to the sample plane gray level image of mark, by the mark
Sample plane gray level image be divided into training set and test set, comprising:
For the sample plane gray level image of each mark, the defect type of the sample plane gray level image of the mark is counted
Quantity, if the defect type quantity of statistics is more than preset defect type number threshold value, by the sample plane gray level image of the mark
It is placed in training set, otherwise, the sample plane gray level image of the mark is placed in test set.
In the embodiment of the present application, a fairly large number of sample plane gray level image of defect type is put into training set, in this way, can
To effectively reduce the required gray level image quantity of training, the data processing amount being trained is reduced, it can be with training for promotion efficiency.
In the embodiment of the present application, since the factors such as external environmental interference influence, the plane obtained after the conversion of point cloud chart picture is grey
The problems such as degree image may show brightness disproportionation, rotation, high noise.In order to promote the quality of plane gray level image, by institute
It states after sample depth image is converted to sample plane gray level image, according to the defect that the sample includes, to the sample pair
Before the sample plane gray level image answered is labeled, this method further include:
The sample plane gray level image is pre-processed.
In the embodiment of the present application, as an alternative embodiment, pretreatment includes but is not limited to: noise suppressed pretreatment is used
To improve image displaying quality, morphological image pretreatment, to fill and remove the burr in image, rotation transformation is located in advance
Reason, to inhibit anamorphose caused by rotating as object.In this way, by being pre-processed to plane gray level image, it can
Effectively overcome the image-quality problems of image itself, enhances the image effect of plane gray level image, successive depths can be improved
Practise the defect detection ability of network.
In the embodiment of the present application, as an alternative embodiment, when the ratio that the size of defect area occupies in whole image
When example is less than pre-set proportion threshold value, in order to further enhance image processing efficiency, to the corresponding sample of the sample
After plane gray level image is labeled, according to the sample plane gray level image of mark, by the sample plane gray scale of the mark
Image is divided into before training set and test set, and this method can also include:
The all defect region area that the sample plane gray level image includes is counted, all defect area surface is calculated
The long-pending ratio with the area of the sample plane gray level image；
If the ratio is less than pre-set proportion threshold value, the sample plane gray level image is split, is deleted
The segmented image not comprising defect area of segmentation.
In the embodiment of the present application, by modifying image segmentation parameter, do not need to carry out image procossing to delete or cut off
Part, can effectively reduce and subsequent be trained required network iteration time.
In the embodiment of the present application, as an alternative embodiment, in a image of each point divided, lacked comprising integer
Fall into region.
Step 203, the defect area of the sample plane gray level image marked using in the training set is instructed as deep learning
The input for practicing network, the output of training network using the corresponding defect type of the defect area as the deep learning, to described
Deep learning training network is trained；
In the embodiment of the present application, the training parameter of deep learning training network is preset, including but not limited to: network changes
Generation number, network number of plies etc..For example, when the sample plane gray level image quantity of input is greater than pre-set amount threshold,
Meeting is so that the image data amount of input is greater than physical memory size, it is then desired to which the plane gray level image to input carries out network
The number of iterations setting.
In the embodiment of the present application, as an alternative embodiment, deep learning training network is using there is enforcement mechanisms, i.e., to depth
The defects of degree image region is labeled, to mark the pixel of the defect area.
In the embodiment of the present application, as an alternative embodiment, deep learning training network be convolutional neural networks (CNN,
Convolutional Neural Network), convolutional neural networks include: data input layer (Input Layer), convolution meter
Calculate layer (CONV Layer), ReLU excitation layer (ReLU Layer), pond layer (Pooling Layer), full articulamentum (FC
Layer).Wherein,
Data input layer for the image data of input to be normalized, by gray value of image scope limitation 0 to 1 it
Between, for larger-size image, it is also an option that principal component analysis (PCA, Principal Component Analysis)
Operator carries out dimensionality reduction to the image data of input, by setting the parameter of data input layer, advantageously reduces the number of image pixel
Value.
Convolutional calculation layer is one layer important in CNN, for utilizing window sliding, is calculated, is extracted for local data
Characteristic image in image out.Wherein, by the way that convolution kernel size N and step-length L is arranged, the big of the characteristic image of extraction can be determined
It is small.
ReLU excitation layer is used for the excitation of single neuron node, using activation primitive (Sigmoid), completes output result
Nonlinear Mapping.Wherein, activation primitive selection determines the over-fitting degree of deep learning training network and the deep learning of generation
The quality of network can be improved network training model accuracy by the way that suitable activation primitive is arranged.
Pond layer is complicated with the calculating for reducing deep learning training network for compressing to the characteristic image of input
Degree.
Full articulamentum gives output valve to classifier for connecting all features.
Step 204, depth of the defect area of the sample plane gray level image marked using in the test set as training
The input of learning training network obtains the output of deep learning training network of the training as a result, with the defect area pair
The defect type answered is compared, and obtains the neural network accuracy of the deep learning training network of the training；
Step 205, if the neural network accuracy is less than pre-set precision threshold, according to the training set to the training
Deep learning training network continue to train, until the neural network accuracy of the deep learning training network of the training is not less than described
Precision threshold obtains the deep learning network.
In the embodiment of the present application, by the deep learning training net of the sample plane gray level image input training in test set
Network, the defects detection exported is as a result, i.e. output is as a result, by the sample plane gray scale in the defects detection result and test set
The defect type of the corresponding mark of image is compared, and is referred to the digitlization that quantum chemical method goes out deep learning training network detection defect
Mark, for example, recall rate.Deep learning training is determined to evaluate the performance of deep learning training network by digitalization index
Whether network can satisfy preset Testing index (precision threshold), if satisfied, then showing to train completion, obtain scarce for carrying out
Fall into the deep learning network of detection.
In the embodiment of the present application, deep learning training network is that the profound of artificial neural network is used, flat by collecting
Face greyscale image data carries out abstract characteristics extraction to data, instructs according to the feature extracted to deep learning training network
Practice, so that suitable deep learning network is obtained, so that there is autonomous developmental capacity in face of complicated industrial defects detection, it can
Faulty goods is detected to stablize, promotes defects detection efficiency, guarantees product quality.
Fig. 3 is a kind of apparatus structure schematic diagram of testing product defect provided by the embodiments of the present application.As shown in figure 3, should
Device includes:
Depth image obtains module 301 and is converted to the depth image for obtaining the depth image of product to be detected
Plane gray level image；
In the embodiment of the present application, is acquired using 3D sensor or 3D camera and obtain the depth image of product to be detected.
It, can be by the depth information of the depth image of product according to pre- as an alternative embodiment in the embodiment of the present application
The mapping relations being first arranged are mapped as gray value, to obtain plane gray level image.It is also possible in the depth image by product
The corresponding azimuth of every bit cloud be mapped as the gray value of corresponding pixel points, to obtain plane gray level image.
Defects detection module 302 is obtained for the plane gray level image to be input to the deep learning network constructed in advance
To the defects detection result of the product to be detected, wherein construct the deep learning network as follows；
The sample depth image for obtaining sample, is converted to sample plane gray level image for the sample depth image；
According to the defect characteristic that the sample includes, the corresponding sample plane gray level image of the sample is labeled,
According to the sample plane gray level image of mark, the sample plane gray level image of the mark is divided into training set and test
Collection；
The defect area of the sample plane gray level image marked using in the training set trains network as deep learning
Input, the output of training network using the corresponding defect type of the defect area as the deep learning, to the deep learning
Training network is trained；
Deep learning training of the defect area of the sample plane gray level image marked using in the test set as training
The input of network obtains the output of deep learning training network of the training as a result, with the corresponding defect of the defect area
Type is compared, and obtains the neural network accuracy of the deep learning training network of the training；
If the neural network accuracy is less than pre-set precision threshold, according to the training set to the depth of the training
It practises training network to continue to train, until the neural network accuracy of the deep learning training network of the training is not less than the precision threshold
Value, obtains the deep learning network.
In the embodiment of the present application, as an alternative embodiment, the device further include:
Display module (not shown) is labeled with the described flat of the defects detection result for showing on a display screen
Face gray level image.
In the embodiment of the present application, defects detection result includes: with defect and zero defect.For having defective detection
As a result, can also include: each specific defect type and the corresponding specific location of each defect type.
In the embodiment of the present application, as an alternative embodiment, the defect for including according to the sample, to the sample
Corresponding sample plane gray level image mark, comprising:
The defect area that the sample includes is analyzed, the defect characteristic of the defect area is extracted, is lacked according to what is constructed in advance
The corresponding relationship for falling into type and defect characteristic, determines the corresponding defect type of each defect characteristic；
In the defect type mark that the defect area of the corresponding sample plane gray level image of the sample is determined.
In the embodiment of the present application, as an alternative embodiment, the sample plane gray level image according to mark will be described
The sample plane gray level image of mark is divided into training set and test set, comprising:
For the sample plane gray level image of each mark, the defect type of the sample plane gray level image of the mark is counted
Quantity, if the defect type quantity of statistics is more than preset defect type number threshold value, by the sample plane gray level image of the mark
It is placed in training set, otherwise, the sample plane gray level image of the mark is placed in test set.
It also include flawless sample plane gray scale as an alternative embodiment, in training set in the embodiment of the present application
Image also includes flawless sample plane gray level image in test set.
In the embodiment of the present application, as an alternative embodiment, defects detection module 302 is also used to:
The sample plane gray level image is pre-processed.
In the embodiment of the present application, pretreatment includes: noise suppressed pretreatment, morphological image pretreatment and rotation transformation
Pretreatment.
In the embodiment of the present application, as another alternative embodiment, defects detection module 302 is also used to:
The all defect region area that the sample plane gray level image includes is counted, all defect area surface is calculated
The long-pending ratio with the area of the sample plane gray level image；
If the ratio is less than pre-set proportion threshold value, the sample plane gray level image is split, is deleted
The segmented image not comprising defect area of segmentation.
It include integer defect area in a image of each point divided in the embodiment of the present application.
As shown in figure 4, one embodiment of the application provides a kind of computer equipment 400, produced for executing the detection in Fig. 1
The method of product defect, the equipment include memory 401, processor 402 and are stored on the memory 401 and can be in the processor
The computer program run on 402, wherein above-mentioned processor 402 realizes above-mentioned testing product when executing above-mentioned computer program
The step of method of defect.
Specifically, above-mentioned memory 401 and processor 402 can be general memory and processor, do not do have here
Body limits, and when the computer program of 402 run memory 401 of processor storage, is able to carry out above-mentioned testing product defect
Method.
Corresponding to the method for the testing product defect in Fig. 1, computer-readable deposited the embodiment of the present application also provides a kind of
Storage media is stored with computer program on the computer readable storage medium, execution when which is run by processor
The step of method of above-mentioned testing product defect.
Specifically, which can be general storage medium, such as mobile disk, hard disk, on the storage medium
Computer program when being run, the method for being able to carry out above-mentioned testing product defect.
In embodiment provided herein, it should be understood that disclosed device and method, it can be by others side
Formula is realized.The apparatus embodiments described above are merely exemplary, for example, the division of the unit, only one kind are patrolled
Function division is collected, there may be another division manner in actual implementation, in another example, multiple units or components can combine or can
To be integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Coupling, direct-coupling or communication connection can be INDIRECT COUPLING or communication link by some communication interfaces, device or unit
It connects, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
In addition, each functional unit in embodiment provided by the present application can integrate in one processing unit, it can also
To be that each unit physically exists alone, can also be integrated in one unit with two or more units.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product
It is stored in a computer readable storage medium.Based on this understanding, the technical solution of the application is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, server or network equipment etc.) execute each embodiment the method for the application all or part of the steps.
And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), arbitrary access are deposited
The various media that can store program code such as reservoir (Random Access Memory, RAM), magnetic or disk.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing, in addition, term " the
One ", " second ", " third " etc. are only used for distinguishing description, are not understood to indicate or imply relative importance.
Finally, it should be noted that embodiment described above, the only specific embodiment of the application, to illustrate the application
Technical solution, rather than its limitations, the protection scope of the application is not limited thereto, although with reference to the foregoing embodiments to this Shen
It please be described in detail, those skilled in the art should understand that: anyone skilled in the art
Within the technical scope of the present application, it can still modify to technical solution documented by previous embodiment or can be light
It is readily conceivable that variation or equivalent replacement of some of the technical features；And these modifications, variation or replacement, do not make
The essence of corresponding technical solution is detached from the spirit and scope of the embodiment of the present application technical solution.The protection in the application should all be covered
Within the scope of.Therefore, the protection scope of the application shall be subject to the protection scope of the claim.
Claims (10)
1. a kind of method of testing product defect characterized by comprising
The depth image is converted to plane gray level image by the depth image for obtaining product to be detected；
The plane gray level image is input to the deep learning network constructed in advance, obtains the defect inspection of the product to be detected
Survey result, wherein construct the deep learning network as follows；
The sample depth image for obtaining sample, is converted to sample plane gray level image for the sample depth image；
According to the defect characteristic that the sample includes, the corresponding sample plane gray level image of the sample is labeled, according to
The sample plane gray level image of the mark is divided into training set and test set by the sample plane gray level image of mark；
The defect area of the sample plane gray level image marked using in the training set trains the input of network as deep learning,
The output of training network using the corresponding defect type of the defect area as the deep learning, to the deep learning training net
Network is trained；
Deep learning training network of the defect area of the sample plane gray level image marked using in the test set as training
Input, obtain the output of deep learning training network of the training as a result, with the corresponding defect type of the defect area
It is compared, obtains the neural network accuracy of the deep learning training network of the training；
If the neural network accuracy is less than pre-set precision threshold, instructed according to deep learning of the training set to the training
Practice network to continue to train, until the neural network accuracy of the deep learning training network of the training is not less than the precision threshold, obtain
To the deep learning network.
2. the method as described in claim 1, which is characterized in that the defect for including according to the sample, to the sample
Corresponding sample plane gray level image mark, comprising:
The defect area that the sample includes is analyzed, the defect characteristic of the defect area is extracted, according to the defect class constructed in advance
The corresponding relationship of type and defect characteristic determines the corresponding defect type of each defect characteristic；
In the defect type mark that the defect area of the corresponding sample plane gray level image of the sample is determined.
3. the method as described in claim 1, which is characterized in that the sample plane gray level image according to mark, it will be described
The sample plane gray level image of mark is divided into training set and test set, comprising:
For the sample plane gray level image of each mark, the defect type number of the sample plane gray level image of the mark is counted
Amount sets the sample plane gray level image of the mark if the defect type quantity of statistics is more than preset defect type number threshold value
In training set, otherwise, the sample plane gray level image of the mark is placed in test set.
4. method as claimed in claim 3, which is characterized in that also include flawless sample plane ash in the training set
Image is spent, also includes flawless sample plane gray level image in the test set.
5. the method as described in claim 1, which is characterized in that the sample depth image is being converted to sample plane gray scale
After image, according to the defect that the sample includes, before being labeled to the corresponding sample plane gray level image of the sample,
The method also includes:
The sample plane gray level image is pre-processed.
6. method as claimed in claim 5, which is characterized in that the pretreatment includes: noise suppressed pretreatment, image aspects
Learn pretreatment and rotation transformation pretreatment.
7. such as method as claimed in any one of claims 1 to 6, which is characterized in that the sample corresponding sample plane ash
After degree image is labeled, according to the sample plane gray level image of mark, the sample plane gray level image of the mark is drawn
It is divided into before training set and test set, the method also includes:
Count all defect region area that the sample plane gray level image includes, calculate all defect region area with
The ratio of the area of the sample plane gray level image；
If the ratio is less than pre-set proportion threshold value, the sample plane gray level image is split, deletes segmentation
The segmented image not comprising defect area.
8. the method for claim 7, which is characterized in that in each point image divided, lacked comprising integer
Fall into region.
9. a kind of device of testing product defect characterized by comprising
Depth image obtains module, and for obtaining the depth image of product to be detected, the depth image is converted to plane ash
Spend image；
Defects detection module obtains described for the plane gray level image to be input to the deep learning network constructed in advance
The defects detection result of product to be detected, wherein construct the deep learning network as follows；
The sample depth image for obtaining sample, is converted to sample plane gray level image for the sample depth image；
According to the defect characteristic that the sample includes, the corresponding sample plane gray level image of the sample is labeled, according to
The sample plane gray level image of the mark is divided into training set and test set by the sample plane gray level image of mark；
The defect area of the sample plane gray level image marked using in the training set trains the input of network as deep learning,
The output of training network using the corresponding defect type of the defect area as the deep learning, to the deep learning training net
Network is trained；
Deep learning training network of the defect area of the sample plane gray level image marked using in the test set as training
Input, obtain the output of deep learning training network of the training as a result, with the corresponding defect type of the defect area
It is compared, obtains the neural network accuracy of the deep learning training network of the training；
If the neural network accuracy is less than pre-set precision threshold, instructed according to deep learning of the training set to the training
Practice network to continue to train, until the neural network accuracy of the deep learning training network of the training is not less than the precision threshold, obtain
To the deep learning network.
10. device as claimed in claim 9, which is characterized in that described device further include:
Display module, for showing the plane gray level image for being labeled with the defects detection result on a display screen.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910431943.2A CN110136130A (en) | 2019-05-23 | 2019-05-23 | A kind of method and device of testing product defect |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910431943.2A CN110136130A (en) | 2019-05-23 | 2019-05-23 | A kind of method and device of testing product defect |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110136130A true CN110136130A (en) | 2019-08-16 |
Family
ID=67572663
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910431943.2A Pending CN110136130A (en) | 2019-05-23 | 2019-05-23 | A kind of method and device of testing product defect |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110136130A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110648323A (en) * | 2019-09-26 | 2020-01-03 | 上海御微半导体技术有限公司 | Defect detection classification system and method thereof |
| CN110672601A (en) * | 2019-09-06 | 2020-01-10 | 深圳新视智科技术有限公司 | Textile density detection method, device, terminal and storage medium |
| CN110852373A (en) * | 2019-11-08 | 2020-02-28 | 深圳市深视创新科技有限公司 | Defect-free sample deep learning network training method based on vision |
| CN111371965A (en) * | 2020-03-23 | 2020-07-03 | 华兴源创(成都)科技有限公司 | Image processing method, device and equipment for eliminating interference in brightness shooting of display screen |
| CN111598863A (en) * | 2020-05-13 | 2020-08-28 | 北京阿丘机器人科技有限公司 | Defect detection method, device, equipment and readable storage medium |
| CN113129264A (en) * | 2021-03-17 | 2021-07-16 | 联想(北京)有限公司 | Image processing method and device |
| CN113763355A (en) * | 2021-09-07 | 2021-12-07 | 创新奇智(青岛)科技有限公司 | Defect detection method and device, electronic equipment and storage medium |
| CN116721104A (en) * | 2023-08-10 | 2023-09-08 | 武汉大学 | Live three-dimensional model defect detection method and device, electronic equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105574905A (en) * | 2015-12-15 | 2016-05-11 | 大连理工大学 | Two-dimensional imagination expression method of three-dimensional laser-point cloud data |
| CN106295601A (en) * | 2016-08-18 | 2017-01-04 | 合肥工业大学 | A kind of Safe belt detection method of improvement |
| CN107423698A (en) * | 2017-07-14 | 2017-12-01 | 华中科技大学 | A kind of gesture method of estimation based on convolutional neural networks in parallel |
| CN108830837A (en) * | 2018-05-25 | 2018-11-16 | 北京百度网讯科技有限公司 | A kind of method and apparatus for detecting ladle corrosion defect |
| CN108921846A (en) * | 2018-07-17 | 2018-11-30 | 北京航空航天大学 | A kind of rail tread defect identification method combined based on gray level image and depth image |
| CN109165603A (en) * | 2018-08-28 | 2019-01-08 | 中国科学院遥感与数字地球研究所 | A kind of Ship Detection and device |
| CN109344793A (en) * | 2018-10-19 | 2019-02-15 | 北京百度网讯科技有限公司 | Aerial hand-written method, apparatus, equipment and computer readable storage medium for identification |
| CN109389599A (en) * | 2018-10-25 | 2019-02-26 | 北京阿丘机器人科技有限公司 | A kind of defect inspection method and device based on deep learning |
-
2019
- 2019-05-23 CN CN201910431943.2A patent/CN110136130A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105574905A (en) * | 2015-12-15 | 2016-05-11 | 大连理工大学 | Two-dimensional imagination expression method of three-dimensional laser-point cloud data |
| CN106295601A (en) * | 2016-08-18 | 2017-01-04 | 合肥工业大学 | A kind of Safe belt detection method of improvement |
| CN107423698A (en) * | 2017-07-14 | 2017-12-01 | 华中科技大学 | A kind of gesture method of estimation based on convolutional neural networks in parallel |
| CN108830837A (en) * | 2018-05-25 | 2018-11-16 | 北京百度网讯科技有限公司 | A kind of method and apparatus for detecting ladle corrosion defect |
| CN108921846A (en) * | 2018-07-17 | 2018-11-30 | 北京航空航天大学 | A kind of rail tread defect identification method combined based on gray level image and depth image |
| CN109165603A (en) * | 2018-08-28 | 2019-01-08 | 中国科学院遥感与数字地球研究所 | A kind of Ship Detection and device |
| CN109344793A (en) * | 2018-10-19 | 2019-02-15 | 北京百度网讯科技有限公司 | Aerial hand-written method, apparatus, equipment and computer readable storage medium for identification |
| CN109389599A (en) * | 2018-10-25 | 2019-02-26 | 北京阿丘机器人科技有限公司 | A kind of defect inspection method and device based on deep learning |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110672601A (en) * | 2019-09-06 | 2020-01-10 | 深圳新视智科技术有限公司 | Textile density detection method, device, terminal and storage medium |
| CN110648323A (en) * | 2019-09-26 | 2020-01-03 | 上海御微半导体技术有限公司 | Defect detection classification system and method thereof |
| CN110852373A (en) * | 2019-11-08 | 2020-02-28 | 深圳市深视创新科技有限公司 | Defect-free sample deep learning network training method based on vision |
| CN111371965A (en) * | 2020-03-23 | 2020-07-03 | 华兴源创(成都)科技有限公司 | Image processing method, device and equipment for eliminating interference in brightness shooting of display screen |
| CN111598863A (en) * | 2020-05-13 | 2020-08-28 | 北京阿丘机器人科技有限公司 | Defect detection method, device, equipment and readable storage medium |
| CN111598863B (en) * | 2020-05-13 | 2023-08-22 | 北京阿丘机器人科技有限公司 | Defect detection method, device, equipment and readable storage medium |
| CN113129264A (en) * | 2021-03-17 | 2021-07-16 | 联想(北京)有限公司 | Image processing method and device |
| CN113763355A (en) * | 2021-09-07 | 2021-12-07 | 创新奇智(青岛)科技有限公司 | Defect detection method and device, electronic equipment and storage medium |
| CN116721104A (en) * | 2023-08-10 | 2023-09-08 | 武汉大学 | Live three-dimensional model defect detection method and device, electronic equipment and storage medium |
| CN116721104B (en) * | 2023-08-10 | 2023-11-07 | 武汉大学 | Live three-dimensional model defect detection method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110136130A (en) | A kind of method and device of testing product defect | |
| CN111553929B (en) | Mobile phone screen defect segmentation method, device and equipment based on converged network | |
| CN113160192B (en) | Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background | |
| CN108830285B (en) | Target detection method for reinforcement learning based on fast-RCNN | |
| CN109584227A (en) | A kind of quality of welding spot detection method and its realization system based on deep learning algorithm of target detection | |
| CN104573669B (en) | Image object detection method | |
| CN107123111B (en) | Deep residual error network construction method for mobile phone screen defect detection | |
| CN103530590B (en) | DPM Quick Response Code recognition system | |
| CN107945153A (en) | A kind of road surface crack detection method based on deep learning | |
| CN111368852A (en) | Article identification and pre-sorting system and method based on deep learning and robot | |
| CN106920243A (en) | The ceramic material part method for sequence image segmentation of improved full convolutional neural networks | |
| CN109102543A (en) | Object positioning method, equipment and storage medium based on image segmentation | |
| CN111429418A (en) | Industrial part detection method based on YO L O v3 neural network | |
| CN107563388A (en) | A kind of convolutional neural networks object identification method based on depth information pre-segmentation | |
| CN106780546B (en) | The personal identification method of motion blur encoded point based on convolutional neural networks | |
| CN109242829A (en) | Liquid crystal display defect inspection method, system and device based on small sample deep learning | |
| CN109191424B (en) | Breast mass detection and classification system and computer-readable storage medium | |
| CN109284779A (en) | Object detecting method based on the full convolutional network of depth | |
| CN108648211A (en) | A kind of small target detecting method, device, equipment and medium based on deep learning | |
| CN112507758A (en) | Answer sheet character string identification method, answer sheet character string identification device, terminal and computer storage medium | |
| CN110929795A (en) | Method for quickly identifying and positioning welding spot of high-speed wire welding machine | |
| CN110458791A (en) | Mass defect detection method and detection device | |
| CN110866915A (en) | Circular inkstone quality detection method based on metric learning | |
| CN111611889A (en) | Miniature insect pest recognition device in farmland based on improved convolutional neural network | |
| CN108615401B (en) | Deep learning-based indoor non-uniform light parking space condition identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190816 |
|
| RJ01 | Rejection of invention patent application after publication |