CN111951238A

CN111951238A - Product defect detection method

Info

Publication number: CN111951238A
Application number: CN202010773007.2A
Authority: CN
Inventors: 王罡; 潘正颐
Original assignee: Shanghai Weiyi Intelligent Manufacturing Technology Co ltd; Changzhou Weiyizhi Technology Co Ltd
Current assignee: Shanghai Weiyi Intelligent Manufacturing Technology Co ltd; Changzhou Weiyizhi Technology Co Ltd
Priority date: 2020-08-04
Filing date: 2020-08-04
Publication date: 2020-11-17

Abstract

The invention provides a product defect detection method, which comprises the following steps: a product image acquisition step: the method comprises the steps of setting a camera according to different products and different optical surfaces of the same product to obtain an original image of the camera, wherein the original image is consistent with a reference optical surface, and transmitting the original image to an image processing module for image processing to finally obtain an image capable of carrying out model detection; and (3) detecting the model: and sending the obtained image capable of carrying out model detection to the depth model and the gray level detection model for detection, and finally obtaining a detection result returned by each optical surface of the product. The machine vision detection method based on the deep learning and the traditional image processing is innovatively adopted, so that the requirement of similarity comparison in the image acquisition stage exists, the consistent images are acquired, and the accuracy of subsequent data for depth model detection and the accuracy of image detection are ensured.

Description

Product defect detection method

Technical Field

The invention relates to the field of graph detection, in particular to a product defect detection method.

Background

The traditional optical guidance mainly depends on the subjective experience of an engineer, controls the optical imaging quality by adjusting the focal length, the aperture, the working distance and other modes of a camera, and has the advantages of common effect and poor imaging optical consistency of different machines.

The conventional visual guidance process is generally as follows: the feeding mechanism (mechanical arm, sucker and the like) with the industrial camera takes a picture before grabbing materials every time, and calculates the information of the position deviation and the angle deviation of the materials to be grabbed through visual software so as to ensure that the feeding mechanism can meet the feeding precision during feeding. The existing defect is that the precision cannot be ensured.

The traditional target detection algorithm usually adopts a sliding window strategy to traverse the whole image, then uses a Haar, SIFT, HOG and other feature extractors to extract a target object, and then uses SVM, Adaboost and other classifiers to classify the extracted target, although the exhaustive strategy contains all possible positions of the target, the defects are obvious: too high in temporal complexity, too many redundant windows are generated, which also seriously affects the speed and performance of subsequent feature extraction and classification. Moreover, it is not easy to design a robust feature due to the morphological diversity of the target, the illumination variation diversity, the background diversity, etc.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide a product defect detection method.

The invention provides a product defect detection method, which comprises the following steps:

a product image acquisition step: the method comprises the steps of setting a camera according to different products and different optical surfaces of the same product to obtain an original image of the camera, wherein the original image is consistent with a reference optical surface, and transmitting the original image to an image processing module for image processing to finally obtain an image capable of carrying out model detection;

and (3) detecting the model: sending the obtained image capable of carrying out model detection to a depth model and a gray level detection model for detection, and finally obtaining a detection result returned by each optical surface of the product;

physical quantity filtration step: and filtering the returned detection result by a threshold value or defect length, defect width and defect brightness physical quantity.

Preferably, the product image acquiring step:

step S101: moving the position of the camera and the workpiece to a specified optical point location

Step S102: triggering a camera to take a picture after setting camera parameters and a light source according to the optical surface information;

step S103: receiving an original image returned by a camera, and adding workpiece information corresponding to the original image to the head of the original image;

step S104: storing the camera original image with the head information for tracing the reason when the problem occurs;

step S105: distributing the camera original image with the head information to different optical surface picture preprocessing modules for parallel processing;

step S106: the image preprocessing module performs cutting, compressing, rotating, horizontal mirroring and vertical mirroring algorithm operations on the original camera image according to the number of the workpiece carrying platforms and the number of the machine station channels, and outputs an image meeting the model detection requirement, wherein the image preprocessing module comprises the following conditions:

the camera original image comprises a plurality of workpieces, and the workpieces in the image need to be cut out respectively;

the original image of the camera needs to be compressed to be smaller due to overlarge size so as to improve the model operation speed;

one workpiece is formed by combining a plurality of original camera images, and the original camera images need to be combined after being rotated and mirrored.

Preferably, the model detecting step includes:

the construction step comprises: designing and building a deep learning model for detecting the defects of the workpiece;

and (3) classification step: classifying each pixel point in the learning image to be learned according to categories, and judging the confidence of the type of each pixel point;

deep learning model training: training the learning image subjected to the pixel point class classification and confidence judgment to obtain a trained deep learning model;

and a defect detection step: and detecting the defects of the workpiece by using the trained deep learning model.

Preferably, the step of classifying: classifying each pixel point in the learning image, taking 0 as a background to represent the category of the pixel point, taking 1 as a defect to represent the category of the pixel point, and dividing the learning image into a background area and a defect area according to the category;

the step of classifying includes:

and (3) convolution step: performing feature extraction on an input layer, filtering partial useless information and reserving feature effective information;

a step of pooling: reducing the dimension of the input layer and reducing the calculated amount;

and (3) feature fusion step: performing cross-layer connection on different layers with the same dimension;

a category judgment step: quantizing the feature information obtained in the feature fusion step into confidence of a certain category;

an output step: outputting a multi-dimensional array vector as a result, representing the category and confidence of each pixel value in a learning image;

the multidimensional array vector comprises a [ m, n, c, s ] vector, wherein: m denotes the image width, n denotes the image height, c denotes the class, and s denotes the confidence.

Preferably, the deep learning model training step: and setting a set training step number, training the non-defective images and the defect images in the training set in a single-double alternative mode during training, stopping training until loss is not obviously reduced, and outputting a model corresponding to the step number at the moment as an output model of the training.

Preferably, the method further comprises the gray model detection step of: detecting the defects of the workpiece through gray level transformation and spatial filtering;

the gray model detecting step includes a top depression detecting step:

obtaining a rotation and translation matrix of the image through shape template matching positioning;

the affine transformed top region is obtained by rotating the translation matrix,

performing sub-pixel threshold segmentation on the top area of the affine transformation, and adding the edge line segment obtained by segmentation into the metering model;

the maximum and minimum distances from the edge point to the base line are calculated, and the difference between the maximum distance and the minimum distance is the value of the depression.

Preferably, the gray model detecting step includes: and a burr detection step:

finding an inner hole area through threshold segmentation, erasing an inner angle area of the inner hole area after affine transformation, and detecting burrs through closed operation;

the gray model detecting step includes: a water gap height detection step:

judging whether the height of the water gap reaches the standard or not through the rotation angle of the matrix;

the gray scale model detection step comprises a top crack detection step:

carrying out threshold segmentation on the picture to find out a detected region, and carrying out Fourier transform on the picture to transform the time threshold of the picture to a frequency domain;

filtering the intermediate frequency component by a Gaussian filter, converting the intermediate frequency component into a time threshold, and solving the shape of the line by a second derivative mode.

Preferably, the physical quantity filtering step:

setting a filtering rule: different products, different defects and different parts are judged to have different parameter conditions corresponding to the defects, and different rules are set to be used as the conditions for judging the defects; the filtering rules can set combination rules and distinguish rule priorities, and comparison is carried out on the rule priorities; if the first rule is compared and accords with the defect rule, the detection record of the product is directly judged to be the corresponding defect record, and other rules are not compared; if the defect rule is not met, continuing to compare the second rule; if the defect is not met, continuing to compare until all defect rules are compared; if the quality is not met, judging the detection record as a good product record at present;

setting rule conditions: the rule condition refers to a linearly quantized numerical value that can be taken as a condition, and includes the following physical quantity conditions: the defect threshold value, the defect length, the defect width, the defect area, the defect average brightness, the defect contrast, the defect gradient and the defect length-width ratio are combined into a judgment rule by setting one or more rules;

a non-detection area filtering step: aiming at different optical surfaces of a product, corresponding non-detection areas are provided, defects do not need to be detected in the areas, the defects are conditionally shielded and detected by setting area detection rule conditions, and then the target of accurate detection is achieved.

According to the present invention, there is provided a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the product defect detection method of any one of the above.

Compared with the prior art, the invention has the following beneficial effects:

1. according to the invention, through deep learning detection, the time complexity is greatly reduced, the generation of redundant windows is reduced, and the speed and performance of subsequent feature extraction and classification are greatly improved;

2. according to the invention, through deep learning detection, the characteristics of image robustness are improved;

3. the invention combines a plurality of detection modes, and the detection is more comprehensive and accurate.

4. The machine vision detection method based on deep learning and traditional image processing is innovatively adopted, so that the requirement of similarity comparison in the image acquisition stage exists, and the requirement is used for ensuring that consistent images are acquired, so that the accuracy of subsequent data for depth model detection and the accuracy of image detection are ensured.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a schematic flow chart of steps of a model detection method.

Detailed Description

The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.

The present invention will be described more specifically below with reference to preferred examples.

Preferred example 1:

firstly, acquiring a product image

The camera original image consistent with the reference optical surface is obtained by carrying out camera setting, exposure, the position of an interested area, line frequency, photographing delay and other parameters on different optical surfaces of different products and the same product, and the obtained original image is transmitted to an image processing module for image processing (including algorithms of cutting, compression, rotation, horizontal mirror image, vertical mirror image, combination and the like), and finally an image capable of being subjected to model detection is obtained.

The specific steps and algorithm for obtaining the product image are as follows:

1) moving the position of the camera and the workpiece to a specified optical point location

2) After camera parameters (including exposure value, gamma value, line frequency, region of interest and the like) and a light source are set according to the optical surface information, triggering the camera to take a picture

3) Receiving the original image returned by the camera, adding the workpiece information (including the workpiece number and the channel number) corresponding to the original image to the head of the original image

4) Storing camera original pictures with header information for tracing reasons when problems occur

5) Distributing camera original image with head information to different optical surface picture preprocessing modules for parallel processing

6) The image preprocessing module performs algorithm operations such as cutting, compressing, rotating, horizontal mirroring and vertical mirroring on the original camera image according to information such as the number of the workpiece carrying platforms and the number of machine passages, and outputs an image meeting the model detection requirement. The method specifically comprises the following conditions:

a) a camera original image comprises a plurality of workpieces, and the workpieces in the image need to be cut out respectively

b) The original image of the camera needs to be compressed to be smaller due to the overlarge size of the original image so as to improve the model operation speed

c) One workpiece is formed by combining a plurality of original images of cameras, and the original images of the cameras need to be combined after being rotated and mirrored

Second, model detection

As shown in fig. 1, the obtained model detection map is sent to a model pipeline service for detection, wherein a part of different products and different optical surfaces are sent to a depth model for detection, and a part of different products and different optical surfaces are sent to a gray detection model for detection, and finally detection information (including defect, defect threshold position information x coordinate, y coordinate, width, height, defect length, defect height, defect area, defect average brightness information, defect gradient information, defect contrast information, defect brightest 20% average brightness information, and defect darkest 20% average brightness information) returned by each optical surface of the product is obtained.

The model detection is divided into depth model detection and gray model detection.

1. The depth model detection comprises the following specific implementation steps:

1) the deep learning model is designed and set up and used for detecting the defects of the workpiece and is composed of a segmentation network and a classification network two-stage, wherein the deep learning model mainly comprises a convolution module, a pooling module, a feature fusion module, a category judgment module and an output module.

Dividing classification categories to which each pixel point in the network learning image belongs, expressing the pixel category as a background by 0, expressing the pixel category as a defect by 1, and dividing an image into a background area and a defect area according to the categories; the classification network judges each pixel point in the extracted background area and defect area on the basis of the segmentation network to give the possibility that each pixel point belongs to a certain category, namely confidence.

The convolution layer is characterized in that the input layer is subjected to characteristic extraction, part of useless information is filtered, and most of effective information of the characteristics is reserved; the pooling layer is characterized in that dimension reduction is carried out on the input layer to reduce the calculated amount; the characteristic fusion layer is characterized in that different layers with the same dimensionality are connected in a cross-layer mode to obtain richer characteristic information; the category judgment layer is characterized in that the feature information obtained by the feature fusion layer is quantized into a probability value of a certain category; the output layer is characterized in that after convolution pooling feature fusion and the like, a vector [ m, n, c, s ] is output as a result, and the category and the confidence coefficient of each pixel value in an image are represented.

2) Circularly training a deep learning model by using the divided data sets;

specifically, the embodiment (5) includes:

the deep learning training method is characterized in that all images in a well-divided training set folder are trained, the number of training steps is set to be more than 1000, good images and defect images in a training set are trained in a single-double alternative mode during training, and until loss is not reduced obviously, training is stopped to output a model corresponding to the number of steps at the moment, and the model is used as an output model of the training.

3) Carrying out appearance defect detection on a real scene workpiece by using a deep learning model, and judging and quantifying a detection result;

when the appearance defects of the real scene workpiece are detected, the output model is used for detecting the workpiece formed by injecting the metal powder, and the result is output according to the vector [ m, n, c, s ].

2. The detection of the gray-scale model is carried out,

the gray model detection is to use a gray conversion and spatial filtering mode for detection, and different detection modes are used for different defects. Specific defect types are: top depression, flash burr, nozzle height, impact defect, crack defect, deformation defect, etc. Image inversion, piecewise linear transformation, histogram equalization and matching, spatial filtering and the like exist in the algorithm, some special detections such as bruises and cracks are processed from a time domain to a frequency domain, and the detection is performed by using the characteristics of the frequency domain, and the specific algorithm is as follows:

1) detection of top recession

And positioning through shape template matching to obtain a matrix of the rotation and translation of the image. And obtaining a top area after affine transformation by rotating and translating the matrix, performing sub-pixel threshold segmentation on the top area after affine transformation, adding an edge line segment obtained by segmentation into the metering model, and then calculating the maximum and minimum distances from the edge points to the base line, wherein the difference between the maximum distance and the minimum distance is a depression value.

2) Detection of burrs on burrs

And positioning through shape template matching to obtain a matrix of the rotation and translation of the image. And finding an inner hole area through threshold segmentation, then erasing an inner angle area of the inner hole area after affine transformation, and finally detecting burrs through closed operation.

3) Height of water gap

And positioning through shape template matching to obtain a matrix of the rotation and translation of the image. Because the length of the bottom is fixed, whether the height of the water gap reaches the standard is judged through the rotation angle of the matrix.

4) Detection of roof cracks

The image is subjected to threshold segmentation to find out a detected area, the image is subjected to threshold transformation sometimes to a frequency domain through Fourier transformation, then components of an intermediate frequency are filtered through a Gaussian filter, and then the image is subjected to threshold transformation to obtain the shape of a line through a second derivative mode.

Thirdly, physical quantity filtration

Meets the requirement of dynamic adjustment of the on-site shipment yield of the customer, adopts a linear physical quantity filtering mode,

and (4) filtering physical quantities such as a threshold value, or defect length, defect width, defect brightness and the like of a returned detection result of the model.

The physical quantity filtration step was as follows:

1) setting filtering rules

Different products, defects and parts are judged to have different parameter conditions corresponding to the defects, so different rules are required to be set as the conditions for judging the defects. The filtering rules can set combination rules and distinguish rule priorities, and comparison is carried out on the rule priorities; if the first rule is compared and accords with the defect rule, the detection record of the product is directly judged to be the corresponding defect record, and other rules are not compared; if the defect rule is not met, continuing to compare the second rule; if the defect is not met, continuing to compare until all defect rules are compared; and if the quality is not met, judging the detection record as a good product record at present.

2) Setting rule conditions

The rule condition refers to a linear quantization value which can be used as a condition, and includes physical quantity conditions such as a defect threshold, a defect length, a defect width, a defect area, a defect average brightness, a defect contrast, a defect gradient, a defect length-width ratio and the like, and can be combined into a judgment rule by setting one or more rules, such as a scratch large-area rule: the area >3mm & & threshold >0.4& & defect length >0.5mm is judged as the defect record.

3) Non-detection region filtering

And if the area has over-killed defect records (the defect is detected by conditionally shielding by setting area detection rule conditions to achieve the aim of accurate detection), filtering and shielding. Supplementary explanation: and filtering and shielding defects occurring in the non-detection area.

In the description of the present application, it is to be understood that the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience in describing the present application and simplifying the description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present application.

Those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.

The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims

1. A method for detecting product defects, comprising:

2. The product defect detection method of claim 1, wherein the product image acquisition step:

3. The product defect detection method of claim 1, wherein the model detection step comprises:

4. The product defect detection method of claim 3, wherein the classifying step: classifying each pixel point in the learning image, taking 0 as a background to represent the category of the pixel point, taking 1 as a defect to represent the category of the pixel point, and dividing the learning image into a background area and a defect area according to the category;

the step of classifying includes:

5. The product defect detection method of claim 4, wherein the deep learning model training step: and setting a set training step number, training the non-defective images and the defect images in the training set in a single-double alternative mode during training, stopping training until loss is not obviously reduced, and outputting a model corresponding to the step number at the moment as an output model of the training.

6. The method of claim 4, further comprising a gray model inspection step: detecting the defects of the workpiece through gray level transformation and spatial filtering;

the gray model detecting step includes a top depression detecting step:

7. The product defect detecting method according to claim 6, wherein the gray model detecting step comprises: and a burr detection step:

the gray model detecting step includes: a water gap height detection step:

the gray scale model detection step comprises a top crack detection step:

8. The product defect detection method according to claim 1, wherein the physical quantity filtering step:

9. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the product defect detection method of any one of claims 1 to 8.