CN110689539A - A detection method for workpiece surface defects based on deep learning - Google Patents

Abstract

The invention discloses a workpiece surface defect detection method based on deep learning. A workpiece surface defect detection system is constructed by using the deep learning technology. Rapid identification and feedback of workpiece surface defects in the production environment ensures the accuracy and efficiency of the system. The system captures the surface of the workpiece through the image acquisition device, and uploads it to the processing computer after preprocessing by the capture terminal. The processing computer will call the predictor based on the deep neural network model to identify the image and output the predicted vector. Finally, the processing center publishes the prediction vector to the display terminal, so that the surface defect status of the workpiece can be visually displayed.

Description

A detection method for workpiece surface defects based on deep learning

technical field

The invention relates to the technical field of machine vision and detection, in particular to a workpiece surface defect detection method based on deep learning.

Background technique

Quality control is an important part of industrial upgrading. Taking workpiece defect detection as an example, at present, many manufacturers are still dominated by manpower, which is not only inefficient, but also increases labor costs. Adhering to the orientation of manufacturing informatization, replacing human vision with machine vision to participate in product quality control has become the development trend of today's manufacturing industry, and some manufacturers have begun to try to adopt traditional machine vision solutions. The technical solutions that have been applied so far mainly include foreground detection algorithms based on background modeling and learning algorithms based on support vector machines. Among them, the efficiency of the foreground detection algorithm based on background modeling depends on the scale of the captured image, that is, the amount of computation and the input scale show a power-increasing relationship. When the resolution of the captured image increases, the system efficiency often drops sharply. Compared with the foreground detection algorithm based on background modeling, the learning algorithm shows an improvement in system efficiency, but because the space consumption of the support vector machine is mainly used to store training samples and kernel matrices, when the matrix order (related to the number of input samples) When it is very large, it will consume a lot of storage space and machine memory. In addition, support vector machines are more difficult to solve multi-classification problems.

In recent years, deep neural networks (deep learning) have ushered in a blowout period of development with the great success of commercialization. It is widely used in the intelligent industry and brings huge economic benefits to the human society. Convolutional Neural Networks (CNN), as a typical deep neural network, have better ability to extract complex features than traditional algorithms, and are widely used in machine vision and image processing and other fields. Compared with traditional machine vision algorithms, if the parameters are properly optimized, it has better adaptability and accuracy, and the system efficiency is greatly improved. Compared with the learning algorithm based on support vector machine, its computational and storage cost is small, and it can be used to solve multi-classification problems. Therefore, it is necessary to develop an efficient and accurate workpiece defect detection technology based on deep learning for quality control.

SUMMARY OF THE INVENTION

Purpose of the invention: In order to solve the problems existing in the above background technology, the present invention uses deep learning technology to construct a workpiece surface defect detection system, solves the shortcomings of traditional methods such as high labor cost, low system efficiency, and poor adaptability, and can be used in production environments. Rapid identification and feedback of workpiece surface defects ensures the accuracy and efficiency of the system. Technical scheme: In order to realize the above-mentioned purpose, the technical scheme adopted in the present invention is:

A method for detecting surface defects of workpieces based on deep learning, comprising the following specific steps:

Step 1. Use multiple sets of cameras and light sources to construct an image acquisition system based on multi-eye vision to capture images of the workpiece at different angles;

Step 2: Constructing a distributed image processing system; using a pipeline monitoring mechanism, the processing computer maintains a specific message pipeline, and the capture terminal monitors the pipeline; the capture terminal will preprocess the collected images in batches during each image acquisition, and then upload them to the system. Database; using the persistence mechanism, the processing computer can save the preprocessed image locally as the input of the classifier;

Step 3. Collect training samples; put the prepared and marked workpieces with partial defects into the collection system to collect training data; set training and verification sets according to the ratio of 8:2, and set labels;

Step 4. Build a deep learning model; build a network model including a convolutional layer and a fully connected layer based on a convolutional neural network. Pooling, regularization, and normalization modules are combined between layers to optimize feature extraction, improve nonlinearity;

Step 5. Build the predictor program framework; use the Tensorflow API to build the detection system framework; build the program logic based on the factory mode, which is convenient for the upgrade, expansion and optimization of the system software;

Step 6. Place the workpiece to be detected in the image acquisition system, and run the predictor program; when running online, the predictor program interface will receive the preprocessed workpiece capture image as input; then call the trained static model to output the prediction vector;

Step 7: The predictor publishes the prediction vector message to the display terminal through a preset channel. The display terminal will automatically identify the location of the workpiece defect according to the predicted vector.

Further, performing batch preprocessing on the captured images in the step 2 includes converting the color space of the captured images; the capture terminal converts the color space of the input image from RGB format to YUV format.

Further, the network model construction steps in the step 4 are as follows: the data set is stored in the specified directory of the training script, the training script is run, the initial learning rate is set to 0.001, and the Adam optimizer is used; the specific parameters of the training model are in order They are:

(1) Convolutional layer 1, a total of 64 convolution kernels, the size of the convolution kernel is 11×11, the stride is set to 4, and the filling mode is set to SAME; the activation function is set to ReLU; composite 2×2 pooling and perform local response normalization;

(2) Convolutional layer 2, a total of 256 convolution kernels, the size of the convolution kernel is 5×5, the padding mode is set to SAME, the activation function is set to ReLU, compound 2×2 pooling and perform local response normalization;

(3) Convolution layer 3, a total of 256 convolution kernels, the size of the convolution kernel is 3×3, the padding mode is set to SAME, the activation function is set to ReLU, compound 2×2 pooling and perform local response normalization;

(4) Fully connected layer 1, mapped to 4096 dimensions;

(5) Fully connected layer 2, mapped to N dimension, where N represents the number of labels; the loss function is softmax;

After parameter adjustment, the optimized dynamic model is finally obtained; after running the conversion script, the static model in .pb format is obtained.

Further, the multiple groups of cameras in step 1 are preferably digital cameras.

Further, in step 1, the light source is preferably an LED light source.

Beneficial effects: Aiming at the problems of large consumption of human resources and low efficiency in traditional workpiece surface defect detection, the present invention proposes a workpiece surface defect detection method based on deep learning. The image acquisition system based on multi-vision vision ensures the inference accuracy, while the distributed image processing system greatly improves the system efficiency and robustness; the generalization ability of the predictor network model is improved by expanding the training set, ensuring system accuracy. Therefore, the labor cost is greatly reduced, and the system efficiency is improved while ensuring the accuracy.

Description of drawings

Fig. 1 is the main flow chart of a kind of workpiece surface defect detection method based on deep learning of the present invention;

Figure 2 is a schematic diagram of the spatial orientation of an image acquisition system based on multi-eye vision;

Fig. 3 is a schematic diagram of predictor software flow;

Fig. 4 is a schematic diagram of capturing terminal cluster distribution connection;

Fig. 5 is a predictor network model diagram;

6 is a structural diagram of a workpiece surface defect detection system based on deep learning of the present invention;

FIG. 7 is a diagram showing the specific implementation steps of a workpiece surface defect detection system based on deep learning.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings.

A specific embodiment is provided below:

The deep learning-based workpiece surface defect detection method of the present invention is divided into the following three modules: an image acquisition and preprocessing module, a predictor module, and a communication and data persistence module.

Image acquisition and preprocessing module:

(1) Image acquisition module:

The image acquisition system is designed as a cubic container, and from the perspective of space, it includes a total of 6 surfaces including up, down, left, right, front, and rear. However, since the bottom surface is used to hold the workpiece, there are actually only 5 surfaces to be inspected. According to the surface to be detected, it can be divided into four angles: side view angle (ie, front, back, left and right), side edge view angle (front left edge, front right edge, left rear edge, right rear edge), upper side view angle (top surface), upper side Edge angle (upper front edge, upper rear edge, upper left edge, upper right edge). Due to the limited field of view, the side and side edge perspectives need to be divided into upper and lower capture points in the spatial layout, so a total of (4+4)×2+4+1=21 cameras are required. In this embodiment, a Raspberry Pi is used as the terminal and is equipped with a camera.

As shown in Figure 2, the numbers 1, 4, and 5 represent the visible surfaces, and the numbers 2, 3, and 6 represent the current invisible surfaces. The meaning of each number is 1: front 2: back 3: left 4: right 5: top 6: bottom. Each edge identification can be uniquely determined by two faces, consisting of two digits, representing the two intersecting faces that generate the edge, respectively 1-3; front left edge 1-4; front right edge 3-2; left Posterior edge 4-2; right posterior edge 5-1; upper anterior edge 5-2; upper posterior edge 5-3; upper left edge 5-4; upper right edge.

(2) Light source selection and lighting control

The light source is an important prerequisite for image acquisition. The quality of the acquired image as the input of the defect detection system directly determines the accuracy of the prediction result to a large extent. A suitable light source can not only magnify defect details, but even mask out interfering features, thereby reducing the noise of inspected samples.

The types of light sources can be divided into natural light and artificial light. Since the image acquisition system is a closed configuration, the artificial light source is selected for this system. Considering that the LED has the characteristics of long life and low power consumption, the LED is selected as the illumination generator in this embodiment.

(3) Camera selection

Cameras are mainly divided into two categories: analog cameras and digital cameras. Among them, analog cameras are relatively simple, but because the computer can only recognize digital signals, the imaging results need to be converted by DA before being input into the computer. The digital camera directly collects digital signals, which can be input to the computer terminal without going through the DA conversion circuit. Compared with the two, digital cameras are mostly used in short-distance, low-interference space environments. Considering the small spatial scale of the image acquisition system, a digital camera is used for the convenience of system design.

Predictor module:

(1) Predictor hardware design

The predictor mainly consists of a display module and a data processing host. Among them, the operating system platform of the processing host is linux, and the hardware platform is equipped with high-performance GPU and CPU.

(2) Predictor software design

As shown in Figure 3, the predictor software is mainly composed of three parts: an input module, a predictor module, and an output module. The operating mechanism and functions of each module are:

Input module: Provides a unified interface for detecting sample input, and uses CVMat_to_TensorAPI provided by CV2 to uniformly compress the preprocessed original image to the input layer size (224×224) of the model, and convert it to Tensor as the input of the model.

Predictor module: Load the trained and optimized static model (.pb format), set the node name of the input layer and the node name of the output layer to be the same as the corresponding node names in the static network model, and call the corresponding API to run the network.

Output module: After the prediction module is executed, it will generate a tensor type queue (prediction vector), and use the argmax method to extract the largest prediction component in each vector as the prediction result. The system adopts a binary classification method, that is, the prediction vector has two dimensions, [1,0] indicates that the workpiece surface is defective, and [0,1] indicates that the workpiece surface is intact. Table 1 lists several important functions related to predictors:

Table 1

Communication and data persistence module:

(1) Communication system construction

As shown in FIG. 4 , the basic communication mode utilized by this method is implemented as a bridge mode. The implementation method is to connect a plurality of network ports expanded on a terminal to each other. That is, one raspberry is connected to the other four Raspberry Pis as a "cluster head" to form a "cluster structure", and then the "cluster head" is connected to the switch.

(2) Data persistence

In this embodiment, the Redis database is used to implement data persistence. Redis is a distributed key-value database, which has the characteristics of high performance, rich data types, and support for atomicity, thus ensuring the efficiency of the system.

The system mainly maintains two pipelines, Channel@1 and Channel@2. Its functions are:

Channel@1 is used as the message channel between the host computer and the image acquisition terminal: the host computer periodically sends out the "image acquisition" command, and all acquisition terminals monitoring the pipeline will call the capture script to capture the workpiece image. After the preprocessing step , which stores the image in the database as a binary string. After the acquisition is completed, persistence is performed, and the upper computer obtains the image blocks captured from various angles of the current workpiece as the original input of the predictor.

Channel@2 is used as the message channel between the host computer and the display terminal: when the predictor outputs the result, the data processing host will release the prediction vector to the display terminal through this channel. After receiving the prediction vector message, the display terminal will call the parser to parse the prediction vector, and feed back the prediction result in the form of an image.

Detailed embodiments of the present invention are given below with reference to FIGS. 1-7 .

Step 1. Use the Raspberry Pi as the platform for capturing the terminal, and the external camera as the image capturer. A total of 21 digital cameras are arranged on the side, side edge, upper side and upper side edge of the inner surface of the image acquisition container. Considering the size of the capture surface, it is necessary to construct a line light source, which is evenly distributed on the inner surface of the collection container.

Step 2. Start the Redis service on the processing computer, and set the listening network segment and port number (the default is port 6379). After the Redis service is started, the processing computer runs the command publishing process, and sets two message pipes Channel@1 and Channel@2 in it. Then, remotely connect to the capture terminal and the display terminal in batches, start the monitoring process of the capture terminal and the display terminal, the capture terminal monitors the channel Channel@1, and the display terminal monitors the channel Channel@2. Start acquiring training images. Whenever the capture terminal captures a frame of original image (size is 2952×1944), the script will be called to fill and segment the background. After obtaining a segmented block with a size of 800×800, convert the image in RGB format to YUV format, and store it in the database in the form of a binary string.

Step 3: The processing computer performs a persistence operation to acquire the image captured by the capture terminal. Manual annotation is performed according to the defect display status of the collected workpiece image blocks. After collecting 100 workpieces and a total of 5,300 image blocks, image enhancement and deformation are performed in batches, and the training set is expanded to about 20,000.

Step 4. Store the data set in the directory specified by the training script, run the training script, set the initial learning rate to 0.001, and use the Adam optimizer; the specific parameters of the training model in order are:

(1) Convolutional layer 1, a total of 64 convolution kernels, the size of the convolution kernel is 11×11, the stride is set to 4, and the filling mode is set to SAME; the activation function is set to ReLU; composite 2×2 pooling and perform local response normalization;

(2) Convolutional layer 2, a total of 256 convolution kernels, the size of the convolution kernel is 5×5, the padding mode is set to SAME, the activation function is set to ReLU, compound 2×2 pooling and perform local response normalization;

(3) Convolution layer 3, a total of 256 convolution kernels, the size of the convolution kernel is 3×3, the padding mode is set to SAME, the activation function is set to ReLU, compound 2×2 pooling and perform local response normalization;

(4) Fully connected layer 1, mapped to 4096 dimensions;

(5) Fully connected layer 2, mapped to N dimension, where N represents the number of labels; the loss function is softmax;

After parameter adjustment, the optimized dynamic model is finally obtained; after running the conversion script, the static model in .pb format is obtained, as shown in Figure 5.

Step 5. Modify the configuration file of the predictor program, and specify the path of the input image and the storage path of the static model. Once set up, run the predictor program to load the static model.

Step 6. After the predictor program successfully loads the static model, it enters the prediction mode. The workpiece to be detected is placed in the collection container, and the collection instruction is issued to the capture terminal on the processing computer. After receiving the acquisition instruction, the capture terminal will automatically capture the image of the workpiece. The preprocessed image blocks are stored in the database, and at the same time, the processing computer performs persistence intermittently.

Step 7. Once the predictor reads the image block input, it will use the library function to convert the image data into Tensor as the input of the network model, and then run the network. After the network outputs the prediction vector, the processing computer transmits the information to the message publishing process running in the background, and publishes the prediction vector to the display terminal via Channel@2. After receiving the prediction vector, the display terminal will feed back the defect status of the workpiece to be inspected in a graphical form.

The above is only the preferred embodiment of the present invention, it should be pointed out that: for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can also be made, and these improvements and modifications are also It should be regarded as the protection scope of the present invention.

Claims (5)

1. A workpiece surface defect detection method based on deep learning is characterized in that: the method comprises the following steps:

step 1, constructing an image acquisition system based on multi-view vision by adopting a plurality of groups of cameras and illumination sources, and capturing images of a workpiece at different angles;

step 2, constructing a distributed image processing system; a pipeline monitoring mechanism is adopted, a processing computer maintains a specific message pipeline, and a capture terminal monitors the pipeline; the method comprises the steps that a capture terminal carries out batch preprocessing on collected images during image collection each time and then uploads the preprocessed images to a database; by utilizing a persistence mechanism, the processing computer can store the preprocessed pictures to the local as the input of the classifier;

step 3, collecting training samples; putting the prepared and marked workpiece containing partial defects into an acquisition system for training data acquisition; according to the following steps of 8: 2, setting a training and verification set according to the proportion, and setting a label;

step 4, constructing a deep learning model; constructing a network model comprising convolutional layers and full-connection layers based on a convolutional neural network, wherein pooling, regularization and normalization modules are compounded among the layers and used for optimizing feature extraction and improving nonlinearity;

step 5, constructing a predictor program framework; constructing a detection system framework by adopting a Tensorflow API; program logic is constructed based on a factory mode, so that system software is convenient to upgrade, expand and optimize;

step 6, placing the workpiece to be detected in an image acquisition system, and operating a predictor program; when the online operation is carried out, the predictor program interface takes the workpiece capture image which is received and preprocessed as input; then, calling the trained static model to output a prediction vector;

step 7, the predictor distributes the prediction vector information to the display terminal through a preset channel; and the display terminal automatically identifies the position of the workpiece defect according to the prediction vector.

2. The workpiece surface defect detection method based on deep learning of claim 1, wherein: in the step 2, the collected images are subjected to batch preprocessing, including converting the color space of the captured pictures; the capture terminal converts the color space of the input image into a YUV system in an RGB system mode.

3. The workpiece surface defect detection method based on deep learning of claim 1, wherein: the network model construction step in the step 4 is specifically as follows: storing the data set in a specified directory of a training script, operating the training script, setting the initial learning rate to be 0.001, and adopting an Adam optimizer; the specific parameters of the training model in sequence are respectively as follows:

(1) convolutional layer 1, with a total of 64 convolutional kernels, the size of the convolutional kernels is 11 × 11, the step size is set to 4, and the fill mode is set to SAME; the activation function is set to ReLU; compounding 2 × 2 pooling and performing local response normalization;

(2) convolutional layer 2, with a total of 256 convolutional kernels, with the convolutional kernel size 5 × 5, the fill mode set to SAME, the activation function set to ReLU, composite 2 × 2 pooling and local response normalization performed;

(3) convolutional layer 3, totaling 256 convolutional kernels, with the convolutional kernel size of 3 × 3, the fill mode set to SAME, the activation function set to ReLU, composite 2 × 2 pooling and local response normalization performed;

(4) fully connected layer 1, mapped as 4096 dimensions;

(5) a fully connected layer 2, mapped to N dimensions, where N represents the number of tags; the loss function is softmax;

finally obtaining an optimized dynamic model through parameter adjustment; and after the conversion script is operated, obtaining the static model in the pb format.

4. The workpiece surface defect detection method based on deep learning of claim 1, wherein: the multiple groups of cameras in the step 1 are preferably digital cameras.

5. The workpiece surface defect detection method based on deep learning of claim 1, wherein: the illumination source in the step 1 is preferably an LED light source.

Images