WO2023168972A1

WO2023168972A1 - Linear array camera-based copper surface defect detection method and apparatus

Info

Publication number: WO2023168972A1
Application number: PCT/CN2022/130842
Authority: WO
Inventors: 吴俊义; 张弛; 顾献代; 方明; 吴家乐; 任禹桥; 梅值
Original assignee: 三门三友科技股份有限公司
Priority date: 2022-03-09
Filing date: 2022-11-09
Publication date: 2023-09-14
Also published as: CN114638797A

Abstract

Disclosed is a linear array camera-based copper surface defect detection method, comprising: a linear array camera collecting and synthesizing surface images which comprise the front side and the back side of a copper cathode plate; preprocessing and partitioning the surface images into several image blocks; a copper particle detection system, which has undergone deep learning training, performing defect recognition on the preprocessed image blocks, and synthesizing the recognition results into a surface defect distribution map of the copper cathode plate; and setting a defect parameter threshold according to quality requirements for copper cathode, and classifying the copper cathode plate according to the surface defect distribution map of the copper cathode plate. Further disclosed is an apparatus, comprising a transport module; a linear array camera collection module; and an industrial computer and a control module. For the entire detection process of the present invention, conventional manual detection is changed into automatic detection, which reduces the impact of man-made factors on a defect detection result, reduces the missed detection rate, further unifies a defect detection standard, and yields a more accurate result. In addition, a linear array camera is used to collect images, so that setting a position for image collection is more flexible and convenient.

Description

A copper surface defect detection method and device based on a line array camera

Technical field

The invention relates to the technical field of cathode copper quality detection, and in particular to a copper surface defect detection method and device based on a line array camera.

Background technique

Due to the particularity of the electrolysis process of metallurgical refining of cathode copper, the surface condition of cathode copper can basically reflect the quality of cathode copper, including surface texture, defects, color, etc. There are certain problems in the entire cathode copper surface quality inspection process: taking pictures, the existing cathode copper surface quality mainly relies on manual inspection, which is manually inspected one by one in the stripping process of the unit; data analysis, the cathode copper runs at high speed in the stripping unit assembly line , it is impossible to accurately calculate the size and distribution of copper cathode particles manually, and can only be judged subjectively based on experience; to determine the output, when a problematic cathode copper is found, it needs to be manually clicked to eliminate, and different people will give different results, which is labor-intensive and leakage occurs. The risk of inspection is high; subsequent re-inspection by quality inspectors is required. If the re-inspection finds that there is a problem with the cathode copper, it needs to be unpacked and processed. After unpacking, it will be handled manually by a forklift and then repacked. The process is cumbersome and increases production costs.

"A method, device and system for detecting verticality of copper electrolytic cathode plates" disclosed in Chinese patent documents, its publication number is CN106066169A, and the publication date is 2016-11-02, including obtaining each preset detection point on the cathode plate The center position of Find the highest point and the lowest point of the preset detection point surface, and calculate the preset detection point surface difference based on the highest point and the lowest point; determine the maximum value from the center position difference and the preset detection point surface difference, and determine the The maximum value of is determined as the cathode plate verticality. Through real-time and automatic collection of the dynamic ranging data of the cathode plate during the copper electrolysis process, and by processing the dynamic ranging data, the verticality of the cathode plate can be quickly obtained. The obtained verticality of the cathode plate is very accurate, which greatly improves the detection efficiency. and the accuracy of detection data. However, this technology detects the overall verticality by sampling multiple points on the surface of the cathode plate. When it comes to detecting defects at local locations on the surface, the defect data on the surface of the cathode copper plate cannot be accurately obtained, and there may even be omissions. Therefore improvements are still needed.

Contents of the invention

The present invention is to overcome the problems in the prior art of using manual methods to detect the quality of cathode copper plates, such as high labor intensity, high risk of missed inspections, cumbersome detection process and increased cost due to different judgment standards, and provides a method based on a line array camera. The copper surface defect detection method and device can automate the entire cathode copper surface defect detection process, reduce the impact of human factors on defect detection results, and at the same time improve the accuracy and precision of detection results by setting unified defect judgment standards for defect thresholds in advance. Spend.

In order to achieve the above objects, the present invention adopts the following technical solutions:

A copper surface defect detection method based on a line array camera, including:

S1. The line array camera collects and synthesizes surface images including the front and back sides of the cathode copper plate;

S2. Preprocess the collected surface image of the cathode copper plate and divide it into several image blocks;

S3. The copper particle detection system trained by deep learning performs defect identification on the preprocessed image blocks, and synthesizes the identification results of each image block into a surface defect distribution map of the cathode copper plate;

S4. Set the defect parameter threshold according to the cathode copper quality requirements, and classify the cathode copper plates based on the surface defect distribution map of the cathode copper plates.

In the present invention, there will be problems such as equipment blocking during the transmission of copper cathode, making it impossible to capture the entire copper cathode surface at one time. Therefore, a line array camera is used to continuously collect linear image data of the copper cathode surface and synthesize a complete two-dimensional For images, there is no need to find a specific location to collect a complete image, and the location of the image collection point is more flexible. At the same time, a copper particle detection system trained by deep learning is used for defect identification, which can more accurately identify and analyze defect data, and can set different defect parameter thresholds according to actual needs to classify cathode copper quality, so that it can Flexibly meet the differentiated requirements of different users for cathode copper.

Preferably, the process of preprocessing the surface image of the cathode copper plate in S2 includes: identifying and intercepting the image of the area where the copper plate is located; dividing the image into image blocks of m rows and n columns; performing Gaussian filtering on each image block to obtain mn Each image block is the preprocessed image.

The complete cathode copper plate image in the present invention is generally a large image in meters. After the image is divided into multiple small image blocks through preprocessing, it can be individually classified, labeled, learned and identified according to the types of defects existing in the image blocks. Classification and identification are carried out according to the external light conditions when collecting different image blocks, which solves the problem of inconsistent picture quality in different areas of large-size images and tries to ensure the uniformity of picture quality in a single image block.

Preferably, the deep learning training process of the copper particle detection system in S3 includes:

S31. Perform defect annotation on the preprocessed image blocks with existing defect detection results to generate annotation images, and together form a copper particle defect data set;

S32. Construct a copper particle detection system using the convolutional neural network module and the pyramid pooling module;

S33. Repeat the image recognition training of the copper particle detection system using the copper particle defect data set;

S34. The training is completed when the copper particle identification pixel accuracy of the copper particle detection system reaches the specified value.

The copper particle defect data set in the present invention includes pre-processed images that have completed defect detection and their corresponding annotated images. The annotated images refer to images that are reversely selected after defect annotation and graffiti on the pre-processed images. The preprocessed image is used as the input value, and the annotated image is used as the theoretical output value. The copper particle detection system is trained for image recognition. Different groups of preprocessed images and annotated images are selected for repeated training until the results recognized by the copper particle detection system are consistent with Training is completed when the difference between the labeled images converges or is less than a fixed value.

Preferably, the annotation information of the annotated image in the copper particle defect data set includes defect type, defect shape and size, copper particle color depth and particle aggregation degree. The more complete types of defects are included in the copper particle defect data set in the present invention and the richer the various defect data, the better the recognition effect of the copper particle detection system after training is completed and the higher the accuracy.

Preferably, the image recognition training process includes:

Select a set of samples from the copper particle defect data set containing input data and theoretical output values;

Input the input data into the copper particle detection system to obtain the corresponding actual output value;

Compare and calculate the difference between the theoretical output value and the actual output value;

Adjust the parameters in the copper particle detection system in a manner that minimizes the error until the difference converges.

The copper particle detection system in the present invention is mainly divided into two parts: a convolutional neural network module and a pyramid pooling module. The characteristic map of the image is obtained through convolution operation, and then the image is subjected to multi-resolution convolution processing through the pyramid pooling module. After fusion, the detection system can segment the background pixels and different copper particle pixels in the image, and then compare the actual identified copper particle output results with the theoretically output results to learn and train the copper particle detection system, and finally obtain a consistent Required detection system.

Preferably, in S4, for the surface defect distribution map of the cathode copper plate, the defective part image is separated through the defect parameter threshold set in advance; the regional characteristics of the defective part image are extracted, and the basic parameters of the defect are calculated according to the pixels in the area.

When the copper particle detection system of the present invention identifies pre-processed images, the defect threshold can be set in advance. Each pre-processed image will have an actual output value after recognition. After normalization processing, , if the actual output value is greater than the set defect threshold, the part can be considered to be a defect, otherwise it is considered to belong to the normal background.

A copper surface defect detection device based on a line array camera, including:

Transfer module, used to transfer cathode copper plates;

Line array camera acquisition module, including line array camera and light source, used to collect cathode copper plate surface images;

Industrial computer, used to receive the collected surface image data of the cathode copper plate and conduct defect identification and analysis;

The control module is used to receive the identification and analysis results of the industrial computer and control the work of the entire device.

In the present invention, the transmission module is responsible for transmitting the cathode copper plate from the production site to different locations according to the quality of the cathode copper plate after defect detection; the line array camera and the light source cooperate to clearly collect the image of the surface of the cathode copper plate. The industrial computer and the line The array camera is connected to receive the image data collected by the line array camera. At the same time, the industrial computer will transmit the results of image data analysis and identification to the connected control module. The control module controls the transmission module to transmit cathode copper plates of different qualities to different place.

Preferably, the line array camera acquisition module is provided with light sources and line array cameras in sequence according to the direction of transmission of the cathode copper plate; the light source illuminates the surface of the cathode copper plate at a certain incident angle, and the direction facing the line array camera lens is vertical on the surface of the cathode copper plate.

In the present invention, it is necessary to control the frequency of the line array camera shooting and collecting images according to the transmission speed of the cathode copper plate, so that the continuously collected images can completely synthesize the surface image of the cathode copper plate; the relative positions of the line array camera and the light source are set to ensure the collection The clarity of each frame of image; at the same time, the line array camera acquisition module composed of a line array camera and a light source can be flexibly set at different positions of the transmission module according to actual needs.

The invention has the following beneficial effects: using a line array camera for image collection, there is no need to find a specific position where the entire surface of the cathode copper plate can be completely seen, making the position setting of the collection device more flexible and convenient; the entire detection process is performed by traditional manual detection Switching to automatic detection reduces the impact of human factors on defect detection results, reduces the missed detection rate, makes defect detection standards more unified, and results are more accurate; different defect parameter thresholds can be set according to actual needs, and different judgment standards can be used to detect defects. The surface defects of the cathode copper plate are detected to meet the differentiated requirements of customers.

Description of the drawings

Figure 1 is a flow chart of the defect detection method of the present invention;

Figure 2 is a schematic diagram of the surface image acquisition device in Embodiment 1 of the present invention;

Figure 3 is a schematic diagram of the surface image acquisition device in Embodiment 2 of the present invention;

Figure 4 is a schematic diagram of deep learning training in an embodiment of the present invention;

In the picture: 1. Reverse light source; 2. Reverse camera; 3. Cathode copper plate; 4. Front light source; 5. Front camera; 6. Industrial computer; 7. Control module; 8. Board fork.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

As shown in Figure 1, a copper surface defect detection method based on a line array camera includes:

S1. The line array camera collects and synthesizes surface images including the front and back sides of the cathode copper plate.

S2. Preprocess the collected surface image of the cathode copper plate and divide it into several image blocks. The process of preprocessing the surface image of the cathode copper plate in S2 includes: identifying and intercepting the image of the area where the copper plate is located; dividing the image into m rows and n columns. image blocks; the mn image blocks obtained by performing Gaussian filtering on each image block are the preprocessed images.

S3. The copper particle detection system trained by deep learning performs defect identification on the preprocessed image blocks, and synthesizes the recognition results of each image block into a surface defect distribution map of the cathode copper plate; deep learning training of the copper particle detection system The process includes:

S4. Set the defect parameter threshold according to the cathode copper quality requirements, and classify the cathode copper plates based on the surface defect distribution map of the cathode copper plate. In S4, for the surface defect distribution map of the cathode copper plate, the defective parts are separated through the defect parameter threshold set in advance. Image; extract the regional features of the defective part of the image, and calculate the basic parameters of the defect based on the pixels in the area.

The annotation information of the annotated images in the copper particle defect data set includes defect types, defect shapes and sizes, copper particle color depth and particle aggregation degree.

The image recognition training process includes: selecting a set of samples containing input data and theoretical output values from the copper particle defect data set; inputting the input data into the copper particle detection system to obtain the corresponding actual output value; comparing and calculating the theoretical output value and the actual output value The difference; adjust the parameters in the copper particle detection system by minimizing the error until the difference converges.

The copper particle defect data set in the present invention includes pre-processed images with existing defect detection results and their corresponding annotated images. The annotated image refers to an image that is reversely selected after defect annotation and graffiti on the pre-processed image. Using the preprocessed image as the input value and the annotated image as the theoretical output value, the copper particle detection system is trained for image recognition. Different groups of preprocessed images and annotated images are selected for repeated training until the results are recognized by the copper particle detection system. When the difference from the annotated image converges or is less than a fixed value, the training is completed.

The more complete types of defects are included in the copper particle defect data set in the present invention and the richer the various defect data, the better the recognition effect of the copper particle detection system after training is completed and the higher the accuracy.

When the copper particle detection system of the present invention identifies pre-processed images, the defect threshold can be set in advance. Each pre-processed image will have an actual output value after recognition. After normalization processing, , if the actual output value is greater than the set defect threshold, the part can be considered to be a defect, otherwise it is considered to belong to the normal background. In the present invention, the quality of the cathode copper plate is determined based on the customer's quality evaluation standards for copper. For example, a cathode copper plate with more than 10 single defects per unit area and an area greater than 100 square millimeters belongs to Class B copper. If each customer's quality assessment level is not exactly the same, the assessment standards can be adjusted according to the actual needs of the customer. Then, the images collected during the detection process are in color. The patina is green and the crystal is blue. Statistics can be distinguished by color, the results can be analyzed, and they can be classified according to the different conditions they meet.

A copper surface defect detection device based on a line array camera, including: a transmission module for transmitting a cathode copper plate; a line array camera acquisition module composed of a line array camera and a light source, for collecting cathode copper plate surface images; an industrial computer, with It is used to receive the collected surface image data of the cathode copper plate and perform defect identification and analysis; the control module is used to receive the identification and analysis results of the industrial computer and control the work of the entire device.

In the line array camera acquisition module, the light source and line array camera are arranged in sequence according to the transmission direction of the cathode copper plate; the light source illuminates the surface of the cathode copper plate at a certain incident angle, and the direction facing the line array camera lens is perpendicular to the surface of the cathode copper plate.

During the implementation of the present invention, images of the front and back sides of the cathode copper plate are first collected through a line array camera. There are two structures available for collecting images on the surface of the cathode copper plate.

Embodiment 1, as shown in Figure 2, is a robot unit-type image acquisition mechanism. The back light source 1 illuminates the vertical part of the transmission module at a certain tilt angle. The back camera 2 is set below the back light source. The back camera 2 The lens faces the vertical part of the transmission module; the front light source 4 illuminates the horizontal part of the transmission module at a certain tilt angle, the front camera 5 is set on the right side of the front light source, and the lens of the front camera faces the horizontal part of the transmission module. Both the front camera and the back camera are connected to the industrial computer 6 for data transmission, and the industrial computer is connected to the control module 7 for data transmission. The cathode copper plate 3 is first transported vertically downward from the transmission module. During this process, the reverse camera collects images of the reverse side of the cathode copper plate and synthesizes it to obtain a complete reverse image of the cathode copper plate. When the cathode copper plate is lowered to meet the connecting fork 8 and is fixed, the connecting fork drives the cathode copper plate to reverse 90 degrees to the right, so that the front side of the cathode copper plate is vertically upward, and then the cathode copper plate is separated from the connecting fork and continues to be conveyed. The module is transmitted to the right. During this process, the front camera collects images of the front of the cathode copper plate and synthesizes it to obtain a complete front image of the cathode copper plate. The reverse camera and the front camera respectively transmit the reverse image and the front image to the industrial computer for identification and analysis of the images. The obtained results are then transmitted to the control module. The control module controls the transmission module to transmit the cathode copper plate to the computer based on the defect results detected. different locations.

Embodiment 2, as shown in Figure 3, is an image acquisition mechanism of a chain unit. The cathode copper plate 3 is suspended vertically for transmission. In the figure, the transmission direction of the cathode copper plate is from left to right, and is set on the front side of the cathode copper plate. There is a front camera 5 and a front light source 4. The front light source illuminates the front of the cathode copper plate at a certain angle. The lens of the front camera faces the front of the cathode copper plate to collect the front image. A reverse camera 2 and a reverse light source 1 are provided on the reverse side of the cathode copper plate. The arrangement of the reverse light source and the reverse camera and the front camera and the front light source are symmetrical with respect to the cathode copper plate. The reverse camera collects reverse images of the cathode copper plate. The reverse camera and the front camera are connected to the industrial computer 6, and the reverse image and the front image are transmitted to the industrial computer. After the industrial computer identifies and analyzes the image data, the result is transmitted to the control module 7 connected to the industrial computer. The control module The transfer module is controlled to transfer the cathode copper plate to different locations according to the recognition results.

After completing the collection of the front image and the back image of the cathode copper plate through Embodiment 1 or 2, the industrial computer performs data analysis on the collected images. In the process of data analysis, two-dimensional image preprocessing is first carried out, the image is converted into HSV color space, the area where the copper plate is located is identified through the color recognition algorithm, the image of the area where the copper plate is located is intercepted, and divided into 6×8 image blocks according to size , perform Gaussian filtering on each image block, and the 48 processed image blocks are the preprocessed images.

Before performing defect identification on preprocessed images, it is necessary to conduct deep learning training on the defect identification tool, namely the copper particle detection system, so that the identification accuracy of the copper particle detection system meets the requirements. As shown in Figure 4, the deep learning training process for the copper particle detection system is as follows:

The first step is to generate a copper particle defect data set. The copper particle defect data set includes the preprocessed image obtained by performing the same preprocessing procedure on the images collected in the historical inspection data and the annotated image corresponding to the image. The annotated image here It is generated after defect annotation of the preprocessed image. The annotated image is the ideal data result obtained by defect detection. In Figure 4, the preprocessed image is the input image, and the annotated image is also shown in the figure; The copper particle detection system in the intermediate link is learned and trained when both the input image data and the annotated image data are known.

The second step is to build a copper particle detection system and apply the pyramid scene analysis network. The copper particle detection system consists of a convolutional neural network module and a pyramid pooling module. The convolutional neural network module performs convolution operations on the preprocessed images. To extract its characteristic map, the pyramid pooling module convolves the image with multi-channel different templates and then fuses it. Finally, the image can be segmented and identified based on the background pixels and different copper particle pixels. The picture shows two convolution kernels performing convolution operations on four ARGB channels. When generating the corresponding map, this convolution kernel corresponds to 4 convolution templates. The four templates corresponding to this convolution kernel are different. The value of the corresponding position of the feature map is obtained by adding the convolution results of the four-core convolution template at the corresponding positions of the four channels and then taking the activation function. Therefore, in the process of obtaining two channels from four channels, the number of parameters It is 4×2×4, where the first 4 represents 4 channels, 2 represents the generation of 2 convolution kernels, and the last 4 represents the convolution kernel size.

The third step is to use the copper particle defect data set to train the copper particle detection system. Training using the defect data set is mainly divided into two processes. The first process is the forward propagation process: selecting a sample from the copper particle defect data set ( x, Yp), where x is the input image data, and Yp is the annotated image data, that is, the ideal output value; input the input image data represented by x into the copper particle detection system, and calculate the corresponding actual output value Op at this time. The second process is the backward propagation process: compare and calculate the difference between the actual output Op and the corresponding ideal output value Yp; according to the difference result, adjust the weight matrix by back propagation by minimizing the error, and repeatedly select copper particle defects. Different samples in the data set are trained until the differences converge. This completes the deep learning training of the copper particle detection system.

After completing the training of the copper particle detection system, the trained copper particle detection system is stored in the industrial computer, and then the formal detection of surface defects of the cathode copper plate is carried out. The front and back images collected by the front and back cameras are transmitted to the industrial computer. The images are first pre-processed, and the pre-processed images are input into the copper particle detection system for identification. Different defect recognition results can be obtained, thereby obtaining a surface defect distribution map. Each corresponding part of the pixel in the picture has a corresponding output value. By setting the defect threshold, the defective part and the normal part of the image are judged, and the segmentation process is In the result image of the deep learning detection, the defect part image is separated, and then through the connected domain label, the discrete defect connected domain is found, and the basic parameters such as area, length, and width of each connected domain are calculated based on pixels, so as to obtain the basic attributes of the defect. , which facilitates the setting of defect definition standards. The definition of defects and the quality judgment of the cathode copper plate can be freely combined according to the shape, size, and quantity of the defects, or can be defined by a combination of parameters such as convexity coefficient or ambiguity or aspect ratio through the operation of the basic attributes of the defects.

After the identification of surface defects of the cathode copper plate is completed, the quality of the inspected cathode copper plate is assessed according to the customer's quality assessment standards for the cathode copper plate. The control module transmits the cathode copper plate to different locations according to the quality of the cathode copper plate. For the embodiment A kind of robot unit type transfer module can place cathode copper plates in different positions according to different quality classifications. For the chain unit transfer module in the second embodiment, cathode copper plates that do not meet the quality requirements are rejected and separated in the later stage. Perform peeling.

The above embodiments are further elaborations and explanations of the present invention to facilitate understanding, and are not intended to limit the present invention in any way. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection of the invention.

Claims

A copper surface defect detection method based on a line array camera, which is characterized by including:

S1. The line array camera collects and synthesizes surface images including the front and back sides of the cathode copper plate;

S2. Preprocess the collected surface image of the cathode copper plate and divide it into several image blocks;

S3. The copper particle detection system trained by deep learning performs defect identification on the preprocessed image blocks, and synthesizes the identification results of each image block into a surface defect distribution map of the cathode copper plate;

S4. Set the defect parameter threshold according to the cathode copper quality requirements, and classify the cathode copper plates based on the surface defect distribution map of the cathode copper plates.
A copper surface defect detection method based on a line array camera according to claim 1, characterized in that the process of preprocessing the surface image of the cathode copper plate in S2 includes: identifying and intercepting the image of the area where the copper plate is located; Divide it into m rows and n columns of image blocks of the same size; perform Gaussian filtering on each image block to obtain mn preprocessed image blocks.
A copper surface defect detection method based on a line array camera according to claim 1 or 2, characterized in that the deep learning training process of the copper particle detection system in S3 includes:

S31. Perform defect annotation on the preprocessed image blocks with existing defect detection results to generate annotation images, and together form a copper particle defect data set;

S32. Construct a copper particle detection system using the convolutional neural network module and the pyramid pooling module;

S33. Repeat the image recognition training of the copper particle detection system using the copper particle defect data set;

S34. The training is completed when the copper particle identification pixel accuracy of the copper particle detection system reaches the specified value.
A copper surface defect detection method based on a line array camera according to claim 3, characterized in that the annotation information of the annotated image in the copper particle defect data set includes defect type, defect shape and size, copper particle color depth and particle aggregation.
A copper surface defect detection method based on a line array camera according to claim 3, characterized in that the image recognition training process includes:

Select a set of samples from the copper particle defect data set containing input data and theoretical output values;

Input the input data into the copper particle detection system to obtain the corresponding actual output value;

Compare and calculate the difference between the theoretical output value and the actual output value;

Adjust the parameters in the copper particle detection system in a manner that minimizes the error until the difference converges.
A copper surface defect detection method based on a line array camera according to claim 1 or 2 or 4 or 5, characterized in that in said S4, for the surface defect distribution map of the cathode copper plate, the defect parameters are set in advance The threshold separates the defective part of the image; extracts the regional features of the defective part of the image, and calculates the basic parameters of the defect based on the pixels in the area.
A copper surface defect detection device based on a line array camera, suitable for a method according to any one of claims 1 to 6, characterized in that it includes:

Transfer module, used to transfer cathode copper plates;

Line array camera acquisition module, including line array camera and light source, used to collect cathode copper plate surface images;

Industrial computer, used to receive the collected surface image data of the cathode copper plate and conduct defect identification and analysis;

The control module is used to receive the identification and analysis results of the industrial computer and control the work of the entire device.
A copper surface defect detection device based on a line array camera according to claim 7, characterized in that the line array camera acquisition module is provided with a light source and a line array camera in sequence according to the direction of transmission of the cathode copper plate; the light source is A certain incident angle illuminates the surface of the cathode copper plate, and the direction facing the lens of the line array camera is perpendicular to the surface of the cathode copper plate.