CN111754538B

CN111754538B - Threshold segmentation method for USB surface defect detection

Info

Publication number: CN111754538B
Application number: CN202010612002.1A
Authority: CN
Inventors: 杨将新; 曹彦鹏; 朱文斌; 曹衍龙; 徐正方; 牛旭; 张思杨
Original assignee: Hangzhou Xurui Machinery Co ltd; Zhejiang University ZJU
Current assignee: Hangzhou Xurui Machinery Co ltd; Zhejiang University ZJU
Priority date: 2019-06-29
Filing date: 2020-06-29
Publication date: 2022-07-29
Anticipated expiration: 2040-06-29
Also published as: CN111754538A

Abstract

The invention discloses a threshold segmentation method for USB surface defect detection, which comprises the steps of setting an initial threshold and update values of a foreground and a background, dividing an image into the foreground and the background according to a pixel gray value according to the initial threshold, introducing an average value of the foreground and the background in the last cycle to calculate the update threshold, and binarizing the image by using the threshold when the update threshold is equal to the initial threshold. The USB edge area and the deformation area after the image segmentation are separated, and the accuracy of USB area identification is improved. The USB edge area and the deformation area after the image segmentation are separated, and the accuracy of USB area identification is improved.

Description

Threshold segmentation method for USB surface defect detection

Technical Field

The invention relates to a method for image threshold segmentation.

Background

The background art is provided only to aid understanding of the technical contents and is not prior art.

The iterative method is also called rolling method, and is a process of continuously recursing new value by using old value of variable, and the iterative method is correspondent to the direct method (or called once-through solution method), i.e. once-through solution of problem. The iterative algorithm is a basic method for solving the problem by using a computer, and utilizes the characteristics of high operation speed and suitability for repetitive operation of the computer to make the computer repeatedly execute a group of instructions (or certain steps), and when the group of instructions (or the steps) are executed each time, a new value of the instructions is deduced from an original value of a variable.

Image segmentation is a crucial pre-processing of image recognition and computer vision. Without a correct segmentation, a correct identification is not possible. However, the only basis for segmentation is the brightness and color of the pixels in the image, and the segmentation is handled automatically by a computer, which has various difficulties. For example, segmentation errors often occur due to uneven lighting, the influence of noise, the presence of unclear portions in an image, shadows, and the like. Image segmentation is therefore a technique that requires further investigation. It is desirable to introduce some artificial knowledge-oriented and artificial intelligence methods for correcting errors in some segmentations, which are promising approaches, but which add complexity to the problem.

The key to the threshold segmentation algorithm is to determine the threshold value, so that the image can be accurately segmented if a suitable threshold value can be determined. After the threshold is determined, the threshold is compared with the gray value of the pixel one by one, pixel segmentation can be performed on each pixel in parallel, and the segmentation result is directly given to an image area.

The threshold segmentation has the advantages of simple calculation, higher operation efficiency and high speed. It is widely used in applications where computational efficiency is important (e.g., for hardware implementation).

In the process of updating the threshold value of the shot USB image by using an iteration method, the deformation defect of the USB can influence the extraction of the USB edge area, so that the USB edge area and the deformation area are connected into a whole, and therefore, how to separate the USB edge from the deformation area in the iteration process is an urgent problem to be solved.

Disclosure of Invention

The invention aims to provide an image threshold segmentation method which is used for calculating and updating a threshold by introducing an iterative method of the average value of a foreground and a background in the last cycle and separating a USB edge from a deformation area.

The invention is realized by adopting the following technical scheme:

the threshold segmentation method for USB surface defect detection is characterized by comprising the following steps: setting an initial threshold value and updating values of the foreground and the background, dividing the image into the foreground and the background according to the gray value of pixels according to the initial threshold value, introducing the average value of the foreground and the background in the last cycle to calculate the updating threshold value, and carrying out binarization on the image by using the threshold value when the updating threshold value is equal to the initial threshold value.

The method is used for partitioning the usb, the usb is a common structural component, so that the main surface of the usb needs to be extracted during appearance detection of the usb, the best method is used for extracting the surface of the usb better by utilizing the characteristics of the usb, and then the window area of the usb is found to be the most obvious and simpler characteristic area (the blue area of the lower image). We can extract the main working surface for usb with its window,

The iterative threshold selection algorithm is an improvement on the two-peak method. The classical iterative thresholding is based on the idea of approximation, and an approximate threshold To is selected first (generally, the average value of the maximum value Pmax and the minimum value Pmin of the gray value of the image pixel, that is, To ═ Pmax + Pmin)/2), the image is divided into two parts, IF and IB, the average values GF and GB of the regions IF and IB are calculated, then a new threshold Toc is selected as (GF + GB)/2, and IF Toc is not equal To, Toc is assigned To; the above process is repeated until Toc and To are equal, and the threshold Toc is the segmentation threshold of the image.

On the basis, the iterative method is modified to a certain extent, and the threshold value is not updated by the respective average values of the foreground and the background in the current loop. But the average values of the foreground and the background in the previous cycle are introduced, and the average values of the foreground and the background in the current cycle are combined to comprehensively update the threshold.

First, an approximate threshold To is selected (typically, the average of the maximum and minimum gray-level values Pmax and Pmin of the image pixels, that is, To ═ Pmax + Pmin)/2), and the updated average values of the initial foreground and background are GF0 ═ 0 and GB ═ 0, respectively. The image is divided into two parts, IF and IB, the mean values GF and GB of the regions IF and IB are calculated, and new foreground and background values GF + GF0 and GB + GB0 are selected, respectively. Toc is then made 3 × GF + GB/8, and if Toc is not equal To, Toc is assigned To; the above process is repeated until Toc and To are equal, and the threshold Toc is the segmentation threshold of the image.

Preferably, the method for calculating the update threshold by introducing the average value of the foreground and the background in the last cycle comprises the following steps:

1) finding the maximum value Pmax of the gray level of the image pixel after filtering and the minimum value Pmin of the gray level of the image pixel, calculating an initial threshold value To be (Pmax + Pmin)/2, setting a foreground initial update value GF0 To be 0, and setting a background initial update value GB0 To be 0;

2) dividing the image into two parts IF and IB according To the image pixel gray value by using a threshold value To, and calculating an average image pixel gray value GB of the average image pixel gray value GF and IB of the IF;

3) calculating the value of (GF + GF0)/2 to be assigned to GF0, and calculating the value of (GB + GB0)/2 to be assigned to GB 0;

4) calculating the updated threshold Toc ═ 3 × (GF0+ GB0)/8

5) When To is not equal To Toc, repeating steps 2-4 until To is equal To Toc, and outputting the value of the current threshold To

6) And dividing the image into two parts according To the gray value of the image pixel by using the current To value, wherein the gray value of the area containing the USB is set To be 255, and the gray value of the area not containing the USB is set To be 0. The invention has the beneficial effects that:

1. the USB edge area and the deformation area after the image segmentation are separated, and the accuracy of USB area identification is improved.

Drawings

Fig. 1 is a diagram illustrating the effect of the OSTU partitioning according to the present invention.

FIG. 2 is a diagram illustrating the effect of the closing operation of the present invention.

FIG. 3 is a graph of the threshold gray value inversion of the present invention.

Fig. 4 shows an original and a processed image according to the present invention.

FIG. 5 is a schematic drawing of the present invention.

FIG. 6 is a diagram of a center point of a window in a pixel coordinate system according to the present invention.

FIG. 7 is a diagram illustrating the effect of ROI truncation according to the present invention.

FIG. 8 is a diagram illustrating image processing according to the present invention.

FIG. 9 is a flow chart of image thresholding based on an improved iterative method according to the present invention

FIG. 10 is a flow chart of image thresholding by the classical iterative method of the present invention

FIG. 11 is a comparison graph of the image threshold segmentation effect of the classical iteration method and the improved iteration method of the present invention

Detailed Description

4) calculating the updated threshold Toc ═ 3 × (GF0+ GB0)/8

6) And dividing the image into two parts according To the gray value of the image pixel by using the current To value, wherein the gray value of the area containing the USB is set To be 255, and the gray value of the area not containing the USB is set To be 0.

Example 1

The data making part of the network training method comprises the following steps:

1.1.1 original Picture taking

The original picture (2448X2048) was taken using a hardware device, as shown in fig. 2.

1.1.2ROI Picture Generation

The software program was executed to crop out the ROI area (1000X800) as shown in fig. 3. The ROI data was divided into two data sets of train and test at a ratio of 7:3, with the specific number of data sets as shown in FIG. 3.

1.1.2 slice data Generation

For the training set and the test set, the software program is executed to cut out slice data (200X200) according to types, as shown in fig. 4: since a large amount of the cut-out slice data does not contain defects, the cut-out slice data needs to be selected and sorted, and the statistics of the number of the sorted-out slices is shown in fig. 4.

1.2 network training part

After preparing the slice data set, we need to rename the slice data set, store the path and the tag of the slice data set in two text files, and then generate an LMDB format that can be read by a caffe deep learning framework. Then, in a caffe deep learning framework, feature learning is carried out on the slice training data based on a SqueezeNet network, the model updates the weight and the bias parameters by reducing the loss between the label value and the predicted value, and the iteration number is set to 10000, so that a relatively optimal model is obtained. It should be noted here that the label value refers to the real classification property of the slice data, which is used to characterize the type of data, and the predicted value is the predicted classification property given by the squeezet network by learning the characteristics of the slice, and the network optimizes the model parameters by continuously reducing the error between the label value and the predicted value.

1.2.1LMDB data Generation

Here we specify the label values for the slice data set as shown in table 3.

Table 3 tag values assigned to different types of slice data

	Speckle	Deformation of	Scratch	Bright line	Embossing	Smudge	Is normal
								Tag value	5	1	4	0	2	5	3

And renaming, saving a path, placing a tag value in a text document and the like on the sliced data through a written matlab software program, and then processing the text document through a tool in a caffe frame to directly generate LMDB format data which can be read by cafe.

1.2.2SqueezeNet learning data characteristics, updating parameters, and generating optimal models

The SqueezeNet network is proposed in 2016, parameters of a model are greatly reduced while learning and classifying accuracy is not reduced, real-time performance and efficiency are greatly improved, and the method has obvious advantages for real-time detection of products on an industrial site.

The SqueezeNet network structure consists of a convolutional layer and a pooling layer at the beginning, a full-connection layer, a drop layer, a convolutional layer and a pooling layer at the end, and 8 identical fire module modules in the middle. The method comprises the steps of extracting features layer by layer through a convolution kernel, fitting the weight and the offset parameter of a network layer, continuously updating the parameter of each network layer in an iteration process by taking the minimum loss as a target, and finally generating a relatively optimal network model.

The network architecture used is shown in fig. 5.

Example 2

The neural network detection method for detecting the USB surface defects comprises the following steps:

step 1, thresholding image

The basic idea of the thresholding operation is to compare each pixel in the image with a given pixel value, and according to the comparison result, perform corresponding operations, for example, setting the gray value of the pixel less than the given pixel value to "0" or 255, and keeping the pixel greater than the given pixel value unchanged, i.e., completing the simplest image segmentation.

The Otsu method (OSTU) thresholding operation is an operation of performing image binarization by adopting an OSTU algorithm. The method divides the gray scale number of an image into 2 parts according to the gray scale by utilizing a maximum inter-class variance method and using a clustering idea, so that the gray scale difference between the two parts is maximum, the gray scale difference between each part is minimum, and a proper threshold value is searched through variance calculation for division. The image is then classified as foreground (white) or background (black) according to a threshold, but is not suitable for the case where the detection area is small.

Assuming that after the OSTU processing, the pixel ratio of the foreground region is w0, the average gray value is u0, the pixel ratio of the background region is w1, and the average gray value is u1, the average gray value of the whole processed image is:

u＝w 0 u 0 +w 1 u 1

The variance between the foreground region and the background region is

g＝w 0 (u 0 -u) 2 +w 1 (u 1 -u) 2 ＝w 0 w 1 (u 0 -u 1 ) 2

When a pixel value t in the traversal image is used as a threshold value, t with the largest g is the most appropriate threshold value, and the difference between the foreground and the background is considered to be the largest, namely the thresholding effect is the best.

The otsu algorithm is considered as the optimal algorithm for selecting the threshold value in image segmentation, is simple to calculate, and is not influenced by the brightness and the contrast of an image. Therefore, the use of the inter-class maximum variance method for segmentation means that the probability of false scores is minimal.

Step 2, morphological treatment

The mathematical morphology processing is an image analysis subject established in the basic knowledge of lattice theory and topology, and is a basic theory of the mathematical morphology processing. Dilation and erosion are fundamental operations of mathematical morphological processing.

The closed operation is the process of expansion followed by corrosion. Through expansion and corrosion, the area of the highlight area is increased and then reduced, black spots in the image are eliminated, and the smoothness of the image is realized.

Dilation is the operation of finding a local maximum. Mathematically, it is a process of convolving an image or a part of an image with a kernel. The kernel may be of any shape and size, with a reference point in the center, and may be generally square or circular. The operation is to convolve the kernel with the image, calculate the maximum value of the pixels in the area covered by the kernel, and copy the maximum value to the pixels specified by the reference point, which will make the highlight area in the image grow gradually.

The mathematical expression for dilation is:

d(x,y)＝max f(x+x',y+y') (0-1)

wherein x 'and y' are the horizontal and vertical coordinate values of the pixel point with (x, y) as the origin, and x 'and y' are the horizontal and vertical coordinate values of the pixel point in the kernel area.

While erosion is a pair of operations that is opposite to dilation, erosion is the operation that seeks the local minimum. The mathematical expression is as follows:

d(x,y)＝min f(x+x',y+y') (0-2)

The closed operation realizes the removal of black holes and the relative smoothness of the edge of the USB area, and divides the background area from the USB area.

Step 3, image pixel value inversion

The pixel value inversion is to invert the gray value of the thresholded image, i.e. the gray value of the foreground region is changed from 255 to 0, and the gray value of the background region is changed from 0 to 255.

Step 4, ROI region extraction method based on USB surface characteristics

As shown in fig. 4 below, the diagram (a) is an original diagram, and the green frame is an area to be detected, after filtering, image thresholding, mathematical morphology processing and image pixel value inversion, the diagram (b) is obtained, and the USB area and the background area are well divided, that is, the area of interest can be cut out from the filtered diagram according to the position and shape of the area of interest in the diagram.

The basic method and principle are as follows. Firstly, the USB plug is produced by an automatic production line, so the basic specification, shape and size of the USB are fixed values. The USB has two window areas, namely a yellow frame selection area in the picture (a), the position and the shape of the yellow frame selection area are fixed, and the central point of the yellow frame selection area can be searched. The position of the central point in the detection area also belongs to a fixed value, and the central point of the connecting line can be solved only by calculating the central points of the two window areas, the position of the whole detection area is calculated, the position of the detection area in the filtered image can be obtained, and the ROI area interception is carried out. Meanwhile, if the USB is placed correctly, i.e. the edge line and the frame line of the detection area are parallel, the abscissa of the two window areas should be the same. According to this condition, the USB which is placed in a skewed state can be corrected to be in a correct position.

First, two window regions are detected. Graph (b) is the image after filtering, thresholding, mathematical morphological operations and grey value inversion, each region being well-defined because the background and foreground regions have been separated. On the basis, all contours in the image are detected by using an opencv library function, as shown in fig. 5 below.

As can be seen from the figure, the USB contour is accurately detected, then the area of each contour is calculated, and the area of the window is actually measured, so that the window can be found out. In order to prevent the detection error caused by mistakenly considering other contours in the image as the window, the limiting conditions are increased, and the accuracy is improved. The area is obtained by detecting the near thousand USB outlines, and the conclusion is drawn, so the outline area of the window is between 35000 and 45000 pixels, namely the following formula is satisfied, namely the window is preliminarily determined.

35000<area<45000

Wherein, area is the area of the detected outline.

It is still possible to have other contour areas within this region, thus adding a constraint to calculate the rectangular rate of the contour.

The formula is as follows:

where rect _ area is the rectangular rate, area is the detected desired outline area, and s is the minimum external rectangular area of the outline. Through actual measurement, the rectangle rate of the window area is above 0.8, but the behavior rate of non-window outlines is below 0.8, so that the window outline and other outlines can be distinguished according to the method. The window is determined as further satisfying the following equation.

0.8<rect_rate

The condition that stains or other defects possibly similar to the shape of the window appear in the subsequent unknown product are not prevented from being detected as the window, a mathematical limitation condition is added, and if the window area is formed, the following formula is satisfied:

5800<G<8500

The range of G is the range obtained by actually measuring the last thousand pictures and calculating the value of G.

Therefore, two windows can be accurately found, and then the central points of the two windows are respectively (x 1, y 1) and (x 2, y 2), so that the image angle can be corrected. Let the coordinates of the connecting line of the two center points be (x 0, y 0), which is shown in fig. 6 under the pixel coordinate system, then the rotation angle is:

for USB manufacturing, the position of the central point (x 0, y 0) of the connecting line of the central points of the two windows in the image is a fixed value, so the distance from the central point to the boundary of the rectangular detection region is measured, and the specific position of the ROI region can be found.

The pixel sizes of the USB detection area are 11mm × 14mm, the lens magnification is 0.275, the CCD pixel size is 3.45 μm × 3.45 μm, and the pixel sizes of the detection area can be found as:

length:

width:

the center point is at a distance of 475 pixels from the upper boundary and 438 pixels from the left boundary. Therefore, coordinate values (x 0-438, y 0-475) of the upper left corner point of the detection area (rectangle) can be obtained, and the length and width of the rectangle are 877 and 1116 respectively. Since the detection is based on the filtered image, the ROI region can be segmented based on the filtered image, and the effect is shown in fig. 7 below. It can be seen that the image with inclined position has good correction effect, and the intercepted ROI area is still accurate.

The contents of the articles, patents, patent applications, and all other documents and electronically available information described or cited herein are hereby incorporated by reference in their entirety to the same extent as if each individual publication was specifically and individually indicated to be incorporated by reference. Applicants reserve the right to incorporate into this application any and all materials and information from any such articles, patents, patent applications, or other documents.

Claims

The threshold segmentation method for USB surface defect detection is characterized by comprising the following steps: setting an initial threshold value and updating values of the foreground and the background, dividing the image into the foreground and the background according to the gray value of pixels according to the initial threshold value, introducing the average value of the foreground and the background in the last cycle to calculate the updating threshold value, and binarizing the image by using the threshold value when the updating threshold value is equal to the initial threshold value;

the method for calculating the updating threshold value by introducing the average value of the foreground and the background in the last cycle comprises the following steps:

1) finding the maximum value Pmax of the gray level of the image pixel after filtering and the minimum value Pmin of the gray level of the image pixel, calculating an initial threshold value To = (Pmax + Pmin)/2, setting a foreground initial updated value GF0=0, and setting a background initial updated value GB0= 0;

2) dividing the image into two parts IF and IB according To the image pixel gray value by using a threshold value To, and calculating an average image pixel gray value GB of the average image pixel gray value GF and IB of the IF;

3) The value of (GF + GF 0)/2 is assigned to GF0, and the value of (GB + GB 0)/2 is assigned to GB 0;

4) calculate update threshold Toc =3 × (GF0+ GB 0)/8;

5) when To is not equal To Toc, assigning Toc To, and repeating the steps 2-4 until To is equal To Toc, and outputting the value of the current threshold To;

6) and dividing the image into two parts according To the gray value of the image pixel by using the current To value, wherein the gray value of the area containing the USB is set To be 255, and the gray value of the area not containing the USB is set To be 0.