WO2021000524A1

WO2021000524A1 - Hole protection cap detection method and apparatus, computer device and storage medium

Info

Publication number: WO2021000524A1
Application number: PCT/CN2019/125058
Authority: WO
Inventors: 戴志威; 邓远志; 陈润康; 林淼; 刘志永; 陈志列
Original assignee: 研祥智能科技股份有限公司
Priority date: 2019-07-03
Filing date: 2019-12-13
Publication date: 2021-01-07
Also published as: CN110930353A; CN110930353B

Abstract

A hole protection cap detection method, comprising: acquiring an image of a device to be detected, said device being provided with a hole having a hole protection cap; performing self-adaptive dynamic threshold segmentation on the image of said device, to determine a hole region; then extracting a greyscale feature subset of the hole region; projecting the greyscale feature subset onto a preset discriminant function line in a two-dimensional coordinate system, to acquire a corresponding projected longitudinal coordinate value; obtaining, according to a magnitude relationship between the longitudinal coordinate value and the threshold of the preset discriminant function line, a defect state of the device corresponding to said image. The threshold of the preset discriminant function line is obtained by the following method: on the basis of extracted greyscale feature subsets of a plurality of hole regions of a plurality of historical images of said device, constructing a training sample set containing at least two types of sample subsets, projecting various types of sample subsets into an optimal discrimination vector space, and calculating a corresponding projected horizontal coordinate mean value, to obtain the threshold of the preset discriminant function line.

Description

Method and device for detecting hole position protection door, computer equipment and storage medium

Cross references to related applications

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on July 3, 2019. The application number is 2019105938923, and the application title is "hole position protection door detection method, device, computer equipment and storage medium." Incorporated in this application by reference.

Technical field

This application relates to the technical field of equipment detection, and in particular to a detection method, device, computer equipment and storage medium for a hole position protection door.

Background technique

The protective door, also known as the safety door, is a kind of safety protective cover, which is generally set on the hole position. When the hole position is not used, it blocks the hole position and prevents foreign objects from entering the hole position.

In the manufacturing process of products with protective doors, it is necessary to detect whether the protective door of the product can be reset normally, and the detection of whether the protective door can be reset normally is usually through manual visual inspection to control the production quality of the product. Due to the different evaluation criteria of each person for manual detection, and the human sensory judgment is easily affected by subjective factors such as personal state and emotion, the efficiency of product detection is low and the false detection rate is high.

In order to improve the level of automation, some automatic detection methods for products equipped with hole protection doors have appeared on the market, but most of these hole protection door status detection methods are detected by a well-arranged assembly line tester or with the help of two plugs or The three-insertion protection door plug-in force detection mechanism, etc. perform detection. There are many interference factors in the operation of the detection instrument, which will cause the current hole position protection door state detection method to have the problem of unstable detection results and low detection efficiency.

Summary of the invention

According to various embodiments disclosed in this application, a method, device, computer equipment and storage medium for detecting a hole position protection door are provided.

A method for detecting hole position protection door includes:

Obtain the image of the device to be tested, the device to be tested is provided with a hole position, and the hole position is provided with a hole position protection door;

Perform adaptive dynamic threshold segmentation based on local statistics on the image of the equipment to be tested to determine the hole location area;

Based on the statistical algorithm of the gray histogram, extract the gray feature subset of the hole location area;

Project the gray feature subset onto the preset discriminant function line in the two-dimensional coordinate system to obtain the projected ordinate value of the gray feature subset. The preset discriminant function line is such that the gray feature subset is A straight line with the largest inter-class distance and the smallest intra-class distance in the subspace; and

Obtain the defect state of the device corresponding to the image of the device to be tested according to the magnitude relationship between the ordinate value after the projection of the gray feature subset and the preset threshold of the discriminant function line;

The preset threshold of the straight line of the discriminant function is obtained in the following way:

Obtain multiple historical images of the equipment to be tested;

Perform adaptive dynamic threshold segmentation based on local statistics on the historical image of the equipment to be tested to determine the hole location area;

Based on the statistical algorithm of the gray histogram, extract the gray feature subsets of multiple hole locations, and construct a training sample set, which includes at least two types of sample subsets; and

Project various sample subsets to the best discriminant vector space, calculate the mean value of the abscissa after the projection of the various sample subsets, and get the preset threshold of the discriminant function straight line. The best discriminant vector space is based on the Fisher linear discriminant analysis algorithm And the training sample set, introducing the best projection direction solved by Lagrangian multiplier method.

A device for detecting the status of a hole position protection door includes:

The image acquisition module is used to read the image of the equipment to be tested and acquire the image of the equipment to be tested. The equipment to be tested is provided with a hole position and a hole position protection door is arranged on the hole position;

The hole location area acquisition module is used to perform adaptive dynamic threshold segmentation based on local statistics on the image of the equipment to be detected to determine the hole location area;

The gray-level feature extraction module is used to extract the gray-level feature subset of the hole location area based on the statistical algorithm of the gray-level histogram;

The projection module is used to project the gray feature subset onto the preset discriminant function line in the two-dimensional coordinate system, and calculate the projected ordinate value of the gray feature subset. The preset discriminant function line is such that the gray feature Set the straight line with the largest inter-class distance and the smallest intra-class distance in the projected subspace;

The defect detection module is used to obtain the defect state of the device corresponding to the image of the device to be detected according to the magnitude relationship between the ordinate value after the projection of the gray feature subset and the threshold value of the preset discriminant function line; and

The preset threshold calculation module is used to obtain multiple historical images of the equipment to be inspected, perform adaptive dynamic threshold segmentation based on local statistics on the historical images of the equipment to be inspected, determine the hole location area, and extract the historical image based on the gray histogram Construct a training sample set of gray feature subsets of multiple hole locations. The training sample set includes at least two types of sample subsets. Project various sample subsets to the best discriminant vector space, and calculate the projections of various sample subsets. After the mean value of the abscissa, the preset threshold of the discriminant function line is obtained. The best discriminant vector space is the best projection direction based on the Fisher linear discriminant analysis algorithm and training sample set, and the Lagrange multiplier method is introduced.

A computer device, including a memory and one or more processors. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the method for detecting the status of the hole position protection door provided in any embodiment of the present application is implemented A step of.

One or more non-volatile storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors implement the holes provided in any of the embodiments of the present application. The steps of the method for detecting the status of the bit protection gate.

The details of one or more embodiments of the application are set forth in the following drawings and description. Other features and advantages of this application will become apparent from the description, drawings and claims.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Fig. 1 is an application scene diagram of the method for detecting the state of the hole position protection door according to one or more embodiments.

Fig. 2 is a schematic flowchart of a method for detecting the status of a hole position protection door according to one or more embodiments.

Fig. 3 is a diagram of an application environment for collecting images of a device to be tested according to one or more embodiments.

FIG. 4 is a schematic flow chart of the step of obtaining the threshold value of the preset discriminant function straight line in another embodiment.

Fig. 5 is a schematic flowchart of a sub-step of hole location area acquisition in another embodiment.

Fig. 6 is a block diagram of a device for detecting the status of a hole position protection door according to one or more embodiments.

Fig. 7 is a structural diagram of a hole position protection door state detection device in another embodiment.

Figure 8 is a block diagram of a computer device according to one or more embodiments.

Detailed ways

In order to make the technical solutions and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

The method for detecting the status of the hole position protection door provided in this application can be applied to the application environment diagram shown in FIG. 1. The server 120 and the camera 103 communicate through a network. The server 120 obtains the image of the device to be tested uploaded by the camera 103, performs adaptive dynamic threshold segmentation based on local statistics on the image of the device to be tested, and determines the hole location area (the hole location area includes a set of suspicious defect areas), based on the gray histogram Based on the statistical algorithm, extract the gray feature subset of the hole location area, project the gray feature subset onto the preset discriminant function line in the two-dimensional coordinate system, and obtain the projected ordinate value of the gray feature subset. Suppose the discriminant function line is the line that makes the gray feature subset have the largest inter-class distance and the smallest intra-class distance in the projected subspace, according to the projected ordinate value of the gray feature subset and the preset discriminant function line The relationship between the magnitude of the threshold and the defect state of the device corresponding to the image of the device to be detected is obtained. The camera 103 can be, but is not limited to, a camera, a smartphone with a camera function, a camera device, etc. The server 120 can be implemented by an independent server or a server cluster composed of multiple servers.

In one of the embodiments, as shown in FIG. 2, a method for detecting the status of a hole position protection door is provided. Taking the method applied to the server in FIG. 1 as an example, the method includes the following steps:

In step S100, an image of a device to be detected is acquired. The device to be detected is provided with a hole position and a hole position protection door is provided on the hole position.

The image of the equipment to be inspected is the image of the equipment to be inspected with the hole position and the hole position protection door collected by the camera 103. The image information includes the hole position area image, where the hole position area image includes the protection door image. Take the socket as an example of the device to be tested. The image of the device to be tested includes the image of the jack area and the image of the protective door on the jack. As shown in Figure 3, in conjunction with Figure 1, in practical applications, the image of the device to be tested can be acquired when the device to be tested 101 on the assembly line (the side with the hole position is up, and the hole position is provided with hole position protection Door) When passing the photoelectric sensor 105, the photoelectric sensor 105 sends the trigger signal to the industrial computer 104. After receiving the signal, the industrial computer 104 outputs the control signal through the IO board. First, turn on the ring-shaped shadowless light source directly above the device to be tested 101 , And then send a soft trigger signal to the camera 103 to collect the image of the device to be detected 101, and then the camera 103 uploads the collected image to the server 120, and the server 120 obtains the image of the device to be detected.

Step S200: Perform adaptive dynamic threshold segmentation based on local statistics on the image of the device to be detected to determine the hole location area.

Through the image of the equipment to be tested collected by the camera device, it can be found that the hole location area in the image and the front of the equipment to be tested (the side with the hole location is defined as the front) has obvious gray difference, and the contrast is more obvious. , Can first extract the area of interest on the front side to be tested by threshold segmentation, that is, all the hole area to be detected on the front side and the protection door area. Due to the complex operating environment of the factory and the need to meet the detection of multiple color equipment to be inspected at the same time, this application uses an adaptive dynamic threshold segmentation method based on local statistics to extract the hole location area, where the hole location area contains suspicious defects. set. Local threshold segmentation is a common method to deal with uneven brightness distribution, and it has a good segmentation effect in some shadow areas with insufficient light. Suppose the original image of the device to be tested is f(x,y), and the image obtained by smoothing filter processing on the original image is set to m(x,y). The filter can be selected as mean filter, binomial filter and Gaussian filter, etc. Set the offset to Offset, compare the pixel gray values of f(x,y) and m(x,y) one by one. When the target to be segmented is brighter than the background, satisfy f(x,y)≥m The pixel position of (x,y)+Offset is output as 1; when the target to be segmented is darker than the background, the pixel position that satisfies f(x,y)≤m(x,y)-Offset is output as 1; When the segmented target may be brighter than the background or darker than the background area, the above two conditions need to be met at the same time. On the basis of local threshold segmentation, this application proposes an adaptive dynamic threshold segmentation method based on local statistics. Based on the average information of the window, the offset Offset is corrected by adding the statistical information of the standard deviation of the window.

Step S300, based on the statistical algorithm of the gray histogram, extract the gray feature subset of the hole location area.

In this embodiment, the gray level feature of the hole location area is extracted based on the statistical characteristics of the gray histogram statistical algorithm. For a two-dimensional image with a total number of pixels N, its gray level is L, and the total number of pixels of the same gray level is n(i), then the first-level grayscale histogram can be defined as:

The gray level histogram can generally reflect the overall gray level and gray distribution probability of the hole location area. According to the statistical characteristics of the gray histogram, the following gray features can be extracted:

(1) The mean value m, the average gray scale, reflects the overall gray level of the image.

(2) Standard deviation σ, a measure of average contrast, which reflects the degree of dispersion of the gray histogram.

After calculating the gray average value and standard deviation of the hole location area, construct the gray feature subset T={(m _T ,σ _T )} according to the gray average and standard deviation, and use the gray feature subset T as the test sample set.

In one of the embodiments, before extracting the gray-scale feature subset of the hole location area, the method further includes: performing smooth filtering processing on the hole location area according to a preset filter to blur the texture information in the hole location area.

In this embodiment, the preset filter is an average filter with a size of 3*3, and the hole position area set D is smoothed and filtered, which can blur part of the texture information of the hole position protection gate in the image, and exclude the texture part of the area. Interference of gray standard deviation. It can be understood that, in other embodiments, the average filter may also have other sizes, and the filter may be a binomial filter, a Gaussian filter, or the like.

Step S400: Project the gray feature subset onto a preset discriminant function line in the two-dimensional coordinate system, and obtain the projected ordinate value of the gray feature subset. The preset discriminant function line is such that the gray feature subset is in There is a straight line with the largest inter-class distance and the smallest intra-class distance in the projected subspace.

In practical applications, the staff assumes that there is a straight line (discrimination function straight line) w in the two-dimensional coordinate system space, so that after all feature points in this space are projected onto this straight line, the intra-class variance is the smallest and the inter-class variance is the largest. Then, it is necessary to calculate the threshold value of the discriminant function line, and then project the gray feature subset onto the discriminant function line to obtain the projected ordinate value of the gray feature subset on the line. Then, it can be the defect The state detection is transformed into a two-class problem. Based on the relationship between the threshold value of the discriminant function line and the projected ordinate value, the defect state of the protection door of the equipment to be tested is obtained.

In one of the embodiments, as shown in FIG. 4, the threshold of the preset discriminant function straight line is obtained in the following manner:

Step S402, acquiring multiple historical images of the equipment to be tested;

Step S404: Perform adaptive dynamic threshold segmentation based on local statistics on the historical image of the device to be detected to determine the hole location area;

Step S406, based on the statistical algorithm of the gray histogram, extract the gray feature subsets of multiple hole location regions, and construct a training sample set, the training sample set includes at least two types of sample subsets;

Step S408: Project various sample subsets into the optimal discriminative vector space, and calculate the mean values of abscissas after the projection of the various sample subsets to obtain the preset discriminant function straight line threshold. The optimal discriminant vector space is based on Fisher linear Discriminant analysis algorithm and training sample set, introducing the best projection direction solved by Lagrangian multiplier method.

Specifically, it can read 50 historical images of the equipment to be tested, extract the hole location area in the historical image of the equipment to be tested as a training sample set, and divide it into two types of subsets, including 25 qualified hole locations. Degree feature subset X ₀ and 25 unqualified hole location regions gray feature subset X ₁ :

X ₀ ＝{(m ₁ ,σ ₁ ),(m ₂ ,σ ₂ ),(m ₃ ,σ ₃ ),…,(m ₂₅ ,σ ₂₅ )}

X ₁ ={(m ₂₆ ,σ ₂₆ ),(m ₂₇ ,σ ₂₇ ),(m ₂₈ ,σ ₂₈ ),…,(m ₅₀ ,σ ₅₀ )}

Calculate the mean vector of various samples

Calculate the covariance matrix of various samples

Define the projection of the centers of the two types of samples on a straight line as: w ^T u ₀ and w ^T u ₁ , the covariance of the two types of samples: w ^T C ₀ w and w ^T C ₁ w, which are the projection points of the samples of the same kind As close as possible, that is, it is necessary to ensure that w ^T C ₀ w+w ^T C ₁ w is as small as possible, in order to make the projection points of heterogeneous samples as far away as possible, that is, to ensure

As large as possible, so maximize

Calculate the intra-class divergence matrix:

S _w ＝C ₀ +C ₁

Calculate the divergence matrix between classes:

S _b ＝(u ₀ -u ₁ )(u ₀ -u ₁ ) ^T

At this time, J can be rewritten as:

Let w ^T S _w w = 1, the equivalent form of maximizing generalized Rayleigh quotient is:

stw ^T S _w w = 1

Using Lagrangian multiplier method, then S _b w = λS _w w; the direction of S _b w is always u ₀ -u ₁ , so let S _b w = λ(u ₀ -u ₁ ), so we can get

Calculate the inverse of the matrix S _w

Then calculate the eigenvector corresponding to the maximum eigenvalue of w. The eigenvector is the projection direction W, and W is a one-dimensional column vector. At this time, the slope of the best projection line k=W(2)/W(1) can be calculated, where , W(2) is the second element value in the one-dimensional column vector, W(1) is the first element value in the one-dimensional column vector, and the final discriminant function is y=kx+b; in this application, b=0; Calculate the projection coordinates of all gray feature subset points in the training sample set on the straight line y=kx+b, according to the point-to-straight projection formula:

Among them, n = 1, 2, 3,..., 50; calculate the mean value of the abscissa of the points after the projection transformation of the two types of subsets, namely:

At this time, the threshold of the discriminant function line

Among them, k=W(2)/W(1).

Step S500: Obtain the defect state of the device corresponding to the image of the device to be tested according to the magnitude relationship between the ordinate value after the projection of the gray feature subset and the threshold value of the preset discriminant function line.

As described in the above embodiment, when the line of the discriminant function is determined, the gray feature subset of the test sample set, namely T={(m _T ,σ _T )}, is projected onto the line of the discriminant function to obtain the gray feature The coordinate value after projection of the subset is:

In this embodiment, detecting the status of the protection door in the hole position is to detect the reset and no reset status of the protection door. Therefore, it is a two-class problem, and its output mark Result ∈ {0, 1} can be recorded. In this embodiment, the classification rules are used to compare the value of y _T with the threshold w ₀ to obtain the classification. The classification rules are as follows:

That is, if the Result output is equal to 0, it means that the protection door in this hole position is not reset; if the Result output is equal to 1, it means that the protection door in this hole position is reset.

The above hole position protection door status detection method, from the perspective of image recognition, first obtains the image of the device to be tested to construct a training sample set, obtains the preset threshold of the discriminant function line according to the training sample set, and then obtains the image of the device to be tested. The adaptive dynamic threshold segmentation based on local statistics can accurately filter out the hole location area. By projecting the gray-scale feature of the extracted hole location area to the preset discriminant function line, it can extract the classification information and compress the feature space dimension. After the projection, the feature subset has the largest inter-class distance and the smallest intra-class distance in the projected subspace to ensure the best separability of the sample in the space. Finally, compare the projected feature subsets The ordinate value of and the threshold value of the preset discriminant function straight line can determine whether the device under test has defects. The above method is convenient, effective, and accurate, and has strong anti-interference ability and stable detection results.

As shown in FIG. 5, in one of the embodiments, performing adaptive dynamic threshold segmentation based on local statistics on the image of the device to be detected, and determining the hole location area includes:

Step S202, according to a preset adaptive dynamic threshold segmentation method based on local statistics, threshold segmentation is performed on the image of the equipment to be inspected to obtain a set of defective pixels;

Step S204: Perform connected domain analysis on the set of defective pixels according to the connection operator to obtain a set of suspicious defective regions;

Step S206: According to the preset hole location area constraint condition and the preset hole location bulkiness, the hole location area is screened out from the set of suspicious defect areas.

The connected area generally refers to an image area composed of foreground pixels that have the same pixel value and are located adjacent to each other. Connected area analysis refers to finding and marking each connected area in an image. In this embodiment, after adding the standard deviation of the filter window to correct the offset Offset, it may be based on the offset Offset, the pixel average of all pixels in the filter window, the preset standard deviation scale factor and the minimum absolute threshold. , Perform threshold segmentation on the image of the equipment to be inspected to segment the set of defect points. The defect points can be regarded as independent small areas. After the defect points are segmented, the connection operator can be used to analyze the connected domain of the defective pixel point set , Get the suspicious defect area set, the so-called suspicious defect area set can be understood as the hole location area that may have defects. Since the size of the hole area of the device to be tested is the same and fixed, the hole area can be further screened by the area characteristics of the hole. Specifically, the area of the pixel passing through the hole may be used as a constraint condition. Take the socket as an example for the equipment to be tested. The socket includes a two-hole socket and a three-hole socket. Generally, the physical size of the socket of the two-hole socket is 0.9cm*0.9cm, and the physical size of the hole of the three-hole socket is 0.9cm*0.2cm. From an image perspective, in the collected images of the equipment to be tested, the area of each of the three holes is about 14,000 pixels, and the area of each of the two holes is about 72,800 pixels. Therefore, in this embodiment, setting the pixel area of the hole position within the two ranges of [13000, 5000] and [70000, 75000] is a constraint condition. Based on this, a set of regions containing hole positions can be filtered from a set of suspicious defect areas . For the segmented binarized image of the equipment to be tested, the area S of the hole location area can be expressed by the following formula:

Specifically, it is assumed that the set of all regions that meet the constraint conditions is expressed as: A={S _i |13000<S _i ＜15000∪70000<S _i ＜75000, S _i εS}. If only a single area feature is used for screening, it is likely to judge the non-hole area as a hole area. Therefore, it is proposed to add the appearance and shape feature of the hole location, that is, the bulkiness of the hole location area as an additional screening condition. Bulkiness:

Bulkiness=π*R _a *R _b /A _i

Where Ra and Rb represent the minimum circumscribed ellipse radius, and A represents the area of the region. When the bulkiness is closer to 1, the smallest circumscribed ellipse representing this area is closer to a circle.

Taking the socket as an example of the equipment to be tested, the bulkiness of the jack area is about 1.05 calculated through statistical analysis of multiple experimental samples, which is obviously different from the bulkiness value of other areas. Therefore, in this embodiment, setting the hole bulkiness in the range of [1.04, 1.06] is the constraint condition, and finally the set D of all hole location regions can be filtered out:

D={A _i |1.04＜π*R _a *R _b /A _i ＜1.06, A _i ∈A}

It is understandable that in addition to the connection operator, other connected domain analysis algorithms can also be used in the connected domain labeling method. The constraint condition of the bulkiness can be the constraint condition of other values, and the specific conditions may be determined. In this embodiment, the preset hole position bulkiness constraint condition can improve the accuracy of hole position area screening.

In one of the embodiments, according to a preset adaptive dynamic threshold segmentation method based on local statistics, threshold segmentation of the image of the device to be inspected to obtain a set of defective pixels includes: based on a preset filter, the image of the device to be inspected Perform smoothing filter processing, calculate the local standard deviation of the image of the device to be tested according to the average value of the pixel gray in the window of the preset filter, according to the local standard deviation, the preset standard deviation scale factor and the preset minimum absolute threshold , Calculate the correction offset of each pixel, obtain the gray value of each pixel in the image of the device to be tested and the average pixel gray value of the template neighborhood of each pixel, according to the gray value of each pixel, each pixel The pixel gray average value of the template neighborhood of the pixel point and the corrected offset of each pixel point are obtained to obtain the defective pixel point set.

In this embodiment, the preset filter can be a mean filter, a binomial filter, a Gaussian filter, etc. The window size of the filter is 150 pixels * 150 pixels, and the local standard deviation is the standard deviation statistics of the filter window. Information, it reflects the changes in the contrast of a local area in an image. Suppose f(i,j) is the gray value of the pixel at the position (i,j) in the image F of the device to be tested, the image size is M×N, W is centered on (i,j), and the size is l× The window of l, l is odd and l>1, W is regarded as a local area in the image, and its local standard deviation Dev _{ij is} calculated, which is defined as follows:

In the formula, i, j≥0, m≤M-1, n≤N-1, sqrt() is the square root operation. Let g(x,y) be the gray value of the pixel at (x,y) of the input image of the device to be tested, m(x,y) is the average gray value information in the neighborhood of the template, d(x,y ) Is the local standard deviation, then the statistical correction offset of the pixel is:

In the formula, std is the standard deviation scale factor, and T _abs is the minimum absolute threshold. In this embodiment, the standard deviation scale factor std is set to 1.36, and the minimum absolute threshold of T _abs is 20. According to the average value m(x, y) and the offset correction amount Offset, a set D of all defective pixels is segmented from the image of the device to be tested through a threshold. The set D can be expressed as:

D={(x,y)|g(x,y)<m(x,y)-Offset∪g(x,y)>m(x,y)+Offset}

It can be understood that in other embodiments, std and T _abs may also be other values, which may be determined according to actual conditions. In this embodiment, the offset Offset is corrected by calculating the local standard deviation of the filter window, which can limit the reasonable change range of the gray value, avoid the defects in small areas from being excessively smoothed and lead to incorrect segmentation, and improve the integrity of defect segmentation .

In one of the embodiments, calculating the local standard deviation of the image of the device to be tested according to the average value of the pixel gray levels in the preset filter window includes: obtaining the gray value of the central pixel point in the window of the preset filter The gray average value of all pixels in the window of the preset filter is calculated based on the gray value of the center pixel and the gray average value to calculate the local standard deviation.

If

It is the average gray value of all pixels in window W, and the gray value of the central pixel in the window of the preset filter can be defined

as follows:

In the formula, i, j≥0, m≤M-1, n≤N-1. In this embodiment, the local standard deviation is calculated based on the pixel gray average value and the center pixel gray value, which can reduce the sensitivity in detecting the edge of the isolated point, and has a certain smoothing filtering effect.

In one of the embodiments, after obtaining the defect state of the device corresponding to the image of the device to be inspected, the method further includes: extracting the edge contour of the hole region according to the gen_contour_region_xld operator, marking the edge contour according to the defect state, and pushing the marked edge Contour image.

In practical applications, the processing of the image of the device to be tested can be based on the Halcon 17.12 progress platform. To facilitate the detection by the staff, the detection result of the image of the device to be tested can be displayed on the software interface. Specifically, it may be based on the gen_contour_region_xld operator to extract the edge contour of the hole location region. The gen_contour_region_xld operator is a type of Halcon operator function, which is used to create an XLD contour (contour) according to the region. In this embodiment, when the output of Result (defect status result) is 0, it means that the protection door in the jack has not been reset. At this time, the edge contour of the hole position is marked with a red outline, if Result (defect status result) If the output is equal to 1, it means that the protection door in this hole position is reset, and the edge contour of the hole position is marked with a green outline, so that the staff can judge whether the protection door of the equipment to be tested is reset by the color of the displayed hole position outline , Convenient and effective. It can be understood that, in other embodiments, the edge contour of the hole position may also be marked with other colors, or marked by other methods that can distinguish whether the hole position protection door is reset.

In order to clearly describe the method for detecting the status of the hole protection door provided by the present application, a specific example and FIG. 2 and FIG. 3 are combined for description. The equipment to be tested 101 is an example of a socket, and the imaging device 103 is an example of a camera:

When the device to be tested 101 on the assembly line facing up, that is, the socket (the side of the socket with the jack is the front), passes the photoelectric sensor 105, the photoelectric sensor 105 sends a trigger signal to the industrial computer 104, and the industrial computer 104 receives the signal Output the control signal through the IO board, first turn on the ring shadowless light source directly above the socket, and then send a soft trigger signal to the camera 103 (the camera model is BasleracA2500-14gm, the horizontal/vertical resolution is 2592pixel*1944pixel), and the The front image of the socket, and then the camera 103 uploads the collected image to the server, and the server obtains the socket image. Based on the Halcon 17.12 progress platform, the socket image is segmented by an adaptive dynamic threshold based on local statistics to obtain the socket area. Among them, the jack area includes the pixel area constraints and bulkiness requirements based on the jack, that is, the pixel area of the jack is between [13000, 15000] or [70000, 75000], and the jack bulk is [1.04, 1.06] ] Range. Then, based on the statistical algorithm of the gray histogram, the gray feature subset T={(m _T ,σ _T )} of the jack area is extracted, and the gray feature subset T={(m _T ,σ _T ) } projected two-dimensional coordinate system preset on the discriminant function straight line y = kx + b, obtaining projection gradation ordinate value feature subset y _T, set the value of ordinate y _T projection with a predetermined gradation characteristic according to the sub- Determine the relationship between the threshold value w ₀ of the straight line of the discriminant function, and obtain the defect status of the socket corresponding to the socket image. If y _T ≤w ₀ , the output is equal to 0, which means that the protection door in the socket is not reset, and the edge of the socket is marked in red Contour; if y _T ＞w ₀ , the output is equal to 1, which means that the protective door in the hole is reset, and the contour of the edge of the jack is marked in green.

It should be understood that although the steps in the flowcharts of FIGS. 2, 4, and 5 are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless specifically stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least part of the steps in Figure 2, Figure 4, and Figure 5 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The order of execution of the sub-steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with other steps or at least a part of the sub-steps or stages of other steps.

In one of the embodiments, as shown in FIG. 6, a device for detecting the status of a hole position protection door is provided, which includes: an image acquisition module 610, a hole position area acquisition module 620, a gray feature extraction module 630, a projection module 640, The defect detection module 650 and the preset threshold calculation module 660, wherein:

The image acquisition module 610 is used to read the image of the device to be tested and acquire the image of the device to be tested. The device to be tested is provided with a hole position and a hole position protection door is arranged on the hole position.

The hole location area acquisition module 620 is configured to perform adaptive dynamic threshold segmentation based on local statistics on the image of the device to be detected to determine the hole location area.

The gray-level feature extraction module 630 is used to extract a gray-level feature subset of the hole location area based on a statistical algorithm of the gray-level histogram.

The projection module 640 is used to project the gray feature subset onto the preset discriminant function line in the two-dimensional coordinate system, and calculate the projection ordinate value of the gray feature subset. The preset discriminant function line is such that the gray feature The subset has a straight line with the largest inter-class distance and the smallest intra-class distance in the projected subspace.

The defect detection module 650 is configured to obtain the defect state of the device corresponding to the image of the device to be detected according to the magnitude relationship between the ordinate value after the projection of the gray feature subset and the threshold of the preset discriminant function line.

The preset threshold calculation module 660 is used to obtain multiple historical images of the equipment to be tested, perform adaptive dynamic threshold segmentation based on local statistics on the historical images of the equipment to be tested, determine the hole location area, and use a statistical algorithm based on gray histograms, Extract the gray-scale feature subsets of multiple hole locations, construct a training sample set, the training sample set includes at least two types of sample subsets, project various sample subsets to the best discriminant vector space, and calculate various sample subsets respectively The average value of the abscissa after projection is used to obtain the preset threshold of the discriminant function line. The best discriminant vector space is the best projection direction based on the Fisher linear discriminant analysis algorithm and training sample set, and the Lagrange multiplier method is introduced.

As shown in FIG. 7, in one of the embodiments, the hole position protection door state detection device further includes a mark pushing module 670, which is used to extract the edge contour of the hole position region according to the gen_contour_region_xld operator, and mark the edge contour according to the defect state, Push the image of the edge contour after marking.

In one of the embodiments, the device for detecting the status of the hole protection door further includes a smoothing filter module 680, configured to perform smoothing filtering processing on the hole location area according to a preset filter to blur the texture information in the hole location area.

In one of the embodiments, the hole location area acquisition module 620 is further configured to perform threshold segmentation on the image of the device to be tested according to a preset adaptive dynamic threshold segmentation method based on local statistics to obtain a set of defective pixels, according to the connection operator , Perform connected domain analysis on the set of defective pixels to obtain a set of suspicious defect areas. According to preset hole area constraint conditions and preset hole position bulkiness, the hole location area is screened out from the set of suspicious defect areas.

In one of the embodiments, the hole location area acquisition module 620 is further configured to perform smoothing filtering processing on the image of the device to be detected based on a preset filter, and calculate the average value of the pixel gray level in the window of the preset filter. The local standard deviation of the image of the detection device is calculated, according to the local standard deviation, the preset standard deviation scale factor and the preset minimum absolute threshold, the corrected offset of each pixel is calculated, and the value of each pixel in the image of the device to be detected is obtained. The gray value and the average pixel gray value of each pixel’s template neighborhood are based on the gray value of each pixel, the average pixel gray value of each pixel’s template neighborhood, and the correction offset of each pixel. Obtain a collection of defective pixels.

In one of the embodiments, the hole location area obtaining module 620 is further configured to obtain the gray value average value of all pixels in the window of the preset filter according to the gray value of the center pixel in the window of the preset filter. Calculate the local standard deviation based on the gray value of the central pixel and the average gray value.

For the specific definition of the hole position protection door state detection device, please refer to the above definition of the hole position protection door state detection method, which will not be repeated here. Each module in the above-mentioned hole position protection door status detection device can be implemented in whole or in part by software, hardware and a combination thereof. The foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.

In one of the embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 8. The computer equipment includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. The computer equipment database is used to store data such as images of the equipment to be tested. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer readable instruction is executed by the processor, a method for detecting the status of the hole position protection door is realized.

Those skilled in the art can understand that the structure shown in FIG. 8 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

A computer device includes a memory and one or more processors. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the one or more processors execute any one of the embodiments of the present application. The steps of the hole position protection door status detection method.

One or more non-volatile storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the holes provided in any embodiment of the present application. The steps of the method for detecting the status of the bit protection gate.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through computer-readable instructions, which can be stored in a non-volatile computer. In a readable storage medium, when the computer-readable instructions are executed, they may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction between the combinations of these technical features, they should It is considered as the range described in this specification.

The above-mentioned embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can be made, and these all fall within the protection scope of this application. Therefore, the scope of protection of the patent of this application shall be subject to the appended claims.

Claims

A method for detecting the status of a hole position protection door, the method comprising:

Acquiring an image of the device to be tested, the device to be tested is provided with a hole position, and the hole position is provided with a hole position protection door;

Performing adaptive dynamic threshold segmentation based on local statistics on the image of the device to be detected to determine the hole location area;

Based on the statistical algorithm of the gray histogram, extract the gray feature subset of the hole location area;

Project the gray-scale feature subset onto a preset discriminant function line in a two-dimensional coordinate system to obtain the projected ordinate value of the gray-scale feature subset, and the preset discriminant function line is such that the gray The degree feature subset has a straight line with the largest inter-class distance and the smallest intra-class distance in the projected subspace; and

Obtaining the defect state of the device corresponding to the image of the device to be tested according to the magnitude relationship between the ordinate value after the projection of the gray-scale feature subset and the preset threshold of the discriminant function line;

The preset threshold of the discriminant function straight line is obtained in the following manner:

Obtain multiple historical images of the equipment to be tested;

Perform adaptive dynamic threshold segmentation based on local statistics on the historical image of the equipment to be tested to determine the hole location area;

Based on the statistical algorithm of the gray histogram, extract the gray feature subsets of multiple hole location regions to construct a training sample set, the training sample set includes at least two types of sample subsets; and

Project various sample subsets into the optimal discriminative vector space, and calculate the mean values of the abscissas after the projection of the various sample subsets to obtain the preset threshold value of the discriminant function straight line, and the optimal discriminant vector space is Based on the Fisher linear discriminant analysis algorithm and the training sample set, the optimal projection direction obtained by the Lagrangian multiplier method is introduced.
The method for detecting hole position protection door status according to claim 1, wherein said performing adaptive dynamic threshold segmentation based on local statistics on the image of said device to be detected, and determining the hole position area comprises:

Perform threshold segmentation on the image of the device to be tested according to a preset adaptive dynamic threshold segmentation method based on local statistics to obtain a set of defective pixels;

According to the connection operator, perform connected domain analysis on the set of defective pixels to obtain a set of suspicious defective regions; and

According to preset hole location area constraints and preset hole location bulkiness, the hole location area is screened out from the set of suspicious defect areas.
The hole position protection door status detection method according to claim 2, wherein the image of the device to be detected is thresholded according to a preset adaptive dynamic threshold segmentation method based on local statistics to obtain defects The pixel point collection includes:

Performing smoothing filtering processing on the image of the device to be detected based on the preset filter, and calculating the local standard deviation of the image of the device to be detected according to the average value of the pixel gray levels in the window of the preset filter;

Calculating the correction offset of each pixel according to the local standard deviation, the preset standard deviation scale factor and the preset minimum absolute threshold;

Acquiring the gray value of each pixel in the image of the device to be tested and the average value of the pixel gray in the template neighborhood of each pixel; and

The set of defective pixels is obtained according to the gray value of each pixel, the average value of the pixel gray of the template neighborhood of each pixel, and the corrected offset of each pixel.
The method for detecting the status of the hole protection door according to claim 3, wherein the calculating the local standard deviation of the image of the device to be detected according to the average value of the pixel gray levels in the preset filter window comprises :

Obtaining the average gray value of all pixels in the window of the preset filter according to the gray value of the central pixel in the window of the preset filter; and

The local standard deviation is calculated according to the gray value of the central pixel point and the gray average value.
The method for detecting the status of the hole position protection door according to claim 1, wherein before said extracting the gray feature subset of the hole position area, the method further comprises:

According to a preset filter, smoothing filtering is performed on the hole location area to blur the texture information in the hole location area.
The method for detecting the status of the hole protection door according to claim 1, wherein after obtaining the defect status of the device corresponding to the image of the device to be detected, the method further comprises:

Extract the edge contour of the hole region according to the gen_contour_region_xld operator;

Mark the edge contour according to the defect state; and

Push the marked image of the edge contour.
A device for detecting the status of a hole position protection door, wherein the device includes:

The image acquisition module is used to read the image of the device to be tested and acquire the image of the device to be tested, the device to be tested is provided with a hole, and the hole is provided with a hole protection door;

A hole location area acquisition module, configured to perform adaptive dynamic threshold segmentation based on local statistics on the image of the device to be detected to determine the hole location area;

The gray-level feature extraction module is used to extract the gray-level feature subset of the hole location area based on the statistical algorithm of the gray-level histogram;

The projection module is used to project the gray-level feature subset onto a preset discriminant function line in a two-dimensional coordinate system, calculate the projected ordinate value of the gray-scale feature subset, and the preset discriminant function line So that the gray feature subset has a straight line with the largest inter-class distance and the smallest intra-class distance in the projected subspace;

The defect detection module is used to obtain the defect state of the device corresponding to the image of the device to be detected according to the magnitude relationship between the ordinate value after the projection of the gray-scale feature subset and the threshold value of the preset discriminant function line; and

The preset threshold calculation module is used to obtain multiple historical images of the equipment to be inspected, perform adaptive dynamic threshold segmentation based on local statistics on the historical images of the equipment to be inspected, determine the hole location area, and extract the historical image based on the gray histogram Construct a training sample set of gray-scale feature subsets of a plurality of hole location regions. The training sample set includes at least two types of sample subsets. The various sample subsets are projected into the optimal discriminant vector space, and the various types of The average value of the abscissa after the projection of the sample subset is used to obtain the preset threshold value of the discriminant function straight line. The optimal discriminant vector space is based on the Fisher linear discriminant analysis algorithm and the training sample set, and Lagrange multipliers are introduced The best projection direction obtained by the method.
The hole position protection door status detection device according to claim 7, wherein the device further comprises:

The marking push module is used to extract the edge contour of the hole location area according to the gen_contour_region_xld operator, mark the edge contour according to the defect state, and push the marked image of the edge contour.
A computer device includes a memory and one or more processors. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the one or more processors, the one or more Each processor performs the following steps:

Acquiring an image of the device to be tested, the device to be tested is provided with a hole position, and the hole position is provided with a hole position protection door;

Performing adaptive dynamic threshold segmentation based on local statistics on the image of the device to be detected to determine the hole location area;

Based on the statistical algorithm of the gray histogram, extract the gray feature subset of the hole location area;

Project the gray-scale feature subset onto a preset discriminant function line in a two-dimensional coordinate system to obtain the projected ordinate value of the gray-scale feature subset, and the preset discriminant function line is such that the gray The degree feature subset has a straight line with the largest inter-class distance and the smallest intra-class distance in the projected subspace; and

Obtaining the defect state of the device corresponding to the image of the device to be tested according to the magnitude relationship between the ordinate value after the projection of the gray-scale feature subset and the preset threshold of the discriminant function line;

The preset threshold of the discriminant function straight line is obtained in the following manner:

Obtain multiple historical images of the equipment to be tested;

Perform adaptive dynamic threshold segmentation based on local statistics on the historical image of the equipment to be tested to determine the hole location area;

Based on the statistical algorithm of the gray histogram, extract the gray feature subsets of multiple hole location regions to construct a training sample set, the training sample set includes at least two types of sample subsets; and

Project various sample subsets into the optimal discriminative vector space, and calculate the mean values of the abscissas after the projection of the various sample subsets to obtain the preset threshold value of the discriminant function straight line, and the optimal discriminant vector space is Based on the Fisher linear discriminant analysis algorithm and the training sample set, the optimal projection direction obtained by the Lagrangian multiplier method is introduced.
The computer device according to claim 9, wherein the processor further executes the following steps when executing the computer-readable instruction:

Perform threshold segmentation on the image of the device to be tested according to a preset adaptive dynamic threshold segmentation method based on local statistics to obtain a set of defective pixels;

According to the connection operator, perform connected domain analysis on the set of defective pixels to obtain a set of suspicious defective regions; and

According to preset hole location area constraints and preset hole location bulkiness, the hole location area is screened out from the set of suspicious defect areas.
The computer device according to claim 9, wherein the processor further executes the following steps when executing the computer-readable instruction:

Performing smoothing filtering processing on the image of the device to be detected based on the preset filter, and calculating the local standard deviation of the image of the device to be detected according to the average value of the pixel gray levels in the window of the preset filter;

Calculating the correction offset of each pixel according to the local standard deviation, the preset standard deviation scale factor and the preset minimum absolute threshold;

Obtaining the gray value of each pixel in the image of the device to be detected and the average value of the pixel gray of the template neighborhood of each pixel; and

The set of defective pixels is obtained according to the gray value of each pixel, the average value of the pixel gray of the template neighborhood of each pixel, and the corrected offset of each pixel.
The computer device according to claim 9, wherein the processor further executes the following steps when executing the computer-readable instruction:

Obtaining the average gray value of all pixels in the window of the preset filter according to the gray value of the central pixel in the window of the preset filter; and

The local standard deviation is calculated according to the gray value of the central pixel point and the gray average value.
The computer device according to claim 9, wherein the processor further executes the following steps when executing the computer-readable instruction:

According to a preset filter, smoothing filtering is performed on the hole location area to blur the texture information in the hole location area.
The computer device according to claim 9, wherein the processor further executes the following steps when executing the computer-readable instruction:

Extract the edge contour of the hole region according to the gen_contour_region_xld operator;

Mark the edge contour according to the defect state; and

Push the marked image of the edge contour.
One or more non-volatile computer-readable storage media storing computer-readable instructions, which when executed by one or more processors, cause the one or more processors to perform the following steps:

Acquiring an image of the device to be tested, the device to be tested is provided with a hole position, and the hole position is provided with a hole position protection door;

Performing adaptive dynamic threshold segmentation based on local statistics on the image of the device to be detected to determine the hole location area;

Based on the statistical algorithm of the gray histogram, extract the gray feature subset of the hole location area;

Project the gray-scale feature subset onto a preset discriminant function line in a two-dimensional coordinate system to obtain the projected ordinate value of the gray-scale feature subset, and the preset discriminant function line is such that the gray The degree feature subset has a straight line with the largest inter-class distance and the smallest intra-class distance in the projected subspace; and

Obtaining the defect state of the device corresponding to the image of the device to be tested according to the magnitude relationship between the ordinate value after the projection of the gray-scale feature subset and the preset threshold of the discriminant function line;

The preset threshold of the discriminant function straight line is obtained in the following manner:

Obtain multiple historical images of the equipment to be tested;

Perform adaptive dynamic threshold segmentation based on local statistics on the historical image of the equipment to be tested to determine the hole location area;

Based on the statistical algorithm of the gray histogram, extract the gray feature subsets of multiple hole location regions to construct a training sample set, the training sample set includes at least two types of sample subsets; and

Project various sample subsets into the optimal discriminative vector space, and calculate the mean values of the abscissas after the projection of the various sample subsets to obtain the preset threshold value of the discriminant function straight line, and the optimal discriminant vector space is Based on the Fisher linear discriminant analysis algorithm and the training sample set, the optimal projection direction obtained by the Lagrangian multiplier method is introduced.
The storage medium according to claim 15, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Perform threshold segmentation on the image of the device to be tested according to a preset adaptive dynamic threshold segmentation method based on local statistics to obtain a set of defective pixels;

According to the connection operator, perform connected domain analysis on the set of defective pixels to obtain a set of suspicious defective regions; and

According to preset hole location area constraints and preset hole location bulkiness, the hole location area is screened out from the set of suspicious defect areas.
The storage medium according to claim 15, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Performing smoothing filtering processing on the image of the device to be detected based on the preset filter, and calculating the local standard deviation of the image of the device to be detected according to the average value of the pixel gray levels in the window of the preset filter;

Calculating the correction offset of each pixel according to the local standard deviation, the preset standard deviation scale factor and the preset minimum absolute threshold;

Obtaining the gray value of each pixel in the image of the device to be detected and the average value of the pixel gray of the template neighborhood of each pixel; and

The set of defective pixels is obtained according to the gray value of each pixel, the average value of the pixel gray of the template neighborhood of each pixel, and the corrected offset of each pixel.
The storage medium according to claim 15, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Obtaining the average gray value of all pixels in the window of the preset filter according to the gray value of the central pixel in the window of the preset filter; and

The local standard deviation is calculated according to the gray value of the central pixel point and the gray average value.
The storage medium according to claim 15, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

According to a preset filter, smoothing filtering is performed on the hole location area to blur the texture information in the hole location area.
The storage medium according to claim 15, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Extract the edge contour of the hole region according to the gen_contour_region_xld operator;

Mark the edge contour according to the defect state; and

Push the marked image of the edge contour.