CN115375629A - Method for detecting line defect and extracting defect information in LCD screen - Google Patents

Method for detecting line defect and extracting defect information in LCD screen Download PDF

Info

Publication number
CN115375629A
CN115375629A CN202210933144.7A CN202210933144A CN115375629A CN 115375629 A CN115375629 A CN 115375629A CN 202210933144 A CN202210933144 A CN 202210933144A CN 115375629 A CN115375629 A CN 115375629A
Authority
CN
China
Prior art keywords
image
line
defect information
detection
defect
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210933144.7A
Other languages
Chinese (zh)
Inventor
吴宗泽
郑杰
任志刚
魏聪
龚文超
梁晓沣
刘臻铭
郭马萨迪
周坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong University of Technology
Original Assignee
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong University of Technology filed Critical Guangdong University of Technology
Priority to CN202210933144.7A priority Critical patent/CN115375629A/en
Publication of CN115375629A publication Critical patent/CN115375629A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/36Applying a local operator, i.e. means to operate on image points situated in the vicinity of a given point; Non-linear local filtering operations, e.g. median filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/48Extraction of image or video features by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/763Non-hierarchical techniques, e.g. based on statistics of modelling distributions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20132Image cropping
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30121CRT, LCD or plasma display

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Quality & Reliability (AREA)
  • Nonlinear Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method, a system and a computer readable storage medium for detecting line defects and extracting defect information in an LCD screen, wherein the method comprises the following steps: s1, acquiring an image to be detected, cutting an ROI (region of interest) image of the image to be detected, and extracting an actual region to be detected in the image; s2, performing Gaussian filtering on the obtained ROI image by using filtering checks with different sizes to obtain a foreground image and a background image of the ROI image; s3, carrying out image difference on the foreground image and the background image to obtain a foreground and background separation image; s4, canny edge detection is carried out on the front background separation image to obtain a Canny detection image which is subjected to edge tracing on the defects; s5, carrying out progressive probability Hough line transformation on the Canny detection image to obtain information of line defects; and S6, re-clustering the extracted defect information to obtain accurate defect information. The invention improves the accuracy and efficiency of detecting the line defects in the screen.

Description

Method for detecting line defect and extracting defect information in LCD screen
Technical Field
The invention relates to the technical field of visual inspection, in particular to a method and a system for detecting line defects and extracting defect information in an LCD screen and a computer readable storage medium.
Background
The line defect detection of LCD screens is an important component of defect detection, and the industry requires that the line defect detection not only be able to mark the defects from the original image, but also be able to extract the information of the line defects, which mainly includes the position, length and width. The extracted defect information has important reference function in the aspect of improving and adjusting the manufacturing process. Therefore, the industrial demand for defect detection has been developed from the initial simple detection of defects and defects to the extraction of defect information.
The existing line defect detection method only can detect whether the defect exists or not, but can not extract the defect information. The existing detection line defects are roughly divided into two modes: one method is to separate the foreground and background of the original image, then to binarize the original image, and to detect the defect contour of the binarized image, so that the defect can be detected, but the detection method only detects one contour for the intersecting lines. Therefore, in order to cope with such a situation, the etching operation is often performed after binarization, vertical lines are extracted after horizontal etching, horizontal lines are extracted after vertical etching, and the etching operation causes a very large interference to the extraction of defect information, and the etching operation has a good effect on the horizontal lines and the vertical lines, and has different oblique line etching effects with different angles. Therefore, this method is not suitable for extracting defect information; and the other method is to carry out Canny edge detection on the original image after the foreground and the background are separated, and then carry out Hough line transformation on the picture to obtain a set of line segments. However, since the image is subjected to gaussian filtering and difference in the early stage and the color of the defect edge in the image gradually changes, the defect edge detected by the Canny edge is not smooth, and thus, a plurality of collinear line segments may be extracted by hough line transformation, which is not favorable for extracting defect information.
The prior art discloses a K-means method-based corridor vanishing point rapid detection algorithm, after image data returned by a robot in real time is obtained, the image data is subjected to pre-processing, including down-sampling, graying processing, histogram equalization and Canny edge detection; extracting lines from the images obtained in the last step by utilizing a probabilistic Hough transform algorithm; and finally, clustering the detected straight lines into four types according to the slope by using a K-means algorithm, calculating the mean value of the middle points of each cluster of straight lines, constructing four straight lines by using the clustered slope and the mean value of the middle points to replace the straight lines detected in the previous step, randomly dividing the four straight lines into two groups, respectively calculating the intersection points of the four straight lines, and taking the middle points of the intersection points as vanishing points. Although the prior art scheme adopts a K-means method, the prior art scheme aims at point detection and does not relate to line detection.
Disclosure of Invention
The invention provides a method and a system for detecting a line defect and extracting defect information in an LCD screen and a computer readable storage medium, which improve the accuracy and efficiency of detecting the line defect in the screen.
The primary objective of the present invention is to solve the above technical problems, and the technical solution of the present invention is as follows:
the invention provides a line defect detection and defect information extraction method in an LCD screen, which comprises the following steps:
s1, acquiring an image to be detected, cutting an ROI (region of interest) image of the image to be detected, and extracting an actual region to be detected in the image;
s2, performing Gaussian filtering on the obtained ROI image by using filtering cores with different sizes to obtain a foreground image and a background image of the ROI image;
s3, carrying out image difference on the foreground image and the background image to obtain a foreground and background separation image;
s4, canny edge detection is carried out on the front background separation image to obtain a Canny detection image which is subjected to edge tracing on the defects;
s5, carrying out progressive probability Hough line transformation on the Canny detection image to obtain information of line defects;
and S6, re-clustering the extracted defect information to obtain accurate defect information.
Further, the ROI image cropping includes two types: 1. manually cutting, secondly, calculating the outline of the outermost edge of the screen, and then cutting out the ROI image according to the outline.
Further, the filtering kernels with different sizes in step S2 include: 5 × 5 small kernels and 105 × 105 large kernels, wherein the 5 × 5 small kernels are used to eliminate the influence of moire as much as possible, and the 105 × 105 large kernels are used to obtain a background image with balanced color.
Further, in step S4, canny edge detection is performed by setting a dual threshold, which is denoted as a high threshold H and a low threshold L, where the size of the high threshold is a set multiple of the low threshold, and the specific detection is as follows:
if the gradient value t of any pixel in the image is larger than H, reserving;
if the gradient value t of any pixel in the image is less than L, discarding;
if the gradient value L < = t < = H of any pixel in the image, searching the gradient value of the pixel from the field of the pixel, if the gradient value of the pixel is higher than a high threshold value, keeping the gradient value, and if the gradient value of the pixel is not higher than the high threshold value, abandoning the gradient value;
the defect edge is delineated from the remaining pixels.
Further, the specific process of performing the progressive probability hough line transformation on the Canny detection image comprises the following steps:
s501, randomly extracting a feature point, namely an edge point, from the Canny detection image, and if the edge point is calibrated to be a point on a certain straight line, continuously extracting an edge point from the rest edge points until all the edge points are extracted;
s502, carrying out Hough transformation on the extracted edge points, and carrying out accumulation and calculation;
s503, selecting a point with the maximum value in the Hough space, if the point is larger than a given threshold value, performing the step S504, otherwise, returning to the step S501;
s504, according to the maximum value obtained by Hough transform, starting from the maximum value point, displacing along the direction of a straight line, and thus finding two end points of the straight line;
s505, calculating the length of the straight line, if the length is larger than a given threshold value, outputting the straight line which is considered to be good, and returning to the step S501;
s506, outputting a set of line segments represented by two line segment end points.
Further, the step of re-clustering the extracted defect information to obtain accurate defect information comprises the following specific steps:
s601, solving the angle theta of each line segment in the line segment set (theta belongs to { theta-90 degrees < theta <90 degrees }), and then clustering the line segments according to angles to obtain clusters of the line segments classified according to the angles; wherein the line segments with different angles are obviously not on the same line
S602, projecting the coordinate system of the line segments in the clusters obtained in the step S601, projecting the line segment with the angle of 0 to the Y axis, projecting the line segment with the angle of 90 to the X axis, rotating the coordinate system to obtain the coordinates of the points of the rotated coordinate system and projecting the coordinates to the coordinate system when the angle is (0,90) or (-90,0);
s603, for the line segment with the angle of (90-90), firstly rotating the coordinate system, and then projecting to the rotated coordinate system;
s604, clustering the projected points;
and S605, processing the line segments which are clustered to obtain the defect information of the line defects.
Further, the coordinate rotation step is as follows:
in the original coordinate system XoY, if the angle θ of the line segment is (0,90), xoY is rotated counterclockwise around the origin by θ degrees to become a coordinate system SoT; if the angle theta of the line segment is (-90,0), the line segment rotates around the origin by theta degrees in the clockwise direction to become a coordinate system SoT;
the coordinate rotation formula is as follows:
if a certain point p is set, the coordinate in the original coordinate system is (x, y), and the rotated new coordinate is (s, t), then:
s=x·cos(θ)+y·sin(θ)
t=y·cos(θ)-x·sin(θ)
the coordinates of the end points of the line segments in the rotated coordinate system SoT are found, and then projected onto the SoT coordinate axis to obtain projected points (s, t).
The second aspect of the present invention provides a line defect detecting and defect information extracting system in an LCD screen, the system comprising: the line defect detection and defect information extraction method program in the LCD screen is executed by the processor to realize the following steps:
s1, acquiring an image to be detected, cutting an ROI (region of interest) image of the image to be detected, and extracting an actual region to be detected in the image;
s2, performing Gaussian filtering on the obtained ROI image by using filtering checks with different sizes to obtain a foreground image and a background image of the ROI image;
s3, carrying out image difference on the foreground image and the background image to obtain a foreground and background separation image;
s4, canny edge detection is carried out on the front background separation image to obtain a Canny detection image which is profiled on the defects;
s5, carrying out progressive probability Hough line transformation on the Canny detection image to obtain information of line defects;
and S6, re-clustering the extracted defect information to obtain accurate defect information.
Further, the ROI image cropping includes two types: 1. manually cutting, secondly, calculating the outline of the outermost edge of the screen, and then cutting out the ROI image according to the outline.
A third aspect of the present invention provides a computer-readable storage medium, where the computer-readable storage medium includes a program of a method for detecting line defects and extracting defect information on an LCD screen, and when the program of the method for detecting line defects and extracting defect information on an LCD screen is executed by a processor, the steps of the method for detecting line defects and extracting defect information on an LCD screen are implemented.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the method extracts the line segment defect information after Hough line transformation by using a clustering method, converts the line segment into projection points on coordinate axes by changing the measurement mode of the line segment relation in the clustering algorithm, namely the projection mode, and obtains accurate defect information by using the distance of the projection points as the measurement mode of measuring the line segment relation, thereby improving the accuracy and efficiency of detecting the line segment defect in the screen.
Drawings
FIG. 1 is a flow chart of a method for detecting line defects and extracting defect information in an LCD screen according to the present invention.
FIG. 2 is a diagram of a camera according to an embodiment of the present invention.
FIG. 3 is a ROI cropping map according to an embodiment of the present invention.
FIG. 4 is a perspective view of an embodiment of the present invention.
FIG. 5 is a background diagram of an embodiment of the present invention.
FIG. 6 is a front view and a back view of the mobile phone according to the embodiment of the present invention.
Fig. 7 is a Canny edge detection diagram according to an embodiment of the present invention.
Fig. 8 is a schematic diagram of a set of defective line segments obtained by progressive probabilistic hough line transformation according to an embodiment of the present invention.
FIG. 9 is a diagram of a clustering process according to an embodiment of the present invention.
FIG. 10 is a graph showing the results of the test according to the embodiment of the present invention.
FIG. 11 is a schematic diagram of points to be clustered in the conventional poly K-means clustering algorithm according to an embodiment of the present invention.
FIG. 12 is a schematic line projection diagram according to an embodiment of the present invention.
FIG. 13 is a schematic diagram of coordinate transformation according to an embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
Example 1
As shown in fig. 1, a first aspect of the present invention provides a method for detecting line defects and extracting defect information from an LCD screen, comprising the following steps:
s1, acquiring an image to be detected, cutting an ROI (region of interest) image of the image to be detected, and extracting an actual region to be detected in the image;
it should be noted that in practical industrial production scenarios, the field of view of the camera tends to be larger than the actual size of the screen. It is intended to photograph the entire area of the screen, but it also causes the image to have a background other than the screen. The image taken by the camera is shown in fig. 2. The detection algorithm is affected by the presence of background outside the screen. Therefore, the screen area in the image is often cut out, and only the ROI image of the screen is sent to a detection algorithm for detection. The ROI image clipping in the invention comprises two types: 1. manually cutting, secondly, calculating the outline of the outermost edge of the screen, and then cutting out the ROI image according to the outline.
S2, performing Gaussian filtering on the obtained ROI image by using filtering cores with different sizes to obtain a foreground image and a background image of the ROI image;
it should be noted that, due to the influence of the grating and the illumination environment during imaging by the camera, the finally imaged picture has moire, and therefore, the influence of moire is eliminated as much as possible by using the small kernel gaussian filter. The Gaussian filtering with large kernel is used for obtaining a background image with balanced colors, and the filtering kernels with different sizes in the invention comprise: a small kernel of 5 × 5, a large kernel of 105 × 105.
S3, carrying out image difference on the foreground image and the background image to obtain a foreground and background separation image;
it should be noted that, since the defect in the ROI image has a significant color difference from the surrounding area, the foreground/background difference is adopted. When an image is processed, the image can be represented as a matrix of pixel values, and the principle of foreground and background separation is to make a difference by a matrix of pixels of the same size, that is, by making a difference by pixel values at the same position. Because the background image is obtained by performing a large kernel gaussian filtering on the ROI image, the pixel value matrix of the background image can be regarded as a pixel matrix with the same pixel value size. Therefore, the pixel value of the defective position after the subtraction will not be 0, and the pixel value of the non-defective position will be 0.
S4, canny edge detection is carried out on the front background separation image to obtain a Canny detection image which is profiled on the defects;
it should be noted that the purpose of Canny edge detection is to delineate defects, which is convenient for extracting defects in the next step of progressive probabilistic hough line transformation. Canny edge detection is carried out by setting double thresholds which are recorded as a high threshold H and a low threshold L, wherein the size of the high threshold is 2-3 times that of the low threshold, and the detection is as follows: if the gradient value t of any pixel in the image is larger than H, reserving;
if the gradient value t of any pixel in the image is less than L, discarding;
if the gradient value L < = t < = H of any pixel in the image, searching the gradient value of the pixel from the field of the pixel, if the gradient value of the pixel is higher than a high threshold value, keeping the gradient value, and if the gradient value of the pixel is not higher than the high threshold value, abandoning the gradient value;
the defect edge is delineated from the remaining pixels.
S5, carrying out progressive probability Hough line transformation on the Canny detection image to obtain information of line defects;
it should be noted that the specific process of performing the progressive probability hough line transformation on the Canny detection diagram is as follows:
s501, randomly extracting a feature point, namely an edge point, from the Canny detection image, and if the edge point is calibrated to be a point on a certain straight line, continuously extracting an edge point from the rest edge points until all the edge points are extracted;
s502, carrying out Hough transformation on the extracted edge points, and carrying out accumulation and calculation;
s503, selecting a point with the maximum value in the Hough space, if the point is larger than a given threshold value, performing the step S504, otherwise, returning to the step S501;
s504, according to the maximum value obtained by Hough transform, starting from the maximum value point, displacing along the direction of a straight line, and thus finding two end points of the straight line;
s505, calculating the length of the straight line, if the length is larger than a given threshold value, outputting the straight line which is considered to be good, and returning to the step S501;
s506, outputting a set of line segments represented by two line segment end points.
And S6, re-clustering the extracted defect information to obtain accurate defect information.
It should be noted that K in the K-means clustering algorithm is a very important parameter, which means that the algorithm can be divided into several clusters.
K-means is a process of repeated iteration, and the algorithm is roughly divided into four steps:
(1) k objects in the data space are randomly selected as initial centers, and each object represents a clustering center.
(2) For the data objects in the sample, according to their Euclidean distances from the cluster centers, they are classified into the class corresponding to the cluster center (most similar) closest to them according to the criterion of closest distance.
(3) Updating a clustering center: and calculating the mean value corresponding to all the objects in each category, and taking the mean value as the next clustering center of the category.
(4) And if the values of the current clustering center and the next clustering center are not changed, the algorithm is converged, and if the values of the current clustering center and the next clustering center are changed, the clustering centers calculated in the third step are clustered again. Fig. 11 is a schematic diagram of points to be clustered by the conventional poly K-means clustering algorithm. The coordinates of the positions of the points are shown in table 1.
TABLE 1
Figure BDA0003782505210000071
Figure BDA0003782505210000081
From the above table it can be seen that these 6 points can be divided into two clusters, thus:
1. choosing K =2 initial centers, i.e. centroids, a and B are chosen
2. Calculating the distance between other points and the initial center
The distance from C to a can also be seen from the figure (pythagorean theorem), which is √ 10=3.16;
the distance of C to B √ ((3-1) 2+ (1-2) 2= √ 5=2.24, so C is closer to B and follows B in a cluster.
Similarly, D, E, F also counts as the position of the point after the first iteration, as in table 2 below:
TABLE 2
A B
C 3.16 2.24
D 11.3 9.22
E 13.5 11.3
F 12.2 10.3
Results after first grouping
Cluster 1: a. The
Cluster 2: B. c, D, E, F
3. Recalculating new centers
The center of cluster 1 is also A = (0,0)
The new center coordinates for cluster 2 are: p = ((1 +3+8+9+ 10)/5, (2 +1+8+10+ 7)/5) = (6.2,5.6)
The positions of the points after the second iteration are given in table 3 below:
TABLE 3
A P
B 2.23 6.32
C 3.16 4.6
D 11.3 3
E 13.5 5.22
F 12.2 4.05
Second grouping of results
Cluster 1: A. b, C
Cluster 2: D. e, F
4. Recalculating the centroid
P1=(1.33,1)
P2=(9,8.33)
The positions of the points after the third iteration are given in table 4 below:
TABLE 4
P1 P2
A 1.64 12.26
B 1.05 10.20
C 1.67 9.47
D 9.64 1.05
E 11.80 1.63
F 10.42 1.66
Third grouping of results
Cluster 1: p1, P2, P3
Cluster 2: p4, P5, P6
And the third grouping result is consistent with the second grouping result, which shows that convergence is achieved and clustering is finished. The selection of the K value is important from the iteration process, and a reference quantity, namely the variance, needs to be introduced in order to ensure that the clustering algorithm can be divided into a proper number of clusters when the K value is uncertain. Variance is a measure of the degree of dispersion when probability theory and statistical variance measure a random variable or a set of data, and thus variance can be used to measure the degree of sparsity of a cluster of points, where a smaller variance means that the cluster is more compact, and a larger variance means that the cluster is more loose. While we want clustered data, its variance is relatively small. Therefore, the variance of each cluster is calculated during clustering, and the average variance of the clusters is obtained after clustering convergence. When the average variance is smaller than the set variance, we consider this K to be a suitable value of K. If the average variance is greater than the set variance, K +1, and clustering again until the average variance is less than the set variance.
B. Distance measurement
When line segments are clustered, a problem is also worth considering how to describe the relationship between line segments, and obviously, the euclidean distance is not enough to describe the relationship between line segments, so that a new way needs to be proposed to describe the relationship between line segments. By way of the foregoing example of clustering a cluster of points, the present invention converts line segments into points and then clusters.
The invention re-clusters the extracted defect information and obtains accurate defect information, which comprises the following specific steps:
s601, solving the angle theta of each line segment in the line segment set (theta belongs to { theta-90 degrees < theta <90 degrees }), and then clustering the line segments according to angles to obtain clusters of the line segments classified according to the angles; wherein the line segments with different angles are obviously not on the same line
S602, projecting the coordinate system of the line segments in the clusters obtained in the step S601, projecting the line segment with the angle of 0 to the Y axis, projecting the line segment with the angle of 90 to the X axis, rotating the coordinate system to obtain the coordinates of the points of the rotated coordinate system and projecting the coordinates to the coordinate system when the angle is (0,90) or (-90,0); the projection of a line segment with an angle of 0 is taken as an example, as shown in fig. 12.
S603, for the line segment with the angle of (90-90), firstly rotating the coordinate system, and then projecting to the rotated coordinate system; the line segment representation after the coordinate system rotation is obtained by a coordinate rotation formula.
The coordinate rotation steps are as follows:
in the original coordinate system XoY, if the angle θ of the line segment is (0,90), xoY is rotated counterclockwise around the origin by θ degrees to become a coordinate system SoT; if the angle theta of the line segment is (-90,0), the line segment rotates around the origin by theta degrees in the clockwise direction to become a coordinate system SoT;
the coordinate rotation formula is as follows:
as shown in fig. 13, if a certain point p is set, the coordinates in the original coordinate system are (x, y), and the new coordinates after rotation are (s, t), then:
oa = y sin (θ) (formula 2.1)
as = x · cos (θ) (formula 2.2)
Combining the two formulas to obtain:
s = os = oa + as = x · cos (θ) + y · sin (θ) (formula 2.3)
t = ot = ay-ab = y · cos (θ) -x · sin (θ) (formula 2.4)
The coordinates of the end points of the line segment in the rotated coordinate system SoT are obtained, and then projected onto the SoT coordinate axis to obtain a projected point (s, t).
S604, clustering the projected points;
and S605, processing the clustered line segments to obtain the defect information of the line defects.
Example 2
The second aspect of the present invention provides a line defect detecting and defect information extracting system in an LCD screen, the system comprising: the memory comprises a line defect detection and defect information extraction method program of the LCD screen, and the line defect detection and defect information extraction method program of the LCD screen realizes the following steps when being executed by the processor:
s1, acquiring an image to be detected, cutting an ROI (region of interest) image of the image to be detected, and extracting an actual region to be detected in the image;
s2, performing Gaussian filtering on the obtained ROI image by using filtering checks with different sizes to obtain a foreground image and a background image of the ROI image;
s3, carrying out image difference on the foreground image and the background image to obtain a foreground and background separation image;
s4, canny edge detection is carried out on the front background separation image to obtain a Canny detection image which is profiled on the defects;
s5, carrying out progressive probability Hough line transformation on the Canny detection image to obtain information of line defects;
and S6, re-clustering the extracted defect information to obtain accurate defect information.
Further, the ROI image cropping includes two types: 1. manually cutting, secondly, calculating the outline of the outermost edge of the screen, and then cutting out the ROI image according to the outline.
A third aspect of the present invention provides a computer-readable storage medium, where the computer-readable storage medium includes a program of a method for detecting line defects and extracting defect information on an LCD screen, and when the program of the method for detecting line defects and extracting defect information on an LCD screen is executed by a processor, the steps of the method for detecting line defects and extracting defect information on an LCD screen are implemented.
Example 3
This example is described in detail with reference to a specific detection procedure.
The product to be detected is placed below the camera, the distance between the product and the camera is adjusted to enable the product to be located in the field of view of the camera, then the screen is lightened by electrifying, then the camera is used for image acquisition, the acquired image is transmitted to the computer end, and the acquired image is as shown in fig. 2. The test is started.
The black area in fig. 2 is the background outside the screen, which may affect the actual detection and therefore needs to be removed. Can cut out the screen area through ROI tailors, make things convenient for subsequent detection procedure. The clipped effect graph is shown in fig. 3.
Since the moire generated in the image acquisition process can interfere with the subsequent detection, the image is subjected to Gaussian filtering with the filtering kernel of 5 multiplied by 5, and the filtered result image is used as a foreground image in the subsequent image difference operation. The foreground map is shown in fig. 4:
for the subsequent image differencing operation, a background image of the image is also acquired. The foreground plot is also gaussian filtered once, but this time with a filter kernel of 105 x 105. The background picture is shown in fig. 5:
and performing Canny edge detection on the foreground and background separation images to outline the line defects. The foreground background separation map is shown in fig. 6 and the Canny edge detection map is shown in fig. 7.
A line segment set can be obtained by performing progressive probabilistic Hough line transformation on the Canny edge detection graph. Taking fig. 6 as an example, ideally, the result graph of Canny edge detection should be two smooth straight lines, but due to some image processing and color gradient of the defect edge in the detection process, the actual Canny edge detection of fig. 6 is not two smooth straight lines, but two straight lines with jaggies. And then, a plurality of line segments are extracted by detecting the two unsmooth straight lines through the progressive probability Hough line transformation. The line segment set is shown in FIG. 8:
marking a line segment set obtained by transforming the progressive probability hough line on the ROI image, wherein a line defect detection result graph is shown in FIG. 9:
in order to extract a few line defects and defect information of each line defect from such a disordered set, the set is clustered, and the convergence variance of the clustering is set to 20.
And reading the points of line segment projection at the same angle before clustering, wherein the projection points are in one-to-one correspondence with the line segments, and taking the points as a clustered data set. Clustering in the present invention is performed three times, and the time with the best effect (i.e., the smallest average variance) among the three times is taken. The initial K of the clustering is 1, when the clustering converges but does not meet the set variance, K is added with one to restart the clustering.
Fig. 10 is a diagram showing the defect detection results. And extracting defect information of each line set after clustering is finished, wherein the extracted defect information is shown in a table 5.
TABLE 5
Figure BDA0003782505210000131
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims (10)

1. A method for detecting line defects and extracting defect information in an LCD screen is characterized by comprising the following steps:
s1, acquiring an image to be detected, cutting an ROI (region of interest) image of the image to be detected, and extracting an actual region to be detected in the image;
s2, performing Gaussian filtering on the obtained ROI image by using filtering cores with different sizes to obtain a foreground image and a background image of the ROI image;
s3, carrying out image difference on the foreground image and the background image to obtain a foreground and background separation image;
s4, canny edge detection is carried out on the front background separation image to obtain a Canny detection image which is profiled on the defects;
s5, carrying out progressive probability Hough line transformation on the Canny detection image to obtain information of line defects;
and S6, re-clustering the extracted defect information to obtain accurate defect information.
2. The method of claim 1, wherein the ROI image cropping comprises two methods: 1. manually cutting, secondly, calculating the outline of the outermost edge of the screen, and then cutting out the ROI image according to the outline.
3. The method as claimed in claim 1, wherein the filtering kernels with different sizes in step S2 comprise: 5 × 5 small kernels and 105 × 105 large kernels, wherein the 5 × 5 small kernels are used to eliminate the influence of moire as much as possible, and the 105 × 105 large kernels are used to obtain a background image with balanced color.
4. The method according to claim 1, wherein Canny edge detection is performed by setting a double threshold in step S4, the double threshold is denoted as a high threshold H and a low threshold L, wherein the high threshold is a set multiple of the low threshold, and the detection is as follows: if the gradient value t of any pixel in the image is larger than H, retaining;
if the gradient value t of any pixel in the image is less than L, discarding;
if the gradient value L < = t < = H of any pixel in the image, searching the gradient value of the pixel from the field of the pixel, if the gradient value of the pixel is higher than a high threshold value, keeping the gradient value, and if the gradient value of the pixel is not higher than the high threshold value, abandoning the gradient value;
the defect edge is delineated from the remaining pixels.
5. The method for detecting the line defect and extracting the defect information in the LCD screen according to claim 1, wherein the specific process of carrying out the gradual probability Hough line transformation on the Canny detection image comprises the following steps:
s501, randomly extracting a feature point, namely an edge point, from the Canny detection graph, and if the edge point is calibrated to be a point on a certain straight line, continuously extracting one edge point from the rest edge points until all the edge points are extracted;
s502, carrying out Hough transformation on the extracted edge points, and carrying out accumulation and calculation;
s503, selecting a point with the maximum value in the Hough space, if the point is larger than a given threshold value, performing the step S504, otherwise, returning to the step S501;
s504, according to the maximum value obtained by Hough transform, starting from the maximum value point, displacing along the direction of a straight line, and thus finding two end points of the straight line;
s505, calculating the length of the straight line, if the length is larger than a given threshold value, outputting the straight line which is considered to be good, and returning to the step S501;
s506, outputting a set of line segments represented by two line segment end points.
6. The method for detecting the line defect and extracting the defect information in the LCD screen according to claim 1, wherein the step of re-clustering the extracted defect information to obtain the accurate defect information comprises the following steps:
s601, solving the angle theta of each line segment in the line segment set (theta belongs to { theta-90 degrees < theta <90 degrees }), and then clustering the line segments according to angles to obtain clusters of the line segments classified according to the angles; wherein the line segments with different angles are obviously not on the same line
S602, projecting a coordinate system of the line segments in the clusters obtained in the step S601, projecting the line segment with the angle of 0 to the Y axis, projecting the line segment with the angle of 90 to the X axis, rotating the coordinate system to obtain the coordinates of the points after the rotation of the coordinate system and projecting the coordinates to the coordinate system when the line segment with the angle of (0,90) or (-90,0) is in the angle;
s603, for the line segment with the angle of (90-90), firstly rotating the coordinate system, and then projecting to the rotated coordinate system;
s604, clustering the projected points;
and S605, processing the clustered line segments to obtain the defect information of the line defects.
7. The method of claim 6, wherein the step of rotating coordinates comprises the steps of:
in the original coordinate system XoY, if the angle θ of the line segment is (0,90), xoY is rotated counterclockwise around the origin by θ degrees to become a coordinate system SoT; if the angle theta of the line segment is (-90,0), the line segment rotates around the origin in the clockwise direction by theta degrees to become a coordinate system SoT;
the coordinate rotation formula is as follows:
if a certain point p is set, the coordinate in the original coordinate system is (x, y), and the new coordinate after rotation is (s, t), then:
s=x·cos(θ)+y·sin(θ)
t=y·cos(θ)-x·sin(θ)
the coordinates of the end points of the line segment in the rotated coordinate system SoT are obtained, and then projected onto the SoT coordinate axis to obtain a projected point (s, t).
8. A system for detecting line defects and extracting defect information in an LCD screen, the system comprising: the line defect detection and defect information extraction method program in the LCD screen is executed by the processor to realize the following steps:
s1, acquiring an image to be detected, cutting an ROI (region of interest) image of the image to be detected, and extracting an actual region to be detected in the image;
s2, performing Gaussian filtering on the obtained ROI image by using filtering cores with different sizes to obtain a foreground image and a background image of the ROI image;
s3, carrying out image difference on the foreground image and the background image to obtain a foreground and background separation image;
s4, canny edge detection is carried out on the front background separation image to obtain a Canny detection image which is subjected to edge tracing on the defects;
s5, carrying out progressive probability Hough line transformation on the Canny detection image to obtain information of line defects;
and S6, re-clustering the extracted defect information to obtain accurate defect information.
9. The method as claimed in claim 8, wherein the ROI image cropping comprises two types: 1. manually cutting, secondly, calculating the outline of the outermost edge of the screen, and then cutting out the ROI image according to the outline.
10. A computer-readable storage medium, wherein the computer-readable storage medium comprises a program of a line defect detection and defect information extraction method in an LCD screen, and when the program of the line defect detection and defect information extraction method in the LCD screen is executed by a processor, the steps of the method for line defect detection and defect information extraction in the LCD screen according to any one of claims 1 to 7 are implemented.
CN202210933144.7A 2022-08-04 2022-08-04 Method for detecting line defect and extracting defect information in LCD screen Pending CN115375629A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210933144.7A CN115375629A (en) 2022-08-04 2022-08-04 Method for detecting line defect and extracting defect information in LCD screen

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210933144.7A CN115375629A (en) 2022-08-04 2022-08-04 Method for detecting line defect and extracting defect information in LCD screen

Publications (1)

Publication Number Publication Date
CN115375629A true CN115375629A (en) 2022-11-22

Family

ID=84064603

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210933144.7A Pending CN115375629A (en) 2022-08-04 2022-08-04 Method for detecting line defect and extracting defect information in LCD screen

Country Status (1)

Country Link
CN (1) CN115375629A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115641337A (en) * 2022-12-23 2023-01-24 中科慧远视觉技术(北京)有限公司 Linear defect detection method, device, medium, equipment and system
CN116165216A (en) * 2023-03-16 2023-05-26 苏州鼎纳自动化技术有限公司 Liquid crystal display micro scratch flaw 3D detection method, system and computing equipment
CN117152103A (en) * 2023-09-08 2023-12-01 深圳市明昌光电科技有限公司 Display screen point defect, line defect and Mura defect judging method, system and device

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115641337A (en) * 2022-12-23 2023-01-24 中科慧远视觉技术(北京)有限公司 Linear defect detection method, device, medium, equipment and system
CN115641337B (en) * 2022-12-23 2023-04-07 中科慧远视觉技术(北京)有限公司 Linear defect detection method, device, medium, equipment and system
CN116165216A (en) * 2023-03-16 2023-05-26 苏州鼎纳自动化技术有限公司 Liquid crystal display micro scratch flaw 3D detection method, system and computing equipment
CN116165216B (en) * 2023-03-16 2023-08-04 苏州鼎纳自动化技术有限公司 Liquid crystal display micro scratch flaw 3D detection method, system and computing equipment
CN117152103A (en) * 2023-09-08 2023-12-01 深圳市明昌光电科技有限公司 Display screen point defect, line defect and Mura defect judging method, system and device
CN117152103B (en) * 2023-09-08 2024-06-07 深圳市明昌光电科技有限公司 Display screen point defect, line defect and Mura defect judging method, system and device

Similar Documents

Publication Publication Date Title
WO2022042579A1 (en) Lcd screen defect detection method and apparatus
CN106846316B (en) Automatic identification method for typical defect images in GIS
CN109978839B (en) Method for detecting wafer low-texture defects
CN115018828B (en) Defect detection method for electronic component
JP6710135B2 (en) Cell image automatic analysis method and system
CN115375629A (en) Method for detecting line defect and extracting defect information in LCD screen
CN113109368B (en) Glass crack detection method, device, equipment and medium
CN110378313B (en) Cell cluster identification method and device and electronic equipment
CN109859226B (en) Detection method of checkerboard corner sub-pixels for graph segmentation
CN106709500B (en) Image feature matching method
WO2017181892A1 (en) Foreground segmentation method and device
CN110648330B (en) Defect detection method for camera glass
CN109447117B (en) Double-layer license plate recognition method and device, computer equipment and storage medium
CN114972575A (en) Linear fitting algorithm based on contour edge
CN111695373A (en) Zebra crossing positioning method, system, medium and device
CN111354047A (en) Camera module positioning method and system based on computer vision
CN115953399B (en) Industrial part structural defect detection method based on contour features and SVDD
CN113129260A (en) Automatic detection method and device for internal defects of lithium battery cell
CN116958058A (en) Lens dirt detection method and device and image detection equipment
CN115880228A (en) Multi-defect merging method and device, computer equipment and storage medium
CN116596899A (en) Method, device, terminal and medium for identifying circulating tumor cells based on fluorescence image
CN111695374A (en) Method, system, medium, and apparatus for segmenting zebra crossing region in monitoring view
CN113963070A (en) Circular calibration plate identification method
CN113840135A (en) Color cast detection method, device, equipment and storage medium
CN109949245B (en) Cross laser detection positioning method and device, storage medium and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination