WO2021168733A1 - 缺陷图像的缺陷检测方法、装置及计算机可读存储介质 - Google Patents

缺陷图像的缺陷检测方法、装置及计算机可读存储介质 Download PDF

Info

Publication number
WO2021168733A1
WO2021168733A1 PCT/CN2020/076958 CN2020076958W WO2021168733A1 WO 2021168733 A1 WO2021168733 A1 WO 2021168733A1 CN 2020076958 W CN2020076958 W CN 2020076958W WO 2021168733 A1 WO2021168733 A1 WO 2021168733A1
Authority
WO
WIPO (PCT)
Prior art keywords
defect
image
category
response
detected
Prior art date
Application number
PCT/CN2020/076958
Other languages
English (en)
French (fr)
Inventor
郭一川
路元元
李昭月
柴栋
王洪
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to PCT/CN2020/076958 priority Critical patent/WO2021168733A1/zh
Priority to CN202080000190.0A priority patent/CN113646801B/zh
Publication of WO2021168733A1 publication Critical patent/WO2021168733A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination

Definitions

  • the present disclosure relates to a defect detection method, device and computer readable storage medium for defect images.
  • the output products will not meet the process requirements or even lead to defects, so it needs to be in each process Then, calculate and identify the bad type, size, location and other information of the bad and defective products that do not meet the requirements in time, and make timely corrections and improvements to avoid the continued occurrence of defects.
  • the Digital Macro (DM) map at the substrate level is used to detect defects at the glass substrate level. It is used to monitor the integrity of the glass substrate during the processing of the panel and is an important supplementary means for quality control. Operators spend much less time and energy on the defect classification of the DM map than the AOI color map, but once the DM map has a problem, it is a very serious accident level failure. Misjudgment and omission of the DM map can lead to minor errors. The subsequent processing time is wasted, and the subsequent processing equipment will be damaged.
  • the defects of the DM map include breakage, debris, area discharge, line discharge, special unevenness, white spots, black bars, and gray bands.
  • the traditional methods for identifying defects in images mainly rely on manual detection. This requires professional training for inspectors. Especially in the case of multiple product models and complex problems, it is necessary for the inspectors to invest a long time and focus on defect finding and related judgments.
  • a defect detection method for a defect image which includes: acquiring a substrate image as a defect image to be detected; and using each defect detection algorithm in a set of defect detection algorithms to perform defects on the defect image to be detected. Detect and generate corresponding responses to obtain a defect detection response set, wherein the defect detection algorithm set includes at least two defect detection algorithms; and based on the defect detection response set and the priority of multiple candidate defect categories, The defect category determines the defect category of the defect image to be detected in the defect category.
  • a defect detection device for a defect image including: a processor; and a memory on which computer-usable instructions are stored.
  • the instructions executes the steps in the method as described above.
  • a computer-readable storage medium having program instructions stored thereon, and the stored program instructions can be read and executed by a processor, so that the processor executes the above-mentioned Steps in the method.
  • Figure 1(a)-(i) shows the defect schematic diagrams corresponding to several example candidate defect categories in the DM diagram
  • Fig. 2 shows a flowchart of a defect detection method for a defect image according to an embodiment of the present disclosure
  • FIG. 3 shows a schematic flowchart of judging various candidate defect categories through various defect detection algorithms in a defect detection algorithm set according to an embodiment of the present disclosure
  • Fig. 4 shows a block diagram of a defect detection device for a defect image according to an embodiment of the present disclosure.
  • These computer program instructions can be provided to the processor of a general-purpose computing device, a special-purpose computing device, and/or other programmable data processing devices, so that the instructions executed via the computing device processor and/or other programmable data processing devices are created for implementation.
  • the present disclosure can also be implemented by hardware and/or software (including firmware, resident software, microcode, etc.).
  • the present disclosure may take the form of a computer program product on a computer-usable or computer-readable storage medium, which has a computer-usable or computer-readable program code implemented in the medium to be used or used by an instruction execution system. Used in conjunction with the instruction execution system.
  • a computer-usable or computer-readable medium can be any medium that can contain, store, communicate, transmit, or transfer a program for use by an instruction execution system, apparatus, or device, or in combination with an instruction execution system, Device or equipment use.
  • artificial intelligence image classification technology can continuously self-learn and use more classified samples to train and strengthen the overall model recognition ability.
  • the accuracy of the system is getting better and better while ensuring stability.
  • the classification technology of artificial intelligence image recognition has a high demand for data, and also has a relatively high demand for computer computing power and resources. For example, in a specific design, because bad image data may be difficult to collect, and the number of samples used for training is limited, problems such as model overfitting are prone to occur, which limits its use to a certain extent.
  • the present disclosure proposes a defect detection method, device, and computer-readable storage medium for a substrate map (such as a DM map).
  • the defect detection method proposed in the present disclosure takes into account that it is difficult to collect bad pictures of DM images, uses traditional image processing algorithms, and manually pre-designs the main features for distinguishing defects of different manifestations, and focuses on the main features for distinguishing various defects.
  • traditional image processing algorithms Compared with neural network algorithms in artificial intelligence, traditional image processing algorithms have lower requirements on the amount of data and can solve the problem of data collection collection that is difficult to cross.
  • the DM map is an image on the glass substrate level, and there are fewer defects on the glass substrate level, there is generally only one defect on a DM map, and the following description is also based on the defect on a DM map. The test is carried out.
  • Figures 1(a)-(i) show schematic diagrams of defects corresponding to several example candidate defect categories in the DM map.
  • Fig. 1(a) shows a broken defect, which means that the glass substrate is broken.
  • Figure 1(b) shows a piece defect, which refers to the presence of glass fragments on the glass substrate.
  • Figure 1(c) shows an area arcing defect, which refers to a wide range of discharge defects on the glass substrate, usually spread over one or several panels on the substrate.
  • Figure 1(d) shows the line arcing defect, which refers to one or several abnormal lines that can be seen on the glass substrate due to the discharge of one or several gate/data lines, usually In the horizontal direction.
  • Figure 1(e) shows a Special Gray Aldistribution defect (hereinafter also referred to as special unevenness in some places). This defect refers to the existence of a gray area with irregular borders on the glass substrate. A special pattern will be formed.
  • Figure 1(f) shows a white spot defect (White Pot) defect, which refers to the presence of white residue on the glass substrate.
  • Figure 1(g)-(h) shows the black strip (Black Slip) defect, which refers to the presence of black strips on the glass substrate (the first type of black strips in Figure 1(g)) or striped lines (the second Type black bars in Figure 1(f)).
  • Black Slip refers to the presence of black strips on the glass substrate (the first type of black strips in Figure 1(g)) or striped lines (the second Type black bars in Figure 1(f)).
  • Figure 1(i) shows a gray gap defect, which means that there are obvious light and dark stripes on the glass substrate.
  • For each form of defect it has its own main feature, and at least two candidate defect categories have the same main feature.
  • the at least two candidate defect categories have similar characteristics in terms of manifestation, so the similar characteristics can be considered as their common (or the same) main characteristics.
  • special uneven defects, chip defects and white spot defects are the main feature of the three candidate defect categories.
  • the main feature of the defect is that the gray level of the defect area is different from the surrounding area; the main feature of the area discharge defect is that it is in one or several adjacent panels.
  • the main feature of the two candidate defect categories of the first type of black bar defects and broken defects is that the pixel gray value in the defect area is zero.
  • defect detection algorithms suitable for detecting the main features of the candidate defects. That is, for each main feature, there is a corresponding defect detection algorithm.
  • the black and white spot detection algorithm is used to detect the candidate defect category defects whose main feature is that the gray in the defect area is different from the surrounding area, including special uneven defects, chip defects, and white spot defects;
  • the edge density detection algorithm is used to detect main features Candidate defect category defects with a wide range of crack-like patterns in one or several adjacent panels, including area discharge defects;
  • Hough transform line detection algorithm is used to detect candidate defect category defects whose main feature is line segment, including second Types of black bar defects and line discharge defects;
  • the vertical projection detection algorithm is used to detect the candidate defect category defects whose main feature is the blocky area with lines or abnormal colors in the vertical direction, including gray band defects;
  • low threshold two-value The chemical detection algorithm is used to detect the candidate defect category defects whose main feature is the pixel gray value of zero in the defect area, including the first type of black bar defects and broken defects.
  • the main feature is that the gray level of the defect area is different from that of the surrounding area
  • the two candidate defect categories, the sub-feature of the main feature is the direction and length of the line segment; and the main feature is
  • the defect with the pixel gray value of zero in the defect area includes the first type of black stripe defect and the broken defect
  • the two candidate defect categories, the sub-feature of the main feature is that there are black pixels in the column of the image binary image The number of black pixels, or the shape and length of the defect area boundary in the image binary image.
  • the present disclosure When detecting the defects in the DM map (the image of the defect to be detected), the present disclosure simultaneously detects the defects in the DM map through various defect detection algorithms, and determines which type of defect the defect in the DM map belongs to according to the detection results category.
  • the DM image is a composite grayscale image, which is finally stitched together by the camera shooting column by column.
  • the uneven brightness of the photos taken by the camera in the horizontal direction results in evenly spaced stitching marks in the vertical direction. Since the splicing trace is not very obvious and has little effect on the algorithm, it can be ignored.
  • the grayscale value of the panel edge is very different from the pixels near the edge, which will interfere with many defect detection algorithms. Therefore, it is necessary to design the panel edge removal algorithm, which will be introduced in detail later.
  • Fig. 2 shows a flowchart of a defect detection method of a defect image according to an embodiment of the present disclosure.
  • step 210 a substrate image is acquired as a defect image to be detected.
  • the substrate image may be a DM grayscale image
  • using the substrate image as the defect image to be detected includes: reading the DM image and resetting the size of the DM image.
  • the DM map itself is a grayscale map.
  • the common grayscale value range of integer is [0, 255] (Unsigned 8 -bit), but [0, 65535] (unsigned 16-bit), [-32768, 32767] (signed 16-bit), etc. are also feasible;
  • the common grayscale value of floating point type is [0.0,1.0] (32-bit and 64-bit commonly used values are [0.0, 1.0], but other values such as [-1.0, 2.0] can also be used).
  • the pixel value of the original image of the DM image is not fixed, but the ratio is close to 1:1.15, a pair of length and width values can be used to resize the original image, such as (400, 460), (600, 690), (800, 920) ), etc. (In order to prevent some small defects from disappearing during resampling, generally smaller pixel values are not used).
  • the gray scale value range selected in the present disclosure is [0, 255] (unsigned 8-bit), and the reset size of the DM map is (600, 690).
  • the panel edge removal algorithm can optionally be designed to improve the detection accuracy.
  • removing the edge of the panel on the substrate mainly includes the following steps: 1) Perform an accumulation operation on the grayscale values of each row and each column of the DM map, respectively, to obtain a one-dimensional vector, and each element pair of the one-dimensional vector It should be the sum of the grayscale values of the row or column; (2) A low-pass filter is used for the obtained one-dimensional vector (the median filter is taken as an example in this disclosure.
  • low-pass filters include average filters, Butterworth low-pass filter, Gaussian low-pass filter, etc.); (3) Use the original one-dimensional vector to subtract the low-frequency component (the vector obtained by low-pass filtering the original vector) to obtain the high-frequency component of the original vector; (4) ) Use a certain threshold to segment the high-frequency components, and the element position that exceeds the threshold is considered as the panel edge; (5) Remove the rows and columns of several pixels in the neighborhood of the panel edge position in the DM map.
  • each defect detection algorithm in the defect detection algorithm set is used to perform defect detection on the defect image to be detected and generate a corresponding response to obtain a defect detection response set, wherein the defect detection algorithm set includes at least two defect detection algorithms.
  • the defect detection algorithm set includes a black and white spot detection algorithm, an edge density detection algorithm, a Hough transform straight line detection algorithm, a projection detection algorithm, and a low-threshold binary detection algorithm.
  • a black and white spot detection algorithm an edge density detection algorithm
  • a Hough transform straight line detection algorithm a projection detection algorithm
  • a low-threshold binary detection algorithm a binary detection algorithm
  • each response in the defect detection response set is one of "response” and “no response”, and each response indicates whether the corresponding defect detection algorithm detects the defect in the defect image to be detected The main feature corresponding to the defect detection algorithm exists in the defect.
  • a “response” or “non-response” response may be generated based on the magnitude relationship between the detected specific parameter related to the main feature and the preset threshold. For example, regarding the determination of "response” and “no response”, for the black and white spot detection algorithm, if the area of the defect area where the black spot or white spot defect is detected is large enough (for example, greater than a predetermined area threshold), then it is “response”, Otherwise, it is “no response”. For another example, for the Hough transform straight line detection algorithm, if the length of the line segment is detected to be sufficiently long (for example, greater than a predetermined length threshold), then it is “response”, otherwise it is “no response”. It is similarly defined for other detection algorithms.
  • Table 1 shows an example diagram of the responses of various defect detection algorithms in the defect detection algorithm set with respect to various candidate defect categories.
  • the defect detection response set corresponding to the line discharge defect and the second type of black bar defect are the same. Therefore, when the line segment defects corresponding to the two are to be subdivided, other unique information should also be considered, such as The length of the horizontal line segment.
  • the defect detection response set corresponding to the special unevenness and the white spot defect is the same, so the black and white spot detection algorithm also includes the distinction between the color of the spot and the black and white spot to obtain a black spot or white spot defect, and it is determined as a white spot.
  • the standard deviation of the gray scale value in the white spot is further calculated to distinguish whether it is a white spot defect or a fragment defect.
  • the defect detection response set corresponding to the damaged defect and the first type of black bar defect is also the same. At this time, further analysis algorithms need to be considered, which will be described later.
  • the defect category of the defect image to be detected is determined among the multiple candidate defect categories based on the defect detection response set and the priority of the multiple candidate defect categories.
  • Each type of defect on the glass substrate has a different severity of the consequences of the entire production process.
  • the staff can choose the appropriate operation according to the different defect types, so various candidate defect categories can also be determined according to the severity.
  • the priority of the production process is sorted, which is of guiding significance for the production process.
  • the priority ranking of each candidate defect category is shown in Table 2.
  • determining the defect category of the defect image to be detected among multiple candidate defect categories based on the defect detection response set and the priority of the defect category includes: starting from determining the candidate defect category with the highest priority as the current candidate category, according to Repeat the following operations in descending order of priority until the defect category of the defect image to be detected is determined: Determine the current candidate defect category, and determine whether the response corresponding to the defect detection algorithm used to detect the main feature of the current candidate defect category is "Responsive" ; In the case that the response of the defect detection algorithm used to detect the main feature of the current candidate defect category in the defect detection response set is "Responsive", the current candidate defect category is determined as the defect category of the defect image to be detected, or Determine the defect category of the defect image to be detected based on the sub-features of the main feature, where the sub-features of the main feature are used to distinguish at least two candidate defect category defects with the same main feature, and are used in the defect detection response set to detect the current When the response corresponding to the defect detection algorithm of the
  • FIG. 3 shows a schematic diagram of the process of detecting various candidate defect categories through various defect detection algorithms in the defect detection algorithm set.
  • the image binary image is further obtained, the vertical projection algorithm is performed on the image binary image, and there are black pixels in the column with black pixels. If the number is less than the sum of the pixels of the image column, the defect type of the image to be detected is determined to be a damaged defect. If the number of black pixels in the column with black pixels is less than the sum of the pixels of the image column, the defect of the image to be detected is determined
  • the defect category is the first black bar type.
  • the response of the low-threshold binarization algorithm in the defect detection response set is "Responsive"
  • further obtain the image binary image perform edge detection on the image binary image, and when the area boundary is detected as a curve , Determining that the defect category of the image to be detected is a broken defect, and when the detected area boundary is a vertical straight line and the length is equal to the column length of the defect image to be detected, the defect category of the image to be detected is determined to be the first black bar type.
  • the black and white spot detection algorithm in the defect detection response set is "Responsive"
  • the black and white spot detection algorithm determines that it is a black spot
  • the black spot detection algorithm determines that it is a white spot
  • the standard deviation of the gray value in the white spot area is further analyzed, and if the standard deviation of the gray value is less than the first preset threshold, it is determined that the defect category is a fragment defect. If the standard deviation is greater than or equal to the first preset threshold, it is determined that the defect category is a white spot defect.
  • the defect category is determined to be an area discharge defect.
  • the line segment detected by the Hough transform line detection algorithm is further analyzed, and if the horizontal length of the line segment is less than the second preset threshold, it is determined
  • the defect category of the image to be inspected is a line discharge defect, and if the horizontal length of the line segment is greater than or equal to the second preset threshold, it is determined that the defect category of the image to be inspected is a second type of black bar defect.
  • the defect category is determined to be a gray belt defect.
  • the main feature is that the gray level in the defect area is different from the surrounding area.
  • This type of defect can also be referred to as a region segmentation defect (which can be subdivided into broken defects or defects).
  • the first type of black bar defects) the main steps are: (1) Use a low preset threshold to perform binary segmentation on the defect image to be detected (threshold is optional (0, 20), and 2 is selected in the embodiment of the disclosure), and the to-be-detected
  • the gray scale of the pixels on the defective image is set to 0 or 255, that is, the entire image presents an obvious black and white effect, and find whether there are positive points (as shown in the following formula, the value of 255 pixels):
  • the binary image of the defect image to be detected that has been processed by the low-threshold binarization detection algorithm is further obtained, and the Binary graph performs edge detection, and when the area boundary is detected as a curve, the final defect category is determined to be a broken defect, and when the area boundary is detected as a vertical straight line and the length is equal to the column length of the image, the defect is determined
  • the category is the first black bar type.
  • the number of black pixels in certain column directions is generally less than the total number of column pixels in the image, while the number of black pixels with the first type of black stripe defect Will be equal to the total number of column pixels of the image, so vertical projection (on the basis of the binary image) can also be used to distinguish these two types of defects.
  • the defect category of the image to be detected is determined to be a damaged defect. If the number of black pixels in the column with black pixels is equal to (or in If the error range is equal to the sum of the pixels of the image column, it is determined that the defect category of the image to be detected is the first black bar type.
  • the main feature is the defect with the pixel gray value of the defect area being zero.
  • This type of defect can also be called a patch defect (which can be subdivided into special uneven defects, fragment defects) And white spot defects)
  • the main steps are: (1) Take the column as a unit to find the median value of a column of pixel grayscale values of the defect image to be detected.
  • Set the threshold as the median-n (the threshold used to detect white spots is correspondingly median + n), where the value of n depends on the specific conditions of the image.
  • the n used in this disclosure is 20, and the formula is:
  • the edge density detection algorithm As the previous analysis, the main feature is the defect with a large range of crack-like patterns in one or several adjacent panels, including area discharge defects.
  • the edge is obtained by first using the edge detection algorithm, and then the edge length is counted. The edge length is higher than the preset threshold to determine this type.
  • the specific steps are: (1) Use an edge detection algorithm (such as Canny operator) to inspect the edge of the defect image to be detected; (2) In each panel, divide the number of pixels at the edge point by the total number of pixels on the panel; 3) Establish a preset threshold to distinguish the ratio. When the preset threshold is exceeded, it indicates that there is a regional discharge defect, so the corresponding response of the edge density detection algorithm is "response".
  • an edge detection algorithm such as Canny operator
  • edge detection algorithms there are many types of edge detection algorithms.
  • Canny operator Sobel operator, Laplace operator, etc. can also be used for edge detection.
  • Canny operator is currently recognized as a very effective edge detection algorithm.
  • the Hough transform straight line detection algorithm As the previous analysis, it mainly focuses on the defects whose main feature is the line segment, including the line discharge defect and the second type of black stripe defect.
  • the specific steps are: (1) Use an edge detection algorithm (such as Canny operator) to detect the edge of the defect image to be detected; (2) Use the Hough line transform to find the straight line in the edge, and when the line is found, the Hough transform line detection algorithm The corresponding response is “response”; (3) In the found straight line, the coordinate value of the endpoint is also used to distinguish the horizontal and vertical straight lines; (4) If the line segment is in the horizontal direction, the length of the line segment is further calculated, and the line The segment length within the range of the splicing width between one panel ⁇ 2 pixels is judged to be the second type of black stripe defect, and the others are line discharge defects.
  • the splicing length between panels is known.
  • the main feature is the defect of the blocky area with abnormal color in the vertical direction, including the gray band defect. Therefore, using the projection algorithm to calculate the total value of each column of pixels, and analyze whether there is a large change in a shorter area, you can distinguish this type.
  • the specific steps are: (1) The grayscale values of each column of the defect image to be detected are respectively accumulated to obtain a one-dimensional vector, and each element of the vector corresponds to the sum of the grayscale values of the column; (2) Pair the obtained One-dimensional vector uses low-pass filtering (such as median filtering); (3) Find the position in the low-pass component of the vector that changes more than m within the width of n pixels, if found, it means that there is the first type of gray band Defects, so that the response corresponding to the vertical projection detection algorithm is "response".
  • n and m used in the present disclosure are 3 and 200, respectively.
  • FIG. 4 is a block diagram of a defect detection device for a defect image according to an embodiment of the present disclosure. Since the operation performed by the defect detection apparatus of this embodiment is the same as the details of the method described above, a detailed description of the same content is omitted here for the sake of simplicity.
  • the defect detection device 400 includes a processor 401 and a memory 402. It should be noted that although the defect detection device in FIG. 4 is shown as including only two devices, this is only illustrative, and the defect detection device may also include one or more other devices.
  • the memory 402 is used to store computer-executable instructions, which, when run by the processor, cause the processor to execute the steps of the method described above.
  • the present disclosure also provides a computer-readable storage medium with program instructions stored thereon, and the stored program instructions can be read and executed by a processor (for example, the processor 401), so that the processor executes as described above. Steps of the method.
  • a processor for example, the processor 401

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Image Analysis (AREA)

Abstract

提供了缺陷图像的缺陷检测方法、装置和计算机可读介质，包括：获取基板图作为待检测缺陷图像；利用缺陷检测算法集中的每种缺陷检测算法分别对所述待检测缺陷图像进行缺陷检测并生成对应的响应，得到缺陷检测响应集，其中所述缺陷检测算法集包括至少两种缺陷检测算法；以及基于所述缺陷检测响应集以及多种候选缺陷类别的优先级来在多种候选缺陷类别中确定所述待检测缺陷图像的缺陷类别。

Description

缺陷图像的缺陷检测方法、装置及计算机可读存储介质 技术领域
本公开涉及一种缺陷图像的缺陷检测方法、装置及计算机可读存储介质。
背景技术
在产品制造过程中，例如半导体产品的制造过程中，由于设备、参数、操作、环境干扰等环节存在的问题，会导致产出的产品不符合工艺要求甚至导致不良出现，所以需要在每道工序后及时把不符合要求的不良缺陷产品的不良种类、不良大小、位置等信息计算识别出来，进行及时的修正和改善，避免不良的继续产生。
目前图像分析可用在产线上用于质量把控。图像中最重要的为AOI彩图，这是对电路层面不良的检测。基板层面的数字宏观(Digital Macro，DM)图是对玻璃基板层面不良的检测，用于监控面板加工过程中玻璃基板的完好情况，是质量把控的重要补充手段。作业员花在DM图的缺陷判级上的时间和精力远低于AOI彩图，但DM图一旦出现问题都是非常严重的事故级不良，对DM图的错判和漏判，轻则导致浪费后续的加工时间，重则导致损坏后续的加工设备。DM图的缺陷包括破损、碎片、区域放电、线放电、特殊不均、白点、黑条、灰带。
目前传统的对图像(包括AOI彩图、DM图等)中的缺陷的识别方法主要是依靠人工检测。这要求对检测人员进行专业培训。尤其在生产的产品型号多、问题复杂的情况下，需要检测人员投入较长时间和专注力去进行缺陷查找和有关判断。
综上，现有技术手段解决上述问题存在效率较低，准确度较低的问题。
发明内容
根据本公开的一方面，提供了一种缺陷图像的缺陷检测方法，包括：获取基板图作为待检测缺陷图像；利用缺陷检测算法集中的每种缺陷检测算法分别对所述待检测缺陷图像进行缺陷检测并生成对应的响应，得到缺陷检测响应集，其中所述缺陷检测算法集包括至少两种缺陷检测算法；以及基于所述缺 陷检测响应集以及多种候选缺陷类别的优先级来在多种候选缺陷类别中确定所述待检测缺陷图像的缺陷类别。
根据本公开的另一方面，提供了一种缺陷图像的缺陷检测装置，包括：处理器；和存储器，其上存储有计算机可用指令，所述指令在由所述处理器执行时，使得所述处理器执行如上所述的方法中的步骤。
根据本公开的又一方面，还提供了一种计算机可读存储介质，其上存储有程序指令，所存储的程序指令可由处理器读取并执行，以使所述处理器执行如上所述的方法中的步骤。
附图说明
为了更清楚地说明本公开至少一实施例的技术方案，下面将对实施例的描述中所需要使用的附图作简单的介绍。下面描述中的附图仅仅是本公开的示例性实施例。
图1(a)-(i)示出了DM图中的几种示例候选缺陷类别对应的缺陷示意图；
图2示出了根据本公开实施例的缺陷图像的缺陷检测方法的流程图；
图3示出了根据本公开实施例的通过缺陷检测算法集中的各种缺陷检测算法判断各种候选缺陷类别的流程示意图；以及
图4示出了根据本公开实施例的缺陷图像的缺陷检测装置的框图。
具体实施方式
以下将参照附图更充分地描述本公开实施例，在附图中示出了本公开实施例。然而，可以用很多不同形式来实施本公开，并且本公开不应理解为受限于在此所阐述的实施例。在全文中，使用相似的标号表示相似的元件。
在此所使用的术语仅用于描述特定实施例的目的，而并非意欲限制本公开。如在此所使用的那样，单数形式的“一个”、“这个”意欲同样包括复数形式，除非上下文清楚地另有所指。还应当理解，当在此使用时，术语“包括”指定出现所声明的特征、整体、步骤、操作、元件和/或组件，但并不排除出现或添加一个或多个其它特征、整体、步骤、操作、元件、组件和/或其群组。
除非另外定义，否则在此所使用的术语(包括技术术语和科学术语)具有与本公开所属领域的普通技术人员所共同理解的相同意义。在此所使用的术 语应解释为具有与其在该说明书的上下文以及有关领域中的意义一致的意义，而不能以理想化的或过于正式的意义来解释，除非在此特意如此定义。以下参照示出根据本公开实施例的方法、装置(系统)和/或计算机程序产品的框图和/或流程图描述本公开。应理解，可以通过计算机程序指令来实现框图和/或流程图示图的一个方框以及方框的组合。可以将这些计算机程序指令提供给通用计算设备、专用计算设备的处理器和/或其它可编程数据处理装置，使得经由计算设备处理器和/或其它可编程数据处理装置执行的指令创建用于实现框图和/或流程图块中所指定的功能/动作的方法。
相应地，还可以用硬件和/或软件(包括固件、驻留软件、微码等)来实施本公开。更进一步地，本公开可以采取计算机可使用或计算机可读存储介质上的计算机程序产品的形式，其具有在介质中实现的计算机可使用或计算机可读程序代码，以由指令执行系统来使用或结合指令执行系统而使用。在本公开上下文中，计算机可使用或计算机可读介质可以是任意介质，其可以包含、存储、通信、传输、或传送程序，以由指令执行系统、装置或设备使用，或结合指令执行系统、装置或设备使用。
目前，如前面所述，人工检测存在诸多的风险和不稳定因素等，均会导致检测过程整体质量的下降，从而给产品质量带来隐患。另一方面，检测过程中，所有数据均是手动录入，效率低下，同时人工在有限的时间内在待检测产品的图像上获取的信息颗粒度较粗，对后续的缺陷原因查找和分析带来不便。
此外，随着人工智能图像识别分类技术和大数据技术的不断发展和应用，人工智能图像分类技术可以不间断的自学习和利用更多分完类的样本训练和加强整体的模型识别能力，使系统的准确度越来越好，同时保证稳定性。但是人工智能图像识别的分类技术对数据的需求高，并且对计算机的计算能力以及资源需求也比较高。例如，在具体设计中，由于不良图像数据可能难以收集，用于训练的样本数量有限，很容易出现模型过拟合等问题，这在一定程度上限制了其使用。
基于上述全部或部分的原因，本公开提出一种用于基板图(例如DM图)的缺陷检测方法、装置以及计算机可读存储介质。本公开提出的缺陷检测方法考虑到DM图不良图片难以收集的情况，使用传统图像处理算法，人工预先设计用于区分不同表现形式的缺陷的主特征，针对用于区分各种缺陷的主特征而选择不同的缺陷检测算法，从而根据各个缺陷检测算法的检测结果来对 待检测缺陷图像中的缺陷进行检测。与人工智能中的神经网络算法相比，传统的图像处理算法对数据数量要求较低，可以解决难以跨越的数据集收集问题。
此外，由于DM图是玻璃基板层面的图像，而玻璃基板层面的缺陷较少，因此，一般一张DM图上只有一个缺陷，并且下面的描述也是基于对一张DM图上的该一个缺陷进行检测来进行的。
同时，虽然本文以DM图为例来描述各种实施例，但这仅仅是示例性的，并且不应被理解为对本公开实施例的限制。本领域技术人员可以容易地理解，本公开提出的各种实施例也可以用于其他类型的基板图。
图1(a)-(i)示出了DM图中的几种示例候选缺陷类别对应的缺陷示意图。
图1(a)示出了破损(Broken)缺陷，该缺陷是指玻璃基板破碎。
图1(b)示出了碎片(Piece)缺陷，该缺陷是指玻璃基板上存在玻璃碎片。
图1(c)示出了区域放电(Area Arcing)缺陷，该缺陷是指玻璃基板上存在大范围的放电不良，通常遍布基板上的一个或几个面板。
图1(d)示出了线放电(Line Arcing)缺陷，该缺陷是指由于某条或某几条栅极/数据线的放电在玻璃基板上可看到的一条或几条异常线，通常在水平方向上。
图1(e)示出了特殊灰阶不均(Special Gray Aldistribution)缺陷(后文在某些地方也简称为特殊不均)，该缺陷是指玻璃基板上存在边界不规则的灰色区域，有时会形成特殊图案。
图1(f)示出了白点缺陷(White Pot)缺陷，该缺陷是指玻璃基板上存在白色残留。
图1(g)-(h)示出了黑条(Black Slip)缺陷，该缺陷是指玻璃基板上存在黑色条带(第一类型黑条图1(g))或条状线条(第二类型黑条图1(f))。一般由线路问题或者图像合成问题导致。
图1(i)示出了灰带(Gray Gap)缺陷，该缺陷是指玻璃基板上存在有明显的明暗条纹。
需要注意的是，上面几种候选缺陷类别的缺陷仅仅是示例性的，本领域技术人员可以根据实际场合和实际需求而对缺陷进行其他分类，这不脱离本公开的范围。
对于每种表现形式的缺陷，都有各自的主特征，并且至少两种候选缺陷类别具有相同主特征。该至少两种候选缺陷类别在表现形式方面存在相似的特征，因此可以将该相似的特征认为是它们共有的(或相同的)主特征。
例如，特殊不均缺陷、碎片缺陷以及白点缺陷这三种候选缺陷类别缺陷的主特征为缺陷区域的灰度不同于周围区域；区域放电缺陷的主特征为在一个或相邻几个面板内有大范围的裂纹状图案；第二类型黑条缺陷与线放电缺陷这两种候选缺陷类别缺陷的主特征为线段；灰带缺陷的主特征为在竖直方向会存在线条或者颜色异常的块状区域；并且第一类型黑条缺陷和破损缺陷这两种候选缺陷类别缺陷的主特征为在缺陷区域内的像素灰度值为零。
对于各种候选缺馅类别缺陷，都有适用于检测该候选缺陷类别缺陷的主特征的缺陷检测算法。即，对于每种主特征，都有一种对应的缺陷检测算法。
例如，黑白斑检测算法用于检测主特征为缺陷区域中的灰度不同于周围区域的候选缺陷类别缺陷，包括特殊不均缺陷、碎片缺陷以及白点缺陷；边缘密度检测算法用于检测主特征为在一个或相邻几个面板内有大范围的裂纹状图案的候选缺陷类别缺陷，包括区域放电缺陷；霍夫变换直线检测算法用于检测主特征为线段的候选缺陷类别缺陷，包括第二类型黑条缺陷与线放电缺陷；竖直投影检测算法用于检测主特征为在竖直方向会存在线条或者颜色异常的块状区域的候选缺陷类别缺陷，包括灰带缺陷；并且低阈值二值化检测算法用于检测主特征为在缺陷区域内的像素灰度值为零的候选缺陷类别缺陷，包括第一类型黑条缺陷和破损缺陷。
而对于具有相同主特征的至少两种候选缺陷类别缺陷，还需要基于主特征的子特征来对该至少两种候选缺陷类别缺陷进行进一步地区分，以确定最终的缺陷类别。例如，主特征为缺陷区域的灰度不同于周围区域的缺陷包括特殊不均缺陷、碎片缺陷以及白点缺陷这三种候选缺陷类别缺陷时，且该主特征的子特征为斑块颜色以及白斑内的灰阶值标准差；主特征为线段缺陷包括第二类型黑条缺陷与线放电缺陷这两种候选缺陷类别缺陷时，该主特征的子特征为线段的方向和长度；并且主特征为在缺陷区域内的像素灰度值为零的缺陷包括第一类型黑条缺陷和破损缺陷这两种候选缺陷类别缺陷时，该主特征的子特征为图像二值图中存在黑像素的列中的黑像素个数、或者图像二值图中缺陷区域边界的形状和长度。
本公开在对DM图(待检测缺陷图像)中的缺陷进行检测时，同时通过 各种缺陷检测算法分别对DM图中的缺陷进行检测，根据检测结果来确定DM图中的缺陷属于哪种缺陷类别。
此外，DM图是一张合成的灰阶图，由相机逐列拍摄最终拼接而成。除了在横竖方向都有比较明显的面板边缘以外，由于相机拍摄的照片在水平方向上亮度不均导致在竖直方向还有等间距的拼接痕迹。由于拼接痕迹不是非常明显，对算法影响较小，因此可以将其忽略。但是，面板边缘在灰阶值上与边缘附近的像素差异较大，会对许多缺陷检测算法造成干扰，因此要设计面板边缘去除算法，这将在后文具体介绍。
图2示出了根据本公开实施例的缺陷图像的缺陷检测方法的流程图。
在步骤210，获取基板图作为待检测缺陷图像。
可选地，该基板图可以为DM灰阶图，并且将基板图作为待检测缺陷图像包括：读取DM图，将该DM图重设大小。此外，DM图本身就是灰阶图，读取DM图时可选择将其灰阶值读取为整型或浮点型；整型的常用灰阶值范围为[0，255](无符号8-bit)，但是[0，65535](无符号16-bit)、[-32768，32767](有符号16-bit)等也是可行的；浮点型的常用灰阶值为[0.0,1.0](32位、64位常用值均为[0.0，1.0]，但也可用其他值如[-1.0，2.0])。由于DM图原图的像素值不固定，但比例接近1:1.15，因此可选用一对长宽值对原图重设大小，例如(400，460)、(600，690)、(800，920)等(为了避免某些小缺陷在重采样时消失，一般不使用更小的像素值)。作为示例，本公开选用的灰阶值范围为[0，255](无符号8-bit)，DM图的重设大小为(600，690)。
此外，由于面板边缘在灰阶值上与边缘附近的像素差异较大，会对许多缺陷检测算法造成干扰，因此可选地可以设计面板边缘去除算法以提高检测的准确性。
可选地，去除基板上的面板边缘主要包括以下步骤：1)对DM图的每一行和每一列的灰阶值分别实施累加操作，分别得到一个一维向量，一维向量的每个元素对应该行或该列的灰阶值总和；(2)对得到的一维向量采用低通滤波器(本公开中以中值滤波器为例，其他的常用低通滤波器还有均值滤波器、巴特沃斯低通滤波器、高斯低通滤波器等)；(3)使用原一维向量减去低频分量(对原向量使用低通滤波得到的向量)得到原向量的高频分量；(4)对高频分量使用某个阈值进行分割，超过阈值的元素位置即被认为是面板边缘；(5)去除DM图中面板边缘位置邻域的几个像素宽度的几行和几列。
在步骤220，利用缺陷检测算法集中的每种缺陷检测算法分别对待检测缺陷图像进行缺陷检测并生成对应的响应，得到缺陷检测响应集，其中缺陷检测算法集包括至少两种缺陷检测算法。
可选地，该缺陷检测算法集包括黑白斑检测算法、边缘密度检测算法、霍夫变换直线检测算法、投影检测算法、低阈值二值化检测算法。这几种检测算法所针对的主特征已在前文描述，这里不再重复。
可选地，缺陷检测响应集中的每个响应为“有响应”和“无响应”中的一种，每个响应分别指示对应的缺陷检测算法在对待检测缺陷图像的缺陷进行检测时是否检测到缺陷中存在该缺陷检测算法对应的主特征。
可选地，可以基于检测到的与主特征相关的特定参数与预设阈值的大小关系来生成“有响应”或“无响应”的响应。例如，关于“有响应”和“无响应”的判定，对于黑白斑检测算法，如果检测到黑斑或白斑缺陷的缺陷区域面积足够大(例如，大于预定面积阈值)则为“有响应”，反之则为“无响应”。再例如，对于霍夫变换直线检测算法，如果检测到线段长度足够长(例如，大于预定长度阈值)则为“有响应”，反之则为“无响应”。对于其他检测算法也类似定义。
同时，为了帮助更好地理解本公开，表1示出了缺陷检测算法集中的各种缺陷检测算法相对于各种候选缺陷类别的响应的一种示例图。
表1
Figure PCTCN2020076958-appb-000001
Figure PCTCN2020076958-appb-000002
在表1中，“○”表示“有响应”，以及“×”表示无响应。
此外，在表1中，线放电缺陷以及第二类型黑条缺陷对应的缺陷检测响应集是相同的，因此在要对两者对应的线段缺陷进行细分时，还应该考虑其他独特信息，例如水平线段长度。特殊不均以及白点缺陷对应的缺陷检测响应集是相同的，因此在黑白斑检测算法中还包括对斑块颜色对黑白斑进行区分，以得到黑斑缺陷或者白斑缺陷，并且在确定为白斑时，进一步对白斑内的灰阶值标准差进行计算，从而区分是白点缺陷还是碎片缺陷。同时，破损缺陷和第一类型黑条缺陷对应的缺陷检测响应集也是相同的，此时也需要考虑进一步的分析算法，这将在后文进行描述。
在步骤230，基于缺陷检测响应集以及多种候选缺陷类别的优先级来在多种候选缺陷类别中确定所述待检测缺陷图像的缺陷类别。
玻璃基板上的每种缺陷类别对整个生产过程所带来的后果的严重性是不同的，工作人员可以根据不同的缺陷类别而选择适当的操作，因此还可以根据严重程度对各种候选缺陷类别的优先级进行排序，这对于生产过程具有指导意义。
每种候选缺陷类别的优先级的排序如表2所示。
表2
优先级 缺陷种类 中文名称
1 Broken 破损
2 Piece 碎片
3 Area Arcing 区域放电
4 Line Arcing 线放电
5 SGM 特殊不均
6 White Pot 白点
7 Black Slip I 第一类型黑条
8 Black Slip II 第二类型黑条
9 Gray Gap 灰带
可选地，基于缺陷检测响应集以及缺陷类别的优先级来在多种候选缺陷类别中确定待检测缺陷图像的缺陷类别包括：从将最高优先级的候选缺陷类别确定为当前候选类别开始，按照优先级降序重复以下操作，直到确定了待检测缺陷图像的缺陷类别：确定当前候选缺陷类别，判断用于检测当前候选缺陷类别缺陷的主特征的缺陷检测算法所对应的响应是否为“有响应”；在缺陷检测响应集中用于检测当前候选缺陷类别缺陷的主特征的缺陷检测算法所对应的响应为“有响应”的情况下，将当前候选缺陷类别确定为待检测缺陷图像的缺陷类别，或者基于主特征的子特征来确定待检测缺陷图像的缺陷类别，其中主特征的子特征用于将具有相同主特征的至少两种候选缺陷类别缺陷进行区分，并且在缺陷检测响应集中用于检测当前候选缺陷类别缺陷的主特征的缺陷检测算法所对应的响应为“无响应”的情况下，将下一优先级的候选缺陷类别确定为当前候选缺陷类别。
下面具体描述各种缺陷检测算法用于检测待检测缺陷图像中的缺陷的过程。图3示出了通过缺陷检测算法集中的各种缺陷检测算法检测各种候选缺陷类别的流程示意图。
在利用缺陷检测算法集中的每种缺陷检测算法分别对待检测缺陷图像进行缺陷检测并生成对应的响应，得到缺陷检测响应集之后，按照如表2所示的缺陷类别优先级，从最高的破损缺陷开始进行如下判断。
如果缺陷检测响应集中低阈值二值化算法对应的响应为“有响应”时，进一步获取图像二值图，对图像二值图执行竖直投影算法，并且在存在黑像素的列中黑像素个数少于图像列像素的总和的情况下，确定待检测图像的缺陷类别是破损缺陷，在存在黑像素的列中黑像素个数少于图像列像素的总和的情况下，确定待检测图像的缺陷类别是第一黑条类型。
或者，如果缺陷检测响应集中低阈值二值化算法对应的响应为“有响应”时，进一步获取图像二值图，对图像二值图进行边缘检测，并且在检测到区域 边界为曲线的情况下，确定待检测图像的缺陷类别是破损缺陷，并且在检测到区域边界为竖直直线并且长度等于待检测缺陷图像的列长度的情况下，确定待检测图像的缺陷类别是第一黑条类型。
在不存在破损缺陷的情况下，进行是否存在碎片缺陷的判断。
如果缺陷检测响应集中黑白斑检测算法对应的响应为“有响应”时，并且在通过黑白斑检测算法确定是黑斑的情况下，确定待检测图像的缺陷类别是特殊不均，并且在通过黑白斑检测算法确定是白斑的情况下，进一步对白斑区域内的灰度值标准差进行分析，并且如果灰度值标准差小于第一预设阈值，则确定缺陷类别是碎片缺陷，如果灰度值标准差大于等于第一预设阈值，则确定缺陷类别是白点缺陷。
在不存在碎片缺陷的情况下，进行是否存在区域放电缺陷的判断。
如果缺陷检测响应集中边缘密度检测算法对应的响应为“有响应”，则确定缺陷类别是区域放电缺陷。
在不存在区域放电缺陷的情况下，进行是否存在线放电缺陷的判断。
如果缺陷检测响应集中霍夫变换直线检测算法对应的响应为“有响应”，进一步对通过霍夫变换直线检测算法检测到的线段进行分析，并且如果线段水平长度小于第二预设阈值，则确定待检测图像的缺陷类别是线放电缺陷，并且如果线段水平长度大于等于第二预设阈值，则确定待检测图像的缺陷类别是第二类型黑条缺陷。
当上面提及的所有候选缺陷类别都未被检测到时，则进一步判断是否存在灰带缺陷。
如果缺陷检测响应集中仅有竖直投影检测算法对应的响应为“有响应”，则确定缺陷类别是灰带缺陷。
为了更完整的公开本公开的实施例，下面对各种缺陷检测算法进行详细描述。
对于低阈值二值化算法：如前面分析，主要针对主特征为缺陷区域中的灰度不同于周围区域的缺陷，该类缺陷后文也可以称为区域分割缺陷(可细分为破损缺陷或第一类型黑条缺陷)，主要步骤为：(1)使用低预设阈值对待检测缺陷图像进行二值分割(阈值可选(0，20]，本公开实施例中选择2)，将待检测缺陷图像上的像素点的灰阶置为0或255，也就是将整个图像呈现出明显的黑白效果，查找是否存在阳性点(如下公式所示，值为255的像素点)：
Figure PCTCN2020076958-appb-000003
(2)使用数字图像处理中的形态学闭操作去除尺度较小的阳性点区域(使用不同大小的核对待检测缺陷图像整体或是局部进行处理，闭操作可以将小于核大小的阳性点区域去除，核大小以像素为单位，本文选用(10，10))，本步骤为可选步骤；(3)寻找待检测缺陷图像的二值图中是否存在区域分割缺陷。由于低阈值二值化算法针对的是区域分割缺陷，此类缺陷一般在二值图中的像素为0的区域较大，因此如果存在分割的区域，则低阈值二值化算法对应的响应为“有响应”。
此外，在确定低阈值二值化算法对该待检测缺陷图像(的缺陷)“有响应”时，进一步获取经低阈值二值化检测算法处理后的待检测缺陷图像的二值图，并对二值图进行边缘检测，并且在检测到区域边界为曲线的情况下，确定最终缺陷类别是破损缺陷，并且在检测到区域边界为竖直直线并且长度等于图像的列长度的情况下，确定缺陷类别是第一黑条类型。
或者，由于如果是破损缺陷，那么经过低阈值二值化算法处理后，在某些列方向上黑像素的个数一般小于图像的列像素总数，而第一类型黑条缺陷黑像素的个数会等于图像的列像素的总数，因此也可以采用竖直投影(在二值图的基础上)来对这两种缺陷进行区分。也就是说，如果存在黑像素的列中黑像素个数少于图像列像素的总数，则确定待检测图像的缺陷类别是破损缺陷，如果存在黑像素的列中黑像素个数等于(或在误差范围内等于)图像列像素的总和，则确定待检测图像的缺陷类别是第一黑条类型。
对于黑白斑检测算法：如前面分析，主要针对主特征为缺陷区域内的像素灰度值为零的缺陷，该类缺陷也可以称为斑块缺陷(可细分为特殊不均缺陷、碎片缺陷和白点缺陷)，主要步骤为：(1)以列为单位，寻找待检测缺陷图像的一列像素灰阶值的中值，由于灰阶值越小表现为越黑，检测黑斑使用的预设阈值为中值-n(检测白斑使用的阈值相应为中值+n)，其中n值根据图像的具体情况而定，本公开中使用的n为20，公式为：
Figure PCTCN2020076958-appb-000004
(2)使用形态学闭操作去除某些较小的阳性点区域，主要针对可能出现线放 电或者黑条缺陷等导致的线状阳性点区域，本文选用的核大小为(5，5)(白斑检测无此步骤，因白点中可能出现尺寸较小的斑点，避免将其去除)；(3)使用边缘检测算法获得响应的边缘和位置信息；(4)通过斑块颜色区分出是黑斑缺陷还是白斑缺陷，从而黑白斑检测算法对应的响应为“有响应”；(5)检测到白斑并获得白斑位置后，对白斑内的像素求取灰阶值的标准差，并将得到的标准差与预设阈值进行比较，以用于区分碎片和白点。
对于边缘密度检测算法：如前面分析，主要针对主特征为在一个或相邻几个面板内有大范围的裂纹状图案的缺陷，包括区域放电缺陷。通过先用边缘检测算法获得边缘，再对边缘长度进行统计，边缘长度高于预设阈值即可判出此类。具体步骤为：(1)使用边缘检测算法(例如Canny算子)检侧待检测缺陷图像的边缘；(2)在每一个面板内部，使用边缘点的像素数除以面板的总像素数；(3)设立预设阈值区分该比例，当超过预设阈值时，说明存在区域放电缺陷，从而边缘密度检测算法对应的响应为“有响应”。
值得注意的是，边缘检测算法种类较多，除了上述Canny算子之外，如Sobel算子、Laplace算子等也可以用于边缘检测。Canny算子是目前公认的效果很好的边缘检测算法。
对于霍夫变换直线检测算法：如前面分析，主要针对主特征为线段的缺陷，包括线放电缺陷和第二类型黑条缺陷。具体步骤为：(1)使用边缘检测算法(例如Canny算子)检测待检测缺陷图像的边缘；(2)使用霍夫线变换寻找边缘中的直线，并且在找到直线时霍夫变换直线检测算法对应的响应“有响应”；(3)在找到的直线中，还利用端点的坐标值区分水平方向和竖直方向的直线；(4)如果线段在水平方向，则进一步计算线段长度，并且在线段长度在一个面板间拼接宽度±2像素范围内的判为第二类型黑条缺陷，其他为线放电缺陷，面板间拼接长度是已知的。
对于投影算法：如前面分析，主要针对主特征为在竖直方向会存在颜色异常的块状区域的缺陷，包括灰带缺陷。因此使用投影算法计算每一列像素的总值，并分析是否有在较短的区域内有较大变化的情况，即可分辨此类。具体步骤为：(1)对待检测缺陷图像的每列的灰阶值分别实施累加操作，得到一个一维向量，向量的每个元素分别对应该列的灰阶值总和；(2)对得到的一维向量采用低通滤波(例如中值滤波)；(3)在向量的低通分量中查找在n个像素的宽度内变化幅度超过m的位置，如果找到，则说明存在第一类型灰带缺陷， 从而竖直投影检测算法对应的响应为“有响应”。例如，本公开使用的n，m分别为3和200。
通过本公开实施例中的缺陷检测方法，针对用于区分各种缺陷的典型特征而选择不同的图像处理算法，从而对缺陷图像中的缺陷进行检测。传统的图像处理算法对数据数量要求较低，可以解决难以跨越的数据集收集问题。
下面将参照图4描述根据本公开实施例的缺陷检测装置。图4是根据本公开实施例的用于缺陷图像的缺陷检测装置的框图。由于本实施例的缺陷检测装置执行的操作与在上文中描述的方法的细节相同，因此在这里为了简单起见，省略对相同内容的详细描述。
如图4所示，缺陷检测装置400包括处理器401和存储器402。需要注意的是，尽管在图4中缺陷检测装置被示出为只包括2个装置，但这只是示意性的，缺陷检测装置也可以包括一个或多个其他装置。
图4中，存储器402用于存储计算机可执行指令，该计算机可执行指令在被处理器运行时使处理器执行如前面所述的方法的各个步骤。
此外，本公开还提供了一种计算机可读存储介质，其上存储有程序指令，所存储的程序指令可由处理器(例如处理器401)读取并执行，以使处理器执行如前面所述的方法的各个步骤。
虽然已经针对本公开的各种具体示例实施例详细描述了本公开，但是每个示例通过解释而不是限制本公开来提供。本领域技术人员在得到对上述内容的理解后，可以容易地做出这样的实施例的变更、变化和等同物。因此，本发明并不排除包括将对本领域普通技术人员显而易见的对本公开的这样的修改、变化和/或添加。例如，作为一个实施例的一部分图示或描述的特征可以与另一实施例一起使用，以产生又一实施例。因此，意图是本公开覆盖这样的变更、变化和等同物。
具体地，尽管本公开的附图出于图示和讨论的目的分别描述了以特定顺序执行的步骤，但是本公开的方法不限于特定图示的顺序或布置。在不偏离本公开的范围的情况下，上述方法的各个步骤可以以各种方式省略、重新布置、组合和/或调整。
本领域技术人员可以理解，本申请的各方面可以通过若干具有可专利性的种类或情况进行说明和描述，包括任何新的和有用的工序、机器、产品或物质的组合，或对他们的任何新的和有用的改进。相应地，本申请的各个方面可 以完全由硬件执行、可以完全由软件(包括固件、常驻软件、微码等)执行、也可以由硬件和软件组合执行。以上硬件或软件均可被称为“数据块”、“模块”、“引擎”、“单元”、“组件”或“系统”。此外，本申请的各方面可能表现为位于一个或多个计算机可读介质中的计算机产品，该产品包括计算机可读程序编码。
除非另有定义，这里使用的所有术语(包括技术和科学术语)具有与本公开所属领域的普通技术人员共同理解的相同含义。还应当理解，诸如在通常字典里定义的那些术语应当被解释为具有与它们在相关技术的上下文中的含义相一致的含义，而不应用理想化或极度形式化的意义来解释，除非这里明确地这样定义。
以上是对本公开的说明，而不应被认为是对其的限制。尽管描述了本公开的若干示例性实施例，但本领域技术人员将容易地理解，在不背离本公开的新颖教学和优点的前提下可以对示例性实施例进行许多修改。因此，所有这些修改都意图包含在权利要求书所限定的本公开范围内。应当理解，上面是对本公开的说明，而不应被认为是限于所公开的特定实施例，并且对所公开的实施例以及其他实施例的修改意图包含在所附权利要求书的范围内。本公开由权利要求书及其等效物限定。

Claims (14)

  1. 一种缺陷图像的缺陷检测方法，包括：
    获取基板图作为待检测缺陷图像；
    利用缺陷检测算法集中的每种缺陷检测算法分别对所述待检测缺陷图像进行缺陷检测并生成对应的响应，得到缺陷检测响应集，其中所述缺陷检测算法集包括至少两种缺陷检测算法；以及
    基于所述缺陷检测响应集以及多种候选缺陷类别的优先级来在多种候选缺陷类别中确定所述待检测缺陷图像的缺陷类别。
  2. 根据权利要求1所述的方法，
    其中，所述缺陷检测方法集中的每种缺陷检测方法用于检测一种主特征，所述主特征是能够区分具有不同表现形式的缺陷的缺陷特征，且至少两种候选缺陷类别缺陷具有相同主特征，
    其中，所述缺陷检测响应集中的每个响应为“有响应”和“无响应”中的一种，每个响应分别指示对应的缺陷检测算法在对待检测缺陷图像的缺陷进行检测时是否检测到缺陷中存在该缺陷检测算法对应的主特征。
  3. 根据权利要求2所述的方法，其中，基于所述缺陷检测响应集以及多种候选缺陷类别的优先级来在多种候选缺陷类别中确定所述待检测缺陷图像的缺陷类别包括：
    从将最高优先级的候选缺陷类别确定为当前候选类别开始，按照优先级降序重复以下操作，直到确定了所述待检测缺陷图像的缺陷类别：
    确定当前候选缺陷类别，
    判断用于检测当前候选缺陷类别缺陷的主特征的缺陷检测算法所对应的响应是否为“有响应”：
    在缺陷检测响应集中用于检测当前候选缺陷类别缺陷的主特征的缺陷检测算法所对应的响应为“有响应”的情况下，将当前候选缺陷类别确定为所述待检测缺陷图像的缺陷类别，或者基于主特征的子特征来确定所述待检测缺陷图像的缺陷类别，其中所述主特征的子特征用于将具有相同主特征的至少两种候选缺陷类别缺陷进行区分，并且
    在所述缺陷检测响应集中用于检测当前候选缺陷类别缺陷的主特征 的缺陷检测算法所对应的响应为“无响应”的情况下，将下一优先级的候选缺陷类别确定为当前候选缺陷类别。
  4. 根据权利要求3所述的方法，其中，所述缺陷检测算法集包括黑白斑检测算法、边缘密度检测算法、霍夫变换直线检测算法、竖直投影检测算法、低阈值二值化检测算法，
    其中，所述黑白斑检测算法用于检测主特征为缺陷区域的灰度不同于周围区域的缺陷；所述边缘密度检测算法用于检测主特征为在一个或相邻几个面板内有大范围的裂纹状图案的缺陷；所述霍夫变换直线检测算法用于检测主特征为线段的缺陷；所述竖直投影检测算法用于检测主特征为在竖直方向会存在线条或者颜色异常的块状区域的缺陷；并且所述低阈值二值化检测算法用于检测主特征为在缺陷区域内的像素灰度值为零的缺陷。
  5. 根据权利要求4所述的方法，其中，
    多种候选缺陷类别按优先级降序排序包括：破损缺陷、碎片缺陷、区域放电缺陷、线段缺陷、特殊不均缺陷、白点缺陷、第一类型黑条缺陷、第二类型黑条缺陷、和灰带缺陷。
  6. 根据权利要求5所述的方法，其中，
    主特征为缺陷区域的灰度不同于周围区域的缺陷包括特殊不均缺陷、碎片缺陷以及白点缺陷，且该主特征的子特征为斑块颜色以及白斑内的灰阶值标准差；
    主特征为在一个或相邻几个面板内有大范围的裂纹状图案的缺陷包括区域放电缺陷；
    主特征为线段的缺陷包括第二类型黑条缺陷与线放电缺陷，且该主特征的子特征为线段的方向和长度；
    主特征为在竖直方向会存在线条或者颜色异常的块状区域的缺陷包括灰带缺陷；并且
    主特征为在缺陷区域内的像素灰度值为零的缺陷包括第一类型黑条缺陷和破损缺陷，且该主特征的子特征为图像二值图中存在黑像素的列中的黑像素个数、或者图像二值图中缺陷区域边界的形状和长度。
  7. 根据权利要求6所述的方法，其中，在缺陷检测响应集中用于检测当前候选缺陷类别的主特征的缺陷检测算法所对应的响应为“有响应”的情况下，将当前候选缺陷类别确定为所述待检测缺陷图像的缺陷类别，或者基于主特征的子特征来确定所述待检测缺陷图像的缺陷类别，包括：
    在确定存在于待检测的缺陷图像中的缺陷是否为破损缺陷时，如果低阈值二值化算法对应的响应为“有响应”时，对所述待检测缺陷图像的图像二值图执行竖直投影算法，并且
    在存在黑像素的列中黑像素个数少于图像列像素的总和的情况下，确定所述待检测图像的缺陷类别是破损缺陷，在存在黑像素的列中黑像素个数等于图像列像素的总和的情况下，确定所述待检测图像的缺陷类别是第一黑条类型。
  8. 根据权利要求6所述的方法，其中，在缺陷检测响应集中用于检测当前候选缺陷类别的主特征的缺陷检测算法所对应的响应为“有响应”的情况下，将当前候选缺陷类别确定为所述待检测缺陷图像的缺陷类别，或者基于主特征的子特征来确定所述待检测缺陷图像的缺陷类别，包括：
    在确定存在于待检测的缺陷图像中的缺陷是否为破损缺陷时，如果低阈值二值化算法对应的响应为“有响应”时，对图像二值图进行边缘检测，并且
    在检测到缺陷区域边界为曲线的情况下，确定所述待检测图像的缺陷类别是破损缺陷，并且在检测到缺陷区域边界为竖直直线并且长度等于待检测缺陷图像的列长度的情况下，确定所述待检测图像的缺陷类别是第一黑条类型。
  9. 根据权利要求6所述的方法，其中，在缺陷检测响应集中用于检测当前候选缺陷类别的主特征的缺陷检测算法所对应的响应为“有响应”的情况下，将当前候选缺陷类别确定为所述待检测缺陷图像的缺陷类别，或者基于主特征的子特征来确定所述待检测缺陷图像的缺陷类别，包括：
    在确定存在于待检测的缺陷图像中的缺陷是否为碎片缺陷时，如果黑白斑检测算法对应的响应为“有响应”时，并且在通过黑白斑检测算法确定 是黑斑的情况下，确定所述待检测图像的缺陷类别是特殊不均，并且
    在通过黑白斑检测算法确定是白斑的情况下，对白斑区域内的灰度值标准差进行分析，并且如果所述灰度值标准差小于第一预设阈值，则确定所述缺陷类别是碎片缺陷，如果所述灰度值标准差大于等于第一预设阈值，则确定所述缺陷类别是白点缺陷。
  10. 根据权利要求6所述的方法，其中，在缺陷检测响应集中用于检测当前候选缺陷类别的主特征的缺陷检测算法所对应的响应为“有响应”的情况下，将当前候选缺陷类别确定为所述待检测缺陷图像的缺陷类别，或者基于主特征的子特征来确定所述待检测缺陷图像的缺陷类别，包括：
    在确定存在于待检测的缺陷图像中的缺陷是否为区域放电缺陷时，如果边缘密度检测算法对应的响应为“有响应”，则确定所述缺陷类别是区域放电缺陷。
  11. 根据权利要求6所述的方法，其中，在缺陷检测响应集中用于检测当前候选缺陷类别的主特征的缺陷检测算法所对应的响应为“有响应”的情况下，将当前候选缺陷类别确定为所述待检测缺陷图像的缺陷类别，或者基于主特征的子特征来确定所述待检测缺陷图像的缺陷类别，包括：
    在确定存在于待检测的缺陷图像中的缺陷是否为线段放电缺陷时，如果霍夫变换直线检测算法对应的响应为“有响应”，对通过霍夫变换直线检测算法检测到的线段进行分析，并且：
    在检测到线段在水平方向时，如果线段水平长度小于第二预设阈值，则确定所述待检测图像的缺陷类别是线放电缺陷，并且如果线段水平长度大于等于第二预设阈值，则确定所述待检测图像的缺陷类别是第二类型黑条缺陷。
  12. 根据权利要求6所述的方法，其中，在缺陷检测响应集中用于检测当前候选缺陷类别的主特征的缺陷检测算法所对应的响应为“有响应”的情况下，将当前候选缺陷类别确定为所述待检测缺陷图像的缺陷类别，或者基于主特征的子特征来确定所述待检测缺陷图像的缺陷类别，包括：
    在确定存在于待检测的缺陷图像中的缺陷是否为灰带缺陷时，如果仅有竖直投影检测算法对应的响应为“有响应”，则确定所述缺陷类别是灰带缺陷。
  13. 一种缺陷图像的缺陷检测装置，包括：
    处理器；和
    存储器，其上存储有计算机可用指令，所述指令在由所述处理器执行时，使得所述处理器执行权利要求1-12中任一项的方法中的步骤。
  14. 一种计算机可读存储介质，其上存储有程序指令，所存储的程序指令可由处理器读取并执行，以使所述处理器执行权利要求1-12中任一项的方法中的步骤。
PCT/CN2020/076958 2020-02-27 2020-02-27 缺陷图像的缺陷检测方法、装置及计算机可读存储介质 WO2021168733A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/076958 WO2021168733A1 (zh) 2020-02-27 2020-02-27 缺陷图像的缺陷检测方法、装置及计算机可读存储介质
CN202080000190.0A CN113646801B (zh) 2020-02-27 2020-02-27 缺陷图像的缺陷检测方法、装置及计算机可读存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/076958 WO2021168733A1 (zh) 2020-02-27 2020-02-27 缺陷图像的缺陷检测方法、装置及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2021168733A1 true WO2021168733A1 (zh) 2021-09-02

Family

ID=77490592

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/076958 WO2021168733A1 (zh) 2020-02-27 2020-02-27 缺陷图像的缺陷检测方法、装置及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN113646801B (zh)
WO (1) WO2021168733A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113873229A (zh) * 2021-09-26 2021-12-31 江西盛泰精密光学有限公司 一种图像坏点检测方法、系统及装置
CN115311280A (zh) * 2022-10-12 2022-11-08 南通迪博西电子有限公司 一种碳化硅材质晶圆的缺陷检测方法
CN115690102A (zh) * 2022-12-29 2023-02-03 摩尔线程智能科技(北京)有限责任公司 缺陷检测方法、装置、电子设备、存储介质和程序产品
CN115797877A (zh) * 2023-02-13 2023-03-14 山东鸿德电力科技有限公司 一种电力传输设备的智能监控方法、系统及介质
CN116048945A (zh) * 2023-03-29 2023-05-02 摩尔线程智能科技(北京)有限责任公司 设备性能检测方法、装置、电子设备及存储介质
CN116503408A (zh) * 2023-06-28 2023-07-28 曲阜远大集团工程有限公司 基于扫描技术的钢结构表面缺陷检测方法
CN117152444A (zh) * 2023-10-30 2023-12-01 山东泰普锂业科技有限公司 一种面向锂电行业的设备数据采集方法及系统
CN117455870A (zh) * 2023-10-30 2024-01-26 太康精密（中山）有限公司 一种连接线和连接器质量视觉检测方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113870754B (zh) * 2021-12-03 2022-03-11 中导光电设备股份有限公司 一种面板检测电子信号缺陷判定的方法和系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104458755A (zh) * 2014-11-26 2015-03-25 吴晓军 一种基于机器视觉的多类型材质表面缺陷检测方法
US20150098655A1 (en) * 2013-10-08 2015-04-09 National Tsing Hua University Method of Defect Image Classification through Integrating Image Analysis and Data Mining
CN106157303A (zh) * 2016-06-24 2016-11-23 浙江工商大学 一种基于机器视觉对表面检测的方法
CN108734696A (zh) * 2017-04-18 2018-11-02 三星显示有限公司 用于白点Mura检测的系统及方法
CN109801286A (zh) * 2019-01-29 2019-05-24 江南大学 一种lcd导光板的表面缺陷检测算法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6747697B1 (en) * 2000-07-12 2004-06-08 Koninklijke Philips Electronics, N.V. Method and apparatus for digital image defect correction and noise filtering
JP2008170256A (ja) * 2007-01-11 2008-07-24 Seiko Epson Corp 欠陥検出方法、欠陥検出プログラムおよび検査装置
CN108230321A (zh) * 2018-01-19 2018-06-29 深圳市亿图视觉自动化技术有限公司 缺陷检测方法及装置
CN108629775B (zh) * 2018-05-14 2021-08-03 华中科技大学 一种热态高速线材表面图像处理方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150098655A1 (en) * 2013-10-08 2015-04-09 National Tsing Hua University Method of Defect Image Classification through Integrating Image Analysis and Data Mining
CN104458755A (zh) * 2014-11-26 2015-03-25 吴晓军 一种基于机器视觉的多类型材质表面缺陷检测方法
CN106157303A (zh) * 2016-06-24 2016-11-23 浙江工商大学 一种基于机器视觉对表面检测的方法
CN108734696A (zh) * 2017-04-18 2018-11-02 三星显示有限公司 用于白点Mura检测的系统及方法
CN109801286A (zh) * 2019-01-29 2019-05-24 江南大学 一种lcd导光板的表面缺陷检测算法

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113873229B (zh) * 2021-09-26 2024-02-27 江西盛泰精密光学有限公司 一种图像坏点检测方法、系统及装置
CN113873229A (zh) * 2021-09-26 2021-12-31 江西盛泰精密光学有限公司 一种图像坏点检测方法、系统及装置
CN115311280A (zh) * 2022-10-12 2022-11-08 南通迪博西电子有限公司 一种碳化硅材质晶圆的缺陷检测方法
CN115690102A (zh) * 2022-12-29 2023-02-03 摩尔线程智能科技(北京)有限责任公司 缺陷检测方法、装置、电子设备、存储介质和程序产品
CN115797877A (zh) * 2023-02-13 2023-03-14 山东鸿德电力科技有限公司 一种电力传输设备的智能监控方法、系统及介质
CN116048945A (zh) * 2023-03-29 2023-05-02 摩尔线程智能科技(北京)有限责任公司 设备性能检测方法、装置、电子设备及存储介质
CN116048945B (zh) * 2023-03-29 2023-06-23 摩尔线程智能科技(北京)有限责任公司 设备性能检测方法、装置、电子设备及存储介质
CN116503408A (zh) * 2023-06-28 2023-07-28 曲阜远大集团工程有限公司 基于扫描技术的钢结构表面缺陷检测方法
CN116503408B (zh) * 2023-06-28 2023-08-25 曲阜远大集团工程有限公司 基于扫描技术的钢结构表面缺陷检测方法
CN117152444A (zh) * 2023-10-30 2023-12-01 山东泰普锂业科技有限公司 一种面向锂电行业的设备数据采集方法及系统
CN117455870A (zh) * 2023-10-30 2024-01-26 太康精密（中山）有限公司 一种连接线和连接器质量视觉检测方法
CN117152444B (zh) * 2023-10-30 2024-01-26 山东泰普锂业科技有限公司 一种面向锂电行业的设备数据采集方法及系统
CN117455870B (zh) * 2023-10-30 2024-04-16 太康精密（中山）有限公司 一种连接线和连接器质量视觉检测方法

Also Published As

Publication number Publication date
CN113646801A (zh) 2021-11-12
CN113646801B (zh) 2024-04-02

Similar Documents

Publication Publication Date Title
WO2021168733A1 (zh) 缺陷图像的缺陷检测方法、装置及计算机可读存储介质
CN111179225B (zh) 一种基于灰度梯度聚类的试纸表面纹理缺陷检测方法
CN108460757B (zh) 一种手机TFT-LCD屏Mura缺陷在线自动检测方法
CN105894036B (zh) 一种应用于手机屏幕缺陷检测的图像特征模板匹配方法
CN107437243B (zh) 基于x光图像的轮胎杂质检测方法及装置
CN108090494B (zh) 基于Gabor滤波器及支持向量机纺织品瑕疵识别方法
Choi et al. Detection of pinholes in steel slabs using Gabor filter combination and morphological features
JP2018522293A (ja) 線形クラスタリングに基づくマルチタイプのｂｇａチップの視覚識別方法
CN103543394A (zh) 一种高压电气设备放电紫外成像量化参数提取方法
Zhang et al. An accurate fuzzy measure-based detection method for various types of defects on strip steel surfaces
CN110648330B (zh) 摄像头玻璃的缺陷检测方法
JP2019144245A (ja) 前処理を含む線欠陥検出システム及び方法
TW202127371A (zh) 基於圖像的瑕疵檢測方法及電腦可讀存儲介質
TW201512649A (zh) 偵測晶片影像瑕疵方法及其系統與電腦程式產品
CN110426395B (zh) 一种太阳能el电池硅片表面检测方法及装置
CN115866502A (zh) 一种麦克风零件表面缺陷在线检测流程
Liu et al. An automatic defects detection scheme for lithium-ion battery electrode surface
Hashmi et al. Computer-vision based visual inspection and crack detection of railroad tracks
CN110516725B (zh) 基于机器视觉的木板条纹间距和颜色的检测方法
Kuo et al. Automated inspection of micro-defect recognition system for color filter
CN117036314A (zh) 一种高密度柔性ic基板氧化区域检测方法
CN111833350A (zh) 机器视觉检测方法与系统
JP2011196982A (ja) 欠陥検出方法、欠陥検出装置、及びプログラム
JP7469740B2 (ja) ベルト検査システムおよびベルト検査プログラム
CN113192061A (zh) Led封装外观检测图像的提取方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20921554

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20921554

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20921554

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 04.04.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20921554

Country of ref document: EP

Kind code of ref document: A1