WO2023226706A9 - 缺陷检测方法及装置 - Google Patents

缺陷检测方法及装置 Download PDF

Info

Publication number
WO2023226706A9
WO2023226706A9 PCT/CN2023/092048 CN2023092048W WO2023226706A9 WO 2023226706 A9 WO2023226706 A9 WO 2023226706A9 CN 2023092048 W CN2023092048 W CN 2023092048W WO 2023226706 A9 WO2023226706 A9 WO 2023226706A9
Authority
WO
WIPO (PCT)
Prior art keywords
unit
tested
units
under test
suspicious
Prior art date
Application number
PCT/CN2023/092048
Other languages
English (en)
French (fr)
Other versions
WO2023226706A1 (zh
Inventor
刘童
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Publication of WO2023226706A1 publication Critical patent/WO2023226706A1/zh
Publication of WO2023226706A9 publication Critical patent/WO2023226706A9/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Definitions

  • the present disclosure relates to the field of computer vision technology, and in particular, to a defect detection method and device.
  • defect detection there are many methods of defect detection.
  • the traditional defect detection method is through artificial naked eye detection.
  • machine learning and computer vision technology can be used for defect detection to improve detection speed and detection. quality.
  • the resolution of the image frame to be detected is high, the calculation amount is large and the calculation time is long, resulting in a slow detection speed, which is difficult to match the speed requirements of industrial production lines. If the resolution of the image frame to be detected is reduced, it will affect the accuracy of the defect detection results and fail to meet the detection requirements.
  • Embodiments of the present disclosure provide a defect detection method and device, which reduce the computational complexity in the defect detection process, reduce costs, increase detection speed, and obtain more accurate defect detection results.
  • a defect detection method includes: obtaining information of multiple units to be tested in an image frame to be detected, and at least one unit template; performing template matching on the multiple units to be tested with the unit template respectively based on the information of the multiple units to be tested, Obtain a suspicious unit, which is a unit under test whose matching degree with the unit template among the plurality of units to be tested is lower than a preset threshold; use a machine learning algorithm to classify the suspicious unit, and obtain the The defect detection type of the suspect unit.
  • obtaining the at least one unit template includes: randomly selecting M first units to be tested and a group of units to be tested from a plurality of units to be tested in the image frame to be detected, each of the The unit group to be tested includes N second units to be tested, and each of the first units to be tested corresponds to one unit group to be tested; each first unit to be tested and the first unit to be tested are respectively calculated.
  • the similarity of the corresponding N second units to be tested is obtained to obtain the N similarities corresponding to each of the first units to be tested; the Nth similarity is calculated based on the N similarities corresponding to each of the first units to be tested.
  • An average similarity value corresponding to a unit to be tested; among the M first units to be tested, the first unit to be tested with the highest average similarity degree is determined as the unit template.
  • the information of the plurality of units under test includes at least one of position information and offset of at least one unit under test.
  • the method further includes: determining whether the suspicious unit is a normal unit or a defective unit according to the defect detection type of the suspicious unit; when the suspicious unit is a defective unit, generating alarm information and outputting the The defect detection type of the suspect unit.
  • a defect detection device including: a processor and a transceiver; the transceiver is configured to obtain information of a plurality of units to be tested in an image frame to be detected, and at least one unit template; said The processor is configured to: perform template matching on the plurality of units to be tested with the unit template respectively according to the information of the plurality of units to be tested, to obtain a suspicious unit, where the suspicious unit is the plurality of units to be tested.
  • the units to be tested whose matching degree with the unit template is lower than the preset threshold; a machine learning algorithm is used to classify the suspicious units to obtain the defect detection type of the suspicious units.
  • the processor is configured to: randomly select M first units to be tested and a group of units to be tested from a plurality of units to be tested in the image frame to be detected, each of the units to be tested being
  • the unit group includes N second units to be tested, and each of the first units to be tested corresponds to one unit group to be tested; the values corresponding to each of the first units to be tested and the first unit to be tested are respectively calculated.
  • N similarities corresponding to each of the first units to be tested are obtained; and the first units to be tested are calculated based on the N similarities corresponding to each of the first units to be tested.
  • the average similarity value corresponding to the unit under test is determined; among the M first units under test, the first unit under test with the highest average similarity degree is determined as the unit template.
  • the information of the plurality of units under test includes at least one of position information and offset of at least one unit under test.
  • the processor is further configured to: determine whether the suspicious unit is a normal unit or a defective unit according to the defect detection type of the suspicious unit; when the suspicious unit is a defective unit, generate alarm information and Outputs the defect detection type of the suspect unit.
  • the defect detection method provided by the embodiments of the present disclosure achieves rapid defect detection by combining traditional computer vision methods with deep learning technology.
  • Template matching is used to screen the units to be tested in the image to obtain suspicious units, and a machine learning algorithm is used to classify the suspicious units to obtain the defect detection type of the suspicious unit. Therefore, the unit to be tested whose rotation or size change cannot be detected during template matching can be detected again, and accurate detection results can be obtained.
  • Using template matching to filter out normal units and only classify suspicious units can reduce the amount of calculation, reduce computational complexity, and improve the speed of defect detection.
  • an electronic device includes: a processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform any of the above.
  • a defect detection method according to an embodiment.
  • a computer-readable storage medium stores one or more programs, and when executed by a processor, the one or more programs implement any of the above embodiments. Described defect detection method.
  • Figure 1 is a flow chart of a defect detection method according to some embodiments.
  • Figure 2 is a schematic diagram of the application of a defect detection method according to some embodiments.
  • Figure 3 is a flow chart of an adaptive acquisition unit template according to some embodiments.
  • Figure 4 is a schematic diagram of the application of another defect detection method according to some embodiments.
  • Figure 5 is an application schematic diagram of yet another defect detection method according to some embodiments.
  • Figure 6 is an application schematic diagram of yet another defect detection method according to some embodiments.
  • Figure 7 is a structural diagram of a defect detection device according to some embodiments.
  • first and second are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as “first” and “second” may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.
  • At least one of A, B and C has the same meaning as “at least one of A, B or C” and includes the following combinations of A, B and C: A only, B only, C only, A and B The combination of A and C, the combination of B and C, and the combination of A, B and C.
  • a and/or B includes the following three combinations: A only, B only, and a combination of A and B.
  • the term “if” is optionally interpreted to mean “when” or “in response to” or “in response to determining” or “in response to detecting,” depending on the context.
  • the phrase “if it is determined" or “if [stated condition or event] is detected” is optionally interpreted to mean “when it is determined" or “in response to the determination" or “on detection of [stated condition or event]” or “in response to detection of [stated condition or event]”.
  • defect detection through template matching.
  • template matching when template matching is used for defect detection, if the unit to be tested rotates or changes in size in the image frame to be detected, this method cannot identify the unit to be tested that rotates or changes in size.
  • Another defect detection method is defect detection through machine learning and computer vision technology. Although this method can identify the unit to be tested that has rotated or changed in size in the image frame to be detected, when the resolution of the image frame to be detected is high, , using this method for defect detection requires a large amount of calculation and takes a long time, resulting in a slow detection speed and difficulty in matching the speed requirements of industrial production lines. If the resolution of the image frame to be detected is reduced, it will affect the accuracy of the defect detection results and fail to meet the detection requirements.
  • some embodiments of the present disclosure provide a defect detection method that uses template matching to filter the units to be tested in the image frame to be detected to obtain suspicious units, and uses a machine learning algorithm to classify the suspicious units to obtain the suspicious units.
  • the defect detection type of the unit In this way, normal units are filtered out through template matching, and only suspicious units are classified using machine learning algorithms, which can not only reduce the amount of calculations, reduce the computational complexity, and improve the speed of defect detection. It can also detect units under test that have rotated or changed in size, improving the accuracy of defect detection.
  • the defect detection method of the present disclosure is suitable for scenarios where a single product has repeating units, the products are neatly arranged, and the image frame resolution to be detected is high, such as display panels and products on assembly lines in industrial production.
  • the following embodiments take an industrially produced beverage bottle as an example for illustrative description.
  • the present disclosure does not limit the application scenarios of the defect detection method.
  • the defect detection method includes steps 101 to 103.
  • Step 101 Obtain information on multiple units to be tested in the image frame to be detected, and at least one unit template.
  • the unit to be tested in the image frame to be detected refers to the smallest repeated object to be tested in the image frame to be detected, as shown in Figure 2.
  • the unit to be tested is Image of the circuit diagram for each pixel circuit.
  • the unit template is an image of a standard normal unit.
  • the unit template can be used as a template for template matching.
  • the information of the plurality of units under test includes at least one of position information and offset of at least one unit under test.
  • the position information of the unit under test can be calibrated using bounding boxes.
  • the embodiment of the present disclosure does not limit the shape of the bounding box, and the bounding box is related to the type of the unit under test.
  • the bounding box can be a circle, a rectangle, an ellipse, a rhombus, a polygon, etc.
  • the following embodiments take the bounding box as a rectangular bounding box as an example for illustrative description.
  • the position information of each unit to be tested may be manually calibrated. For example, a rectangular bounding box is manually used to calibrate the position of each unit to be tested in the image frame to be detected.
  • the position information of the unit under test may be composed of the position information of at least one unit under test and the image frame to be detected.
  • the offset of adjacent cells in is calculated. For example, based on the manually calibrated position and offset of one unit to be tested in the image frame to be detected, the positions of other units to be tested in the image frame to be detected can be determined.
  • the position information and offset of the at least one unit under test can be input in advance by the user.
  • a fixed method may be used to obtain at least one unit template
  • an adaptive method may be used to obtain at least one unit template.
  • the implementation method of obtaining at least one unit template by using the fixed method and the implementation method of obtaining at least one unit template by using the adaptive method are respectively described in detail below.
  • using a fixed method to obtain at least one unit template includes: randomly selecting an image frame to be detected, sequentially matching each unit to be tested in the image frame to be detected with a normal unit, and selecting the unit to be tested that has the highest matching degree with the normal unit.
  • the test unit serves as a unit template.
  • This unit template can be used as a unit template for template matching in subsequent defect detection.
  • Using the fixation method to obtain at least one unit template may also include: matching multiple normal units in a normal image frame without defects, and selecting the unit with the highest matching degree as the unit template.
  • the unit template when using the adaptive method to obtain at least one unit template, can be selected from each image frame to be detected. For different image frames to be detected, the following steps 301 to 304 can be used to obtain each unit template. The unit template corresponding to the image frame to be detected.
  • the method of obtaining at least one unit template using an adaptive method may include steps 301 to 304.
  • Step 301 Randomly select M first units to be tested from multiple units to be tested in the image frame to be detected.
  • M is an integer greater than or equal to 2.
  • M can be set to a larger value.
  • M is equal to the number of units to be tested in the image frame to be detected, When counting, it is called the exhaustive method.
  • M units to be tested are randomly selected from the units to be tested as the first units to be tested. This disclosure does not limit the numerical value of M.
  • the image frame to be detected includes 16 units to be tested. Taking M as 6 as an example, 6 units to be tested can be randomly selected from the 16 units to be tested.
  • the first unit under test for example, the first unit under test may include the units under test M1 to M6 shown in FIG. 2 .
  • Step 302 Obtain the unit group under test corresponding to each first unit under test.
  • the unit group under test corresponding to each first unit under test includes N second units under test.
  • obtaining the unit group under test corresponding to each first unit under test may include: randomly selecting N units under test from other units under test except the first unit under test in the image frame to be detected.
  • the unit under test is a group of units under test corresponding to the first unit under test, and the N units under test may also be called N second units under test.
  • the numerical values of M and N may be the same or different. This disclosure does not limit the numerical values of M and N. M and N can be preset by the user and entered into the defect detection system.
  • the first unit to be tested M1 when the first unit to be tested is the first unit to be tested M1 in Figure 4, the first unit to be tested M1 can be removed from the image frame to be detected.
  • 7 units under test (second units under test) are randomly selected as the unit under test group corresponding to the first unit under test M1.
  • the 7 second units under test can be as shown in Figure 4
  • the second unit under test N101 to the second unit under test N107 are shown. That is, the unit group under test corresponding to the first unit under test M1 includes the second unit under test N101 to the second unit under test N107.
  • the method of obtaining the unit group to be tested corresponding to the first unit to be tested M2 to the first unit to be tested M6 is similar to the method of obtaining the unit group to be tested corresponding to the first unit to be tested M1, and will not be described again here. .
  • the unit group under test corresponding to each unit under test among the first unit under test M2 to the first unit under test M6 may also be obtained.
  • the same unit under test in the image frame to be detected may be the first unit under test or the second unit under test.
  • the unit under test M2 in the image frame to be detected can be used as the first unit under test.
  • the unit group to be tested corresponding to the first unit to be tested M1 includes 7 second units to be tested. , respectively the second unit to be tested N101 to the second unit to be tested N107 in FIG. 4 .
  • the first unit under test M2 in FIG. 2 and the second unit under test N102 in FIG. 4 are the same unit under test.
  • the first unit under test M3 in FIG. 2 and the second unit under test N104 in FIG. 4 are the same unit under test. That is to say, the same unit under test can be selected as the first unit under test or as the second unit under test.
  • the N second units under test included in the unit under test groups corresponding to different first units under test may be at least partially the same, or may be completely different.
  • groups of units under test corresponding to different first units under test may include the same second unit under test.
  • Step 303 Calculate the similarities between each first unit under test and the N second units under test corresponding to the first unit under test, to obtain N similarities corresponding to each first unit under test.
  • the first unit to be tested M1 is matched with the second units to be tested N101 to N107 included in its corresponding unit group to be tested. , seven similarity values corresponding to the first unit under test M1 are obtained. In the same way, the seven similarity values corresponding to each of the first units to be tested M2 to M6 can also be calculated. That is, 7 similarity values corresponding to each of the first units to be tested M1 to M6 can be obtained.
  • Step 304 Determine the first unit to be tested with the highest average similarity among the M first units to be tested as the unit template.
  • each of the M first units to be tested corresponds to N similarity values.
  • Step 304 may include: calculating the average of the N similarity values corresponding to each first unit to be tested, obtaining the average similarity value corresponding to each first unit to be tested, and adding the similarity values corresponding to the M first units to be tested. Among the degree averages, the first unit to be tested with the highest similarity average is determined as the unit template.
  • the average of the seven similarity values corresponding to the first unit to be tested M1 is calculated to obtain the average similarity value corresponding to the first unit to be tested M1.
  • the average similarity values corresponding to the first unit to be tested M3 to the first unit to be tested M6 can be obtained.
  • the first unit to be tested corresponding to the similarity average value with the highest value is determined as the unit template.
  • the first unit to be tested and the group of units to be tested corresponding to the first unit to be tested are randomly selected from the image frame to be detected, and the first unit to be tested and the group of units to be tested corresponding to the unit to be tested are Each second unit to be tested is matched to obtain multiple similarities, and the average similarity is calculated, and the first unit to be tested with the highest average similarity among the first units to be tested is determined as the unit template.
  • the most appropriate unit can be selected as the unit template in the image frame to be detected, so as to solve the problem of inconsistent illumination received by the product to be detected due to various factors in daily life, resulting in uneven brightness of the image of the unit to be tested, thereby affecting the overall brightness of the unit to be tested.
  • the problem of choosing template quality is the problem of choosing template quality.
  • the unit template can be determined based on factors such as brightness. For example, when the brightness of most of the units to be tested in the image frame to be detected is relatively high, the brightness of the unit template determined using the above adaptive method is relatively high. For another example, when the brightness of most of the units to be tested in the image frame to be detected is low, the brightness of the unit template determined using the above adaptive method is low.
  • step 101 may also include obtaining an image frame to be detected and performing image preprocessing on the image frame to be detected.
  • Step 102 According to the information of the multiple units to be tested, perform template matching on the multiple units to be tested with the unit template respectively to obtain the suspicious units.
  • the above-mentioned suspicious unit is a unit under test whose matching degree with the unit template is lower than a preset threshold among multiple units under test. It can be understood that by template matching multiple units to be tested with the unit template, the normal units in the image frame to be detected can be filtered, and the units to be tested that cannot be determined to be normal units in the template matching can be obtained. These units cannot be determined to be normal units.
  • the unit under test may be called a suspect unit. Since suspicious units also have a probability of being normal units, that is, suspicious units may include normal units. In other words, the suspicious units obtained by template matching may be normal units or defective units.
  • the position information of multiple units under test can be calibrated using a rectangular bounding box, and the unit under test calibrated by each rectangular bounding box is matched with the unit template to obtain the matching degree of each unit under test and the unit template.
  • a high matching degree indicates a high similarity between the unit to be tested and the unit template, and the probability that the unit to be tested is a normal unit is high.
  • a low matching degree means that the similarity between the unit to be tested and the unit template is low, and the probability that the unit to be tested is a normal unit is small.
  • the matching degree between the unit under test and the unit template is higher than or equal to the preset threshold
  • the unit under test is determined to be a normal unit.
  • the matching degree between the unit under test and the unit template is lower than the preset threshold, the unit under test is determined to be a suspicious unit.
  • the above-mentioned suspicious unit may further determine whether the suspicious unit is a defective unit through step 103.
  • the above-mentioned preset threshold can be manually input into the defect detection system in advance.
  • the preset threshold as 90% as an example, when the matching degree between the unit to be tested and the template is greater than or equal to 90%, the unit to be tested is determined to be a normal unit. .
  • the matching degree between the unit under test and the template is lower than 90%, the unit under test is determined to be a suspicious unit.
  • the larger the preset threshold the more accurate the template matching screening results will be, and the higher the defect detection accuracy will be. This disclosure does not limit the size of the preset threshold.
  • Step 103 Use a machine learning algorithm to classify the suspicious units and obtain the defect detection type of the suspicious units.
  • defect detection types include normal types and defect types. Since a suspicious unit refers to a unit under test that has a low matching degree with the unit template in template matching, template matching has its own limitations and can only match the image of the unit under test within a fixed shape area, and cannot detect rotation and size changes. unit, so a machine learning algorithm can be used to further classify suspicious units and obtain the defect detection type of the suspicious unit.
  • the defect type may be pre-entered by the user. In several image frames to be detected, it is manually judged whether there are defective products, and if there are defective products, manual annotation is performed. The specific steps of labeling are: use a rectangular bounding box to encircle the defective unit to be tested, and use numbers to number different defect types in sequence. Enter the marked defective product image and the defect type corresponding to the number into the defect detection system. Mark at least one defective product sample for each defect type as the basis for classification.
  • the defect types may include: rotation, size change, foreign matter, leakage, etc.
  • a machine learning algorithm is used to classify suspicious units in the image frame to be detected, and it is obtained that the image frame to be detected includes 4 defect types.
  • the corresponding relationship between the 4 defect types and the numerical numbers is: 1 means rotation, 2 indicates size change, 3 indicates foreign matter, and 4 indicates solid leakage.
  • the machine learning algorithm includes: a convolutional neural network algorithm or a traditional machine learning algorithm.
  • the model of the convolutional neural network algorithm includes Densenet, ResNet, Shufflenet or MobileNet.
  • the traditional machine learning algorithm includes a support vector machine (SVM) algorithm or decision-making algorithm. tree algorithm. Choose an appropriate convolutional neural network algorithm model based on the complexity of the defect. If the complexity of the defect is high, choose Densenet or ResNet. If the complexity of the defect is low, choose Shufflenet or MobileNet.
  • SVM support vector machine
  • Figure 6 is an application schematic diagram of the defect detection method provided by some implementations of the present disclosure.
  • the input to be detected is The detected image frame is divided into units to obtain multiple units to be tested in the image frame to be detected.
  • the adaptive method or the fixed method is used to obtain the unit template, and each unit to be tested in the image frame to be detected is matched with the unit template in turn to obtain the matching degree of each unit to be tested and the unit template.
  • the unit under test is divided into normal units and suspicious units.
  • normal units obtained by template matching are filtered out to obtain suspicious units.
  • the convolutional neural network is used to classify the suspicious units and obtain the defect detection type of the suspicious units.
  • the above-mentioned step 103 may also include: determining whether the suspicious unit is a normal unit or a defective unit according to the defect detection type of the suspicious unit; and when the suspicious unit is a defective unit, generating alarm information and outputting the defect detection type of the suspicious unit. . It can be understood that if the defect detection type of the suspicious unit is any defect type and the suspicious unit is determined to be a defective unit, alarm information can be generated and the defect type of the suspicious unit can be output, so that relevant personnel can promptly discover defective products and handle them.
  • the defect detection method provided by the embodiment of the present disclosure uses template matching to filter the units to be tested in the image frame to be detected to obtain suspicious units, and uses a machine learning algorithm to classify the suspicious units to obtain the defect detection type of the suspicious unit.
  • the normal units in the image frame to be detected can be filtered out through template matching, and only the suspicious units in the image frame to be detected are classified using machine learning algorithms. This can not only reduce the amount of calculation, reduce the computational complexity, but also improve the speed of defect detection. . It can also detect units under test that have rotated or changed in size, improving the accuracy of defect detection.
  • the defect detection device 700 includes a processor 701 and a transceiver 702 .
  • the transceiver 702 is configured to obtain information of a plurality of units to be tested in the image frame to be detected, and at least one unit template.
  • the information of the plurality of units to be tested includes at least one of position information and offset of at least one unit to be tested.
  • the processor 701 is configured to: perform template matching on the multiple units to be tested with the unit template respectively according to the information of the multiple units to be tested, to obtain a suspicious unit, where the suspicious unit is one of the multiple units to be tested that has a low matching degree with the unit template.
  • the unit to be tested is based on the preset threshold; a machine learning algorithm is used to classify the suspicious units and obtain the defect detection type of the suspicious units.
  • the machine learning algorithm includes: a convolutional neural network algorithm or a traditional machine learning algorithm.
  • the model of the convolutional neural network algorithm includes Densenet, ResNet, Shufflenet or MobileNet.
  • the traditional machine learning algorithm includes a support vector machine (SVM) algorithm or decision-making algorithm. tree algorithm.
  • SVM support vector machine
  • the processor 701 is configured to: randomly select M first units to be tested and a group of units to be tested from a plurality of units to be tested in the image frame to be detected, each group of units to be tested including N second units to be tested, Each first unit to be tested corresponds to a unit group to be tested; the similarity between each first unit to be tested and the N second units to be tested corresponding to the first unit to be tested is calculated to obtain each first unit to be tested.
  • N similarities corresponding to the unit under test calculate the average similarity corresponding to the first unit under test based on the N similarities corresponding to each first unit under test; among the M first units under test, the average similarity The highest first unit under test is determined as the unit template.
  • the processor 701 is also configured to: determine whether the suspicious unit is a normal unit or a defective unit according to the defect detection type of the suspicious unit; when the suspicious unit is a defective unit, generate an alarm information and output the defect detection type of the suspicious unit.
  • Transceiver 702 may be a communication interface, such as an input-output interface.
  • the transceiver 702 is a communication interface.
  • the transceiver 702 may also be a radio frequency unit.
  • the transceiver 702 may be a radio frequency unit connected to an antenna.
  • the above-mentioned defect detection device 700 may also include a memory 703, which is used to store program codes and data corresponding to the defect detection device 700 executing any of the defect detection methods provided above.
  • the memory 703 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (random access memory, RAM), etc.
  • Some embodiments of the present disclosure provide a non-transitory computer-readable storage medium (eg, a non-transitory computer-readable storage medium) having computer program instructions stored therein, and the computer program instructions are stored in a computer (e.g., a non-transitory computer-readable storage medium).
  • a computer e.g., a non-transitory computer-readable storage medium
  • the computer when running on a defect detection device), the computer is caused to execute the defect detection method as described in any of the above embodiments.
  • non-transitory computer-readable storage media may include, but are not limited to: magnetic storage devices (such as hard disks, floppy disks or tapes, etc.), optical disks (such as CD (Compact Disk), DVD (Digital Disk), etc. Versatile Disk, Digital Versatile Disk, etc.), smart cards and flash memory devices (e.g., EPROM (Erasable Programmable Read-Only Memory, Erasable Programmable Read-Only Memory), cards, sticks or key drives, etc.).
  • the various computer-readable storage media described in this disclosure may represent one or more devices and/or other machine-readable storage media for storing information.
  • the term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and/or carrying instructions and/or data.
  • Some embodiments of the present disclosure also provide a computer program product, for example, the computer program product is stored on a non-transitory computer-readable storage medium.
  • the computer program product includes computer program instructions.
  • the computer program instructions When the computer program instructions are executed on a computer (eg, a defect detection device), the computer program instructions cause the computer to perform the defect detection method as described in the above embodiment.
  • Some embodiments of the present disclosure also provide a computer program.
  • the computer program When the computer program is executed on the computer, the computer program causes the computer to perform the defect detection method as described in the above embodiment.

Abstract

一种缺陷检测方法及装置，涉及计算机视觉领域，能够降低缺陷检测过程中的计算复杂程度、降低成本、提升检测速度，得到更准确的缺陷检测结果。该缺陷检测方法包括：首先，获取待检测图像帧中的多个待测单元的信息，以及至少一个单元模板；其次，根据多个待测单元的信息，将多个待测单元分别与单元模板进行模板匹配，得到可疑单元，该可疑单元为多个待测单元中与单元模板的匹配度低于预设阈值的待测单元；最后，采用机器学习算法对可疑单元进行分类，得到该可疑单元的缺陷检测类型。

Description

缺陷检测方法及装置
本申请要求于2022年5月25日提交的、申请号为202210578969.1的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及计算机视觉技术领域，尤其涉及一种缺陷检测方法及装置。
背景技术
目前，缺陷检测的方法有很多，传统的缺陷检测方法是通过人工肉眼检测，随着计算机视觉领域的应用越来越广泛，可以利用机器学习和计算机视觉技术进行缺陷检测，以提升检测速度和检测质量。但是，当待检测图像帧的分辨率较高时，计算量较大，计算时间较长，导致检测速度较慢，难以匹配工业生产线的速度要求。如果降低待检测图像帧的分辨率，会对缺陷检测结果的准确性造成影响，无法达到检测要求。
发明内容
本公开实施例提供一种缺陷检测方法及装置，降低缺陷检测过程中的计算复杂程度，降低成本，提升检测速度，得到更准确的缺陷检测结果。
为达到上述目的，本公开实施例采用如下技术方案：
一方面，提供一种缺陷检测方法。该方法包括：获取待检测图像帧中的多个待测单元的信息，以及至少一个单元模板；根据多个待测单元的信息，将多个待测单元分别与所述单元模板进行模板匹配，得到可疑单元，所述可疑单元为所述多个待测单元中与所述单元模板的匹配度低于预设阈值的待测单元；采用机器学习算法对所述可疑单元进行分类，得到所述可疑单元的缺陷检测类型。
在一些实施例中，获取所述至少一个单元模板，包括：在所述待检测图像帧的多个待测单元中随机选取M个第一待测单元，以及待测单元组，每个所述待测单元组包括N个第二待测单元，每个所述第一待测单元中对应一个所述待测单元组；分别计算每个所述第一待测单元与该第一待测单元对应的N个第二待测单元的相似度，得到每个所述第一待测单元对应的N个相似度；根据每个所述第一待测单元对应的N个相似度计算所述第一待测单元对应的相似度平均值；将所述M个第一待测单元中，相似度平均值最高的所述第一待测单元确定为所述单元模板。
在一些实施例中，所述多个待测单元的信息包括至少一个所述待测单元的位置信息和偏移量中的至少一种。
在一些实施例中，所述方法还包括：根据所述可疑单元的缺陷检测类型确定所述可疑单元为正常单元或缺陷单元；在所述可疑单元为缺陷单元时，产生告警信息并输出所述可疑单元的缺陷检测类型。
另一方面，提供一种缺陷检测装置，包括：处理器和收发器；所述收发器被配置为，获取待检测图像帧中的多个待测单元的信息，以及至少一个单元模板；所述处理器被配置为：根据所述多个待测单元的信息，将所述多个待测单元分别与所述单元模板进行模板匹配，得到可疑单元，所述可疑单元为所述多个待测单元中与所述单元模板的匹配度低于预设阈值的待测单元；采用机器学习算法对所述可疑单元进行分类，得到所述可疑单元的缺陷检测类型。
在一些实施例中，所述处理器被配置为：在所述待检测图像帧的多个待测单元中随机选取M个第一待测单元，以及待测单元组，每个所述待测单元组包括N个第二待测单元，每个所述第一待测单元中对应一个所述待测单元组；分别计算每个所述第一待测单元与该第一待测单元对应的N个第二待测单元的相似度，得到每个所述第一待测单元对应的N个相似度；根据每个所述第一待测单元对应的N个相似度计算所述第一待测单元对应的相似度平均值；将所述M个第一待测单元中，相似度平均值最高的所述第一待测单元确定为所述单元模板。
在一些实施例中，所述多个待测单元的信息包括至少一个所述待测单元的位置信息和偏移量中的至少一种。
在一些实施例中，所述处理器还被配置为：根据所述可疑单元的缺陷检测类型确定所述可疑单元为正常单元或缺陷单元；在所述可疑单元为缺陷单元时，产生告警信息并输出所述可疑单元的缺陷检测类型。
本公开实施例提供的缺陷检测方法，通过结合传统计算机视觉方法与深度学习技术结合实现快速缺陷检测。采用模板匹配对图像中的待测单元进行筛选，得到可疑单元，对可疑单元采用机器学习算法进行分类，得到该可疑单元的缺陷检测类型。从而能够对模板匹配中无法检测的旋转或大小变化的待测单元再次检测，得到准确的检测结果。使用模板匹配过滤掉正常单元，仅对可疑单元进行分类，能够减少计算量降低计算复杂度，提升缺陷检测速度。
又一方面，提供一种电子设备，其中，该电子设备包括：处理器；以及被安排成存储计算机可执行指令的存储器，所述可执行指令在被执行时使所述处理器执行如上述任一实施例所述的缺陷检测方法。
又一方面，提供一种计算机可读存储介质，其中，所述计算机可读存储介质存储一个或多个程序，所述一个或多个程序当被处理器执行时，实现如上述任一实施例所述的缺陷检测方法。
附图说明
为了更清楚地说明本公开中的技术方案，下面将对本公开一些实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例的附图，对于本领域普通技术人员来讲，还可以根据这些附图获得其他的附图。此外，以下描述中的附图可以视作示意图，并非对本公开实施例所涉及的产品的实际尺寸、方法的实际流程、信号的实际时序等的限制。
图1为根据一些实施例的一种缺陷检测方法的流程图；
图2为根据一些实施例的一种缺陷检测方法的应用示意图；
图3为根据一些实施例的一种自适应获取单元模板的流程图；
图4为根据一些实施例的另一种缺陷检测方法的应用示意图；
图5为根据一些实施例的又一种缺陷检测方法的应用示意图；
图6为根据一些实施例的又一种缺陷检测方法的应用示意图；
图7为根据一些实施例的一种缺陷检测装置的结构图。
具体实施方式
下面将结合附图，对本公开一些实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。基于本公开所提供的实施例，本领域普通技术人员所获得的所有其他实施例，都属于本公开保护的范围。
除非上下文另有要求，否则，在整个说明书和权利要求书中，术语“包括(comprise)”及其其他形式例如第三人称单数形式“包括(comprises)”和现在分词形式“包括(comprising)”被解释为开放、包含的意思，即为“包含，但不限于”。在说明书的描述中，术语“一个实施例(one embodiment)”、“一些实施例(some embodiments)”、“示例性实施例(exemplary embodiments)”、“示例(example)”、“特定示例(specific example)”或“一些示例(some examples)”等旨在表明与该实施例或示例相关的特定特征、结构、材料或特性包括在本公开的至少一个实施例或示例中。上述术语的示意性表示不一定是指同一实施例或示例。此外，所述的特定特征、结构、材料或特点可以以任何适当方式包括在任何一个或多个实施例或示例中。
以下，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本公开实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。
在描述一些实施例时，可能使用了“耦接”和“连接”及其衍伸的表达。例如，描述一些实施例时可能使用了术语“连接”以表明两个或两个以上部件彼此间有直接物理接触或电接触。又如，描述一些实施例时可能使用了术语“耦接”以表明两个或两个以上部件有直接物理接触或电接触。然而，术语“耦接”或“通信耦合(communicatively coupled)”也可能指两个或两个以上部件彼此间并无直接接触，但仍彼此协作或相互作用。这里所公开的实施例并不必然限制于本文内容。
“A、B和C中的至少一个”与“A、B或C中的至少一个”具有相同含义，均包括以下A、B和C的组合：仅A，仅B，仅C，A和B的组合，A和C的组合，B和C的组合，及A、B和C的组合。
“A和/或B”，包括以下三种组合：仅A，仅B，及A和B的组合。
如本文中所使用，根据上下文，术语“如果”任选地被解释为意思是“当……时”或“在……时”或“响应于确定”或“响应于检测到”。类似地，根据上下文，短语“如果确定……”或“如果检测到[所陈述的条件或事件]”任选地被解释为是指“在确定……时”或“响应于确定……”或“在检测到[所陈述的条件或事件]时”或“响应于检测到[所陈述的条件或事件]”。
本文中“适用于”或“被配置为”的使用意味着开放和包容性的语言，其不排除适用于或被配置为执行额外任务或步骤的设备。
另外，“基于”的使用意味着开放和包容性，因为“基于”一个或多个所述条件或值的过程、步骤、计算或其他动作在实践中可以基于额外条件或超出所述的值。
如本文所使用的那样，“约”、“大致”或“近似”包括所阐述的值以及处于特定值的可接受偏差范围内的平均值，其中所述可接受偏差范围如由本领域普通技术人员考虑到正在讨论的测量以及与特定量的测量相关的误差(即，测量系统的局限性)所确定。
在整个说明书中，所提到的“一个实施例”意味着所描述的与该实施例相关的特定特征、结构或特性被包括在至少一个实施例中。因此，在整个说明书中，在各个地方出现的短语“在一个实施例中”不一定都指同一个实施例。此外，这些特定特征、结构或特性可以以任意合适的方式组合在一个或多个实施例中。
工业缺陷检测的背景相对固定，对精度要求更高，样本量更少。通常，有两种缺陷检测方法，一种缺陷检测方法是通过模板匹配进行缺陷检测。但是，采用模板匹配进行缺陷检测时，如果待检测图像帧中的待测单元发生旋转或大小变化时，该方法无法识别发生旋转或大小变化的待测单元。另一种缺陷检测方法是通过机器学习和计算机视觉技术进行缺陷检测，该方法虽然可以识别待检测图像帧中发生旋转或大小变化的待测单元，但是当待检测图像帧的分辨率较高时，采用该方法进行缺陷检测的计算量较大，计算时间较长，导致检测速度较慢，难以匹配工业生产线的速度要求。如果降低待检测图像帧的分辨率，会对缺陷检测结果的准确性造成影响，无法达到检测要求。
为此，本公开的一些实施例提供一种缺陷检测方法，通过采用模板匹配对待检测图像帧中的待测单元进行筛选，得到可疑单元，并对可疑单元采用机器学习算法进行分类，得到该可疑单元的缺陷检测类型。如此一来，通过模板匹配过滤掉正常单元，仅对可疑单元采用机器学习算法进行分类，不仅能够减少计算量，降低计算复杂度，提升缺陷检测速度。而且能够检测出发生旋转或大小变化的待测单元，提升缺陷检测的准确性。
在一些实施例中，本公开的缺陷检测方法适用于产品单一具有重复单元、产品排列整齐且待检测图像帧分辨率高的场景，例如显示面板以及工业生产中流水线上的产品。下述实施例以工业生产的饮料瓶为例进行示例性说明，本公开对缺陷检测方法的应用场景并不限定。
本公开的一些实施例提供一种缺陷检测方法，如图1所示，该缺陷检测方法包括步骤101-步骤103。
步骤101、获取待检测图像帧中的多个待测单元的信息，以及至少一个单元模板。
待检测图像帧中的待测单元是指待检测图像帧中最小的重复待测对象，如图2所示，以待检测图像帧为包括多个像素电路的电路图的图像为例，待测单元为每个像素电路的电路图的图像。单元模板是标准的正常单元的图像，单元模板可以作为模板，供模板匹配使用。
多个待测单元的信息包括至少一个待测单元的位置信息和偏移量中的至少一种。待测单元的位置信息可以采用包围框标定。本公开实施例对于包围框的形状并不限定，该包围框与待测单元的类型有关。例如，包围框可以为圆形、矩形、椭圆形、菱形，多边形等形状，下述实施例以包围框为矩形包围框为例进行示例性说明。
在一些实施例中，当多个待测单元的信息包括每个待测单元的位置信息时，该每个待测单元的位置信息可以由人工标定。例如，人工采用矩形包围框标定待检测图像帧中的每个待测单元的位置。
在一些实施例中，当多个待测单元的信息包括至少一个待测单元的位置信息和偏移量时，待测单元的位置信息可以由至少一个待测单元的位置信息和待检测图像帧中相邻单元的偏移量计算得到。例如，根据人工标定的待检测图像帧中的一个待测单元的位置和偏移量，可以确定待检测图像帧中其他待测单元的位置。上述至少一个待测单元的位置信息和偏移量可以由用户预先输入。
在一些实施例中，可以采用固定法获取至少一个单元模板，也可以采用自适应法获取至少一个单元模板。下面分别对采用固定法获取至少一个单元模板的实现方式和采用自适应法获取至少一个单元模板的实现方式进行详细说明。
示例性的，采用固定法获取至少一个单元模板包括：随机选择待检测图像帧，依次将该待检测图像帧中的每个待测单元与正常单元进行匹配，选取与正常单元匹配度最高的待测单元作为单元模板。该单元模板可以作为后续缺陷检测中模板匹配的单元模板。采用固定法获取至少一个单元模板时也可以包括：对不存在缺陷的正常图像帧中的多个正常单元互相进行匹配，并选取配度最高的单元作为单元模板。
在一些实施例中，采用自适应法获取至少一个单元模板时，可以从每个待检测图像帧中选取单元模板，对于不同的待检测图像帧，均可以采用下述步骤301-步骤304获取每个待检测图像帧对应的单元模板。在一些实施例中，如图3所示，采用自适应法获取至少一个单元模板的方法可以包括步骤301-步骤304。
步骤301、在待检测图像帧的多个待测单元中随机选取M个第一待测单元。
在一些实施例中，M为大于等于2的整数，为了获取最标准的单元模板，提升缺陷检测质量，可以将M设置为较大的数值，当M等于待检测图像帧中待测单元的个数时，称为穷尽法。为了保证检测质量的同时减小计算量，在待测单元中随机抽取M个待测单元作为第一待测单元。本公开对M的数值并不限定。
示例性的，如图2所示的待检测图像帧，该待检测图像帧中包括16个待测单元，以M为6为例，可以在16个待测单元中随机选取6个待测单元作为第一待测单元，比如，第一待测单元可以包括图2所示的待测单元M1至待测单元M6。
步骤302、获取每个第一待测单元对应的待测单元组。
每个第一待测单元对应的待测单元组包括N个第二待测单元。在一些实施例中，获取每个第一待测单元对应的待测单元组，可以包括：在待检测图像帧中除该第一待测单元以外的其他待测单元中，随机选取N个待测单元作为该第一待测单元对应的待测单元组，该N个待测单元也可以称为N个第二待测单元。
在一些实施例中，M与N的数值可以相同，也可以不同，本公开对于M与N的数值大小并不限定。M和N可以由用户预先设置并输入缺陷检测系统中。
示例性的，以N＝7为例，如图4所示，当第一待测单元为图4中的第一待测单元M1时，可以在待检测图像帧中除第一待测单元M1以外的其他15个待测单元中，随机选取7个待测单元(第二待测单元)作为第一待测单元M1对应的待测单元组，该7个第二待测单元可以为图4所示的第二待测单元N101至第二待测单元N107。即，第一待测单元M1对应的待测单元组包括第二待测单元N101至第二待测单元N107。可以理解的，获取第一待测单元M2至第一待测单元M6对应的待测单元组的方法与获取第一待测单元M1对应的待测对单元组的方法类似，在此不再赘述。同理，也可以获取第一待测单元M2至第一待测单元M6中每个待测单元对应的待测单元组。
需要说明的是，待检测图像帧中的同一个待测单元可以是第一待测单元，也可以是第二待测单元。例如，如图2所示，待检测图像帧中的待测单元M2可以作为第一待测单元。结合图2，如图4所示，当第一待测单元为图2中的第一待测单元M1时，该第一待测单元M1对应的待测单元组包括7个第二待测单元，分别为图4中的第二待测单元N101至第二待测单元N107。图2中的第一待测单元M2与图4中的第二待测单元N102为同一个待测单元。图2中的第一待测单元M3与图4中的第二待测单元N104为同一个待测单元。也就是说，同一个待测单元可以被选取作为第一待测单元，也可以被选取作为第二待测单元。
在一些实施例中，不同第一待测单元对应的待测单元组分别包括的N个第二待测单元可以至少部分相同，也可以完全不同。例如，不同第一待测单元对应的待测单元组可以包括同一个第二待测单元。
步骤303、分别计算每个第一待测单元与该第一待测单元对应的N个第二待测单元的相似度，得到每个第一待测单元对应的N个相似度。
示例性的，以计算第一待测单元M1的相似度为例，分别将第一待测单元M1与其对应的待测单元组包括的第二待测单元N101至第二待测单元N107进行匹配，得到第一待测单元M1对应的7个相似度数值。同理，也可以计算第一待测单元M2至第一待测单元M6中每个待测单元对应的7个相似度数值。即，可以得到第一待测单元M1至第一待测单元M6中每个待测单元对应的7个相似度数值。
步骤304、将M个第一待测单元中，相似度平均值最高的第一待测单元确定为单元模板。
在一些实施例中，M个第一待测单元中每个待测单元对应N个相似度数值。步骤304可以包括：计算每个第一待测单元对应的N个相似度数值的平均值，得到每个第一待测单元对应的相似度平均值，将M个第一待测单元对应的相似度平均值中，相似度平均值最高的第一待测单元确定为单元模板。
示例性的，结合图2，如图4所示，首先，计算第一待测单元M1对应的7个相似度数值的平均值，得到第一待测单元M1对应的相似度平均值。计算第一待测单元M2对应的7个相似度数值的平均值，得到第一待测单元M2对应的相似度平均值。依次类推，可以得到第一待测单元M3至第一待测单元M6分别对应的相似度平均值。然后，将第一待测单元M1至第一待测单元M6对应的相似度平均值中，数值最高的相似度平均值对应的第一待测单元确定为单元模板。
可以理解的，本公开实施例通过在待检测图像帧中随机选取第一待测单元和该第一待测单元对应的待测单元组，对第一待测单元和其对应的待测单元组中的每个第二待测单元进行匹配得到多个相似度，并计算相似度平均值，将第一待测单元中相似度平均值最高的第一待测单元确定为单元模板。通过自适应法可以在待检测图像帧中选择最合适的单元作为单元模板，解决日常生活中由于多种因素引起的待检测产品接收的光照度不一致，使得待测单元图像亮度不均匀，进而影响所选模板质量的问题。
例如，当待检测图像帧的亮度较高时，如果采用亮度较低的待测单元作为单元模板与亮度较高的待检测图像帧中的待测单元进行匹配，由于亮度不同，可能会导致待检测图像帧中亮度较高的正常单元与亮度较低的单元模板的匹配度较低，认为该亮度较高的正常单元为可疑单元，影响缺陷检测的准确性。而采用上述自适应法在待检测图像帧中确定单元模板时，可以结合亮度等因素确定单元模板。比如，当待检测图像帧中大部分待测单元的亮度较高时，采用上述自适应法确定的单元模板的亮度较高。再比如，当待检测图像帧中大部分待测单元的亮度较低时，采用上述自适应法确定的单元模板的亮度较低。
在一些实施例中，步骤101之前还可以包括获取待检测图像帧，并对待检测图像帧进行图像预处理。
步骤102、根据多个待测单元的信息，将多个待测单元分别与单元模板进行模板匹配，得到可疑单元。
上述可疑单元为多个待测单元中与单元模板的匹配度低于预设阈值的待测单元。可以理解的，通过将多个待测单元与单元模板进行模板匹配，可以过滤待检测图像帧中的正常单元，得到模板匹配中无法确定为正常单元的待测单元，这些无法确定为正常单元的待测单元可以称为可疑单元。由于可疑单元也有概率为正常单元，即可疑单元可以包括正常单元。也就是说，模板匹配得到的可疑单元有可能是正常单元，也有可能是缺陷单元。
在一些实施例中，多个待测单元的位置信息可以采用矩形包围框标定，将每个矩形包围框标定的待测单元与单元模板进行匹配，得到每个待测单元与单元模板的匹配度(也称为相似度)，匹配度高表示待测单元与单元模板相似度高，该待测单元为正常单元的概率值大。匹配度低表示待测单元与单元模板相似度低，该待测单元为正常单元的概率值小。当待测单元与单元模板的匹配度高于或等于预设阈值，确定待测单元为正常单元。当待测单元与单元模板的匹配度低于预设阈值，确定待测单元为可疑单元。上述可疑单元可以进一步通过步骤103确定该可疑单元是否为缺陷单元。
示例性的，上述预设阈值可以由人工预先输入缺陷检测系统，以预设阈值为90％为例，当待测单元与模板的匹配度大于或等于90％，确定该待测单元为正常单元。当待测单元与模板的匹配度低于90％，确定该待测单元为可疑单元。预设阈值越大，模板匹配筛选结果越准确，缺陷检测准确率越高。本公开对于预设阈值的大小并不限定。
步骤103、采用机器学习算法对可疑单元进行分类，得到可疑单元的缺陷检测类型。
在一些实施例中，缺陷检测类型包括正常类型和缺陷类型。由于可疑单元是指模板匹配中与单元模板的匹配度较低的待测单元，模板匹配的自身局限性只能对固定形状区域内的待测单元图像进行匹配，无法检测旋转以及大小发生变化的单元，因此可以采用机器学习算法进一步对可疑单元进行分类，得到可疑单元的缺陷检测类型。
在一些实施例中，缺陷类型可以由用户预先输入。在若干待检测图像帧中，人工判断是否存在缺陷产品，如果存在缺陷产品，进行人工标注。标注的具体步骤为：使用矩形包围框圈出存在缺陷的待测单元，并对不同的缺陷类型依次使用数字进行编号。将所标注的缺陷产品图像，以及编号对应的缺陷类型输入缺陷检测系统。每种缺陷类型至少标注一个缺陷产品作为分类依据的样本。
示例性的，待检测图像帧为包括多个像素电路的电路图的图像时，缺陷类型可以包括：旋转、大小变化、异物、漏固等。如图5所示，采用机器学习算法对待检测图像帧中的可疑单元进行分类，得到待检测图像帧中包括4种缺陷类型，该4种缺陷类型与数字编号的对应关系为：1表示旋转、2表示大小变化、3表示异物、4表示漏固。
在一些实施例中，机器学习算法包括：卷积神经网络算法或传统机器学习算法，卷积神经网络算法的模型包括Densenet、ResNet、Shufflenet或MobileNet，传统机器学习算法包括支持向量机SVM算法或决策树算法。根据缺陷的复杂程度选择合适的卷积神经网络算法的模型，若缺陷复杂度高，选择Densenet或ResNet，若缺陷复杂度低选择Shufflenet或MobileNet。使用传统机器学习决策树时结合LBP，Haar等特征。
图6为本公开一些实施提供的缺陷检测方法的应用示意图，如图6所示，以机器学习算法为卷积神经网络为例，在对待检测图像帧进行缺陷检测时，首先，对输入的待检测图像帧进行划分单元操作，得到待检测图像帧中的多个待测单元。然后，使用自适应法或固定法获取单元模板，并将待检测图像帧中的每个待测单元依次与单元模板进行匹配，得到每个待测单元与单元模板的匹配度。再根据待测单元与单元模板的匹配度与预设阈值进行比较，将待测单元划分为正常单元和可疑单元。然后，筛选掉模板匹配得到的正常单元，得到可疑单元。最后使用卷积神经网络对可疑单元进行分类，得到可疑单元的缺陷检测类型。
在一些实施例中，上述步骤103之后还可以包括：根据可疑单元的缺陷检测类型确定可疑单元为正常单元或缺陷单元；在可疑单元为缺陷单元时，产生告警信息并输出可疑单元的缺陷检测类型。可以理解的，如果可疑单元的缺陷检测类型为任一缺陷类型，确定该可疑单元为缺陷单元，可以产生告警信息并输出该可疑单元的缺陷类型，以使得相关人员及时发现缺陷产品并进行处理。
本公开实施例提供的缺陷检测方法，通过采用模板匹配对待检测图像帧中的待测单元进行筛选，得到可疑单元，并对可疑单元采用机器学习算法进行分类，得到该可疑单元的缺陷检测类型。如此一来，通过模板匹配能够过滤掉待检测图像帧中的正常单元，仅对待检测图像帧中的可疑单元采用机器学习算法进行分类，不仅能够减少计算量，降低计算复杂度，提升缺陷检测速度。而且能够检测出发生旋转或大小变化的待测单元，提升缺陷检测的准确性。
本公开的一些实施例提供了一种缺陷检测装置，如图7所示，该缺陷检测装置700包括处理器701、收发器702。
收发器702被配置为，获取待检测图像帧中的多个待测单元的信息，以及至少一个单元模板。
其中，多个待测单元的信息包括至少一个待测单元的位置信息和偏移量中的至少一种。
处理器701被配置为：根据多个待测单元的信息，将多个待测单元分别与单元模板进行模板匹配，得到可疑单元，可疑单元为多个待测单元中与单元模板的匹配度低于预设阈值的待测单元；采用机器学习算法对可疑单元进行分类，得到可疑单元的缺陷检测类型。
在一些实施例中，机器学习算法包括：卷积神经网络算法或传统机器学习算法，卷积神经网络算法的模型包括Densenet、ResNet、Shufflenet或MobileNet，传统机器学习算法包括支持向量机SVM算法或决策树算法。
处理器701被配置为：在待检测图像帧的多个待测单元中随机选取M个第一待测单元，以及待测单元组，每个待测单元组包括N个第二待测单元，每个第一待测单元中对应一个待测单元组；分别计算每个第一待测单元与该第一待测单元对应的N个第二待测单元的相似度，得到每个第一待测单元对应的N个相似度；根据每个第一待测单元对应的N个相似度计算第一待测单元对应的相似度平均值；将M个第一待测单元中，相似度平均值最高的第一待测单元确定为单元模板。
处理器701还被配置为：根据可疑单元的缺陷检测类型确定可疑单元为正常单元或缺陷单元；在可疑单元为缺陷单元时，产生告警信息并输出可疑单元的缺陷检测类型。
收发器702可以是通信接口，例如，输入输出接口。例如，缺陷检测装置700为芯片时，收发器702为通信接口。该收发器702也可以是射频单元。例如，在缺陷检测装置700是终端设备的情况下，收发器702可以是与天线连接的射频单元。
在一些实施例中，上述缺陷检测装置700还可以包括存储器703，该存储器703用于存储缺陷检测装置700执行上文所提供的任一缺陷检测方法所对应的程序代码和数据。该存储器703可以为只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)等。
本公开的一些实施例提供了一种非瞬态计算机可读存储介质(例如，非暂态计算机可读存储介质)，该计算机可读存储介质中存储有计算机程序指令，计算机程序指令在计算机(例如，缺陷检测装置)上运行时，使得计算机执行如上述实施例中任一实施例所述的缺陷检测方法。
示例性的，上述非瞬态计算机可读存储介质可以包括，但不限于：磁存储器件(例如，硬盘、软盘或磁带等)，光盘(例如，CD(Compact Disk，压缩盘)、DVD(Digital Versatile Disk，数字通用盘)等)，智能卡和闪存器件(例如，EPROM(Erasable Programmable Read-Only Memory，可擦写可编程只读存储器)、卡、棒或钥匙驱动器等)。本公开描述的各种计算机可读存储介质可代表用于存储信息的一个或多个设备和/或其它机器可读存储介质。术语“机器可读存储介质”可包括但不限于，无线信道和能够存储、包含和/或承载指令和/或数据的各种其它介质。
本公开的一些实施例还提供了一种计算机程序产品，例如该计算机程序产品存储在非瞬时性的计算机可读存储介质上。该计算机程序产品包括计算机程序指令，在计算机(例如，缺陷检测装置)上执行该计算机程序指令时，该计算机程序指令使计算机执行如上述实施例所述的缺陷检测方法。
本公开的一些实施例还提供了一种计算机程序。当该计算机程序在计算机上执行时，该计算机程序使计算机执行如上述实施例所述的缺陷检测方法。
上述计算机可读存储介质、计算机程序产品及计算机程序的有益效果和上述一些实施例所述的缺陷检测方法的有益效果相同，此处不再赘述。
以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。

Claims (11)

  1. 一种缺陷检测方法，其特征在于，所述方法包括：
    获取待检测图像帧中的多个待测单元的信息，以及至少一个单元模板；
    根据所述多个待测单元的信息，将所述多个待测单元分别与所述单元模板进行模板匹配，得到可疑单元，所述可疑单元为所述多个待测单元中与所述单元模板的匹配度低于预设阈值的待测单元；
    采用机器学习算法对所述可疑单元进行分类，得到所述可疑单元的缺陷检测类型。
  2. 根据权利要求1所述的方法，其特征在于，获取所述至少一个单元模板，包括：
    在所述待检测图像帧的多个待测单元中随机选取M个第一待测单元，以及待测单元组，每个所述待测单元组包括N个第二待测单元，每个所述第一待测单元中对应一个所述待测单元组；
    分别计算每个所述第一待测单元与该第一待测单元对应的N个第二待测单元的相似度，得到每个所述第一待测单元对应的N个相似度；
    根据每个所述第一待测单元对应的N个相似度计算所述第一待测单元对应的相似度平均值；
    将所述M个第一待测单元中，相似度平均值最高的所述第一待测单元确定为所述单元模板。
  3. 根据权利要求1所述的方法，其特征在于，所述多个待测单元的信息包括至少一个所述待测单元的位置信息和偏移量中的至少一种。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    根据所述可疑单元的缺陷检测类型确定所述可疑单元为正常单元或缺陷单元；
    在所述可疑单元为缺陷单元时，产生告警信息并输出所述可疑单元的缺陷检测类型。
  5. 一种缺陷检测装置，其特征在于，包括：处理器和收发器；
    所述收发器被配置为，获取待检测图像帧中的多个待测单元的信息，以及至少一个单元模板；
    所述处理器被配置为：
    根据所述多个待测单元的信息，将所述多个待测单元分别与所述单元模板进行模板匹配，得到可疑单元，所述可疑单元为所述多个待测单元中与所述单元模板的匹配度低于预设阈值的待测单元；
    采用机器学习算法对所述可疑单元进行分类，得到所述可疑单元的缺陷检测类型。
  6. 根据权利要求5所述的装置，其特征在于，所述处理器被配置为：
    在所述待检测图像帧的多个待测单元中随机选取M个第一待测单元，以及待测单元组，每个所述待测单元组包括N个第二待测单元，每个所述第一待测单元中对应一个所述待测单元组；
    分别计算每个所述第一待测单元与该第一待测单元对应的N个第二待测单元的相似度，得到每个所述第一待测单元对应的N个相似度；
    根据每个所述第一待测单元对应的N个相似度计算所述第一待测单元对应的相似度平均值；
    将所述M个第一待测单元中，相似度平均值最高的所述第一待测单元确定为所述单元模板。
  7. 根据权利要求5所述的装置，其特征在于，所述多个待测单元的信息包括至少一个所述待测单元的位置信息和偏移量中的至少一种。
  8. 根据权利要求5所述的装置，其特征在于，所述处理器还被配置为：
    根据所述可疑单元的缺陷检测类型确定所述可疑单元为正常单元或缺陷单元；
    在所述可疑单元为缺陷单元时，产生告警信息并输出所述可疑单元的缺陷检测类型。
  9. 一种电子设备，其特征在于，该电子设备包括：处理器；以及被安排成存储计算机可执行指令的存储器，所述可执行指令在被执行时使所述处理器执行如权利要求1至4中任一项所述的缺陷检测方法。
  10. 一种非瞬态计算机可读存储介质，其特征在于，所述计算机可读存储介质存储一个或多个程序，所述一个或多个程序当被处理器执行时，实现如权利要求1至4中任一项所述的缺陷检测方法。
  11. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机程序指令，在计算机上执行所述计算机程序产品指令时，所述计算机程序指令使计算机执行如权利要求1至4中任一项所述的缺陷检测方法。
PCT/CN2023/092048 2022-05-25 2023-05-04 缺陷检测方法及装置 WO2023226706A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210578969.1 2022-05-25
CN202210578969.1A CN114881996A (zh) 2022-05-25 2022-05-25 缺陷检测方法及装置

Publications (2)

Publication Number Publication Date
WO2023226706A1 WO2023226706A1 (zh) 2023-11-30
WO2023226706A9 true WO2023226706A9 (zh) 2024-01-11

Family

ID=82677973

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/092048 WO2023226706A1 (zh) 2022-05-25 2023-05-04 缺陷检测方法及装置

Country Status (2)

Country Link
CN (1) CN114881996A (zh)
WO (1) WO2023226706A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114881996A (zh) * 2022-05-25 2022-08-09 京东方科技集团股份有限公司 缺陷检测方法及装置
CN116402820A (zh) * 2023-06-07 2023-07-07 合肥联宝信息技术有限公司 一种检测方法、装置、设备及存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111583223B (zh) * 2020-05-07 2023-12-19 上海闻泰信息技术有限公司 缺陷检测方法、装置、计算机设备及计算机可读存储介质
CN111640091B (zh) * 2020-05-14 2023-08-22 阿丘机器人科技(苏州)有限公司 产品缺陷的检测方法及计算机存储介质
CN113109348B (zh) * 2021-03-12 2022-03-29 华南理工大学 一种基于机器视觉的桨影移印缺陷识别方法
CN113610801B (zh) * 2021-08-05 2023-06-20 上海众壹云计算科技有限公司 基于最小单元的缺陷分类方法、装置、设备和存储介质
CN114881996A (zh) * 2022-05-25 2022-08-09 京东方科技集团股份有限公司 缺陷检测方法及装置

Also Published As

Publication number Publication date
CN114881996A (zh) 2022-08-09
WO2023226706A1 (zh) 2023-11-30

Similar Documents

Publication Publication Date Title
WO2023226706A9 (zh) 缺陷检测方法及装置
US20210374940A1 (en) Product defect detection method, device and system
US11748873B2 (en) Product defect detection method, device and system
CN110276754B (zh) 一种表面缺陷检测方法、终端设备及存储介质
CN111161243B (zh) 基于样本增强的工业产品表面缺陷检测方法
US11741593B2 (en) Product defect detection method, device and system
WO2020248513A1 (zh) 用于综合性能试验的ocr识别方法
CN108896278A (zh) 一种滤光片丝印缺陷检测方法、装置及终端设备
CN110705630A (zh) 半监督学习式目标检测神经网络训练方法、装置及应用
CN111445459A (zh) 一种基于深度孪生网络的图像缺陷检测方法及系统
CN112037222B (zh) 一种神经网络模型的自动更新方法及系统
US20220076404A1 (en) Defect management apparatus, method and non-transitory computer readable medium
CN111310826A (zh) 样本集的标注异常检测方法、装置及电子设备
CN110927167A (zh) 一种禽蛋检测方法、装置、电子设备及存储介质
CN109410241A (zh) 图像区域生长算法的蜕变测试方法
CN115471476A (zh) 一种部件缺陷检测方法、装置、设备及介质
US20240119584A1 (en) Detection method, electronic device and non-transitory computer-readable storage medium
WO2015031350A1 (en) Systems and methods for memory utilization for object detection
CN111191670A (zh) 基于神经网络的分类装置及分类方法
CN115620083B (zh) 模型训练方法、人脸图像质量评价方法、设备及介质
Li et al. Surface defect detection algorithm of aluminum based on improved faster RCNN
CN111813593B (zh) 一种数据处理方法、设备、服务器及存储介质
CN114187292A (zh) 棉纺纸筒的异常检测方法、装置、设备及存储介质
CN112418217A (zh) 用于识别字符的方法、装置、设备和介质
CN112232288A (zh) 一种基于深度学习的卫星图目标识别方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23810779

Country of ref document: EP

Kind code of ref document: A1