WO2024002187A1 - 缺陷检测方法、缺陷检测设备及存储介质 - Google Patents

缺陷检测方法、缺陷检测设备及存储介质 Download PDF

Info

Publication number
WO2024002187A1
WO2024002187A1 PCT/CN2023/103337 CN2023103337W WO2024002187A1 WO 2024002187 A1 WO2024002187 A1 WO 2024002187A1 CN 2023103337 W CN2023103337 W CN 2023103337W WO 2024002187 A1 WO2024002187 A1 WO 2024002187A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
defect
image
defect detection
target image
Prior art date
Application number
PCT/CN2023/103337
Other languages
English (en)
French (fr)
Inventor
严鹏
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2024002187A1 publication Critical patent/WO2024002187A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Definitions

  • the present disclosure relates to the field of artificial intelligence technology, and in particular, to a defect detection method, defect detection equipment and storage medium.
  • Embodiments of the present disclosure provide a defect detection method, defect detection equipment and storage medium.
  • embodiments of the present disclosure provide a defect detection method, which includes: obtaining a target image set of a target to be detected, and determining whether there is a suspected defective area in the target image set; wherein the suspected defective area is a possible defective area on the target to be detected.
  • Existing defective areas obtain target images with suspected defective areas to obtain a defective image set; input the defective image set into a trained classification network model; identify suspected defective areas on the target image in the defective image set through the trained classification network model, Obtain the real defect area of the target image and the corresponding real defect type.
  • inventions of the present disclosure also provide a defect detection device.
  • the defect detection device includes a processor, a memory, a computer program stored on the memory and executable by the processor, and a connection between the processor and the memory.
  • a communication data bus wherein the computer program, when executed by the processor, implements any defect detection method as provided in this disclosure.
  • embodiments of the present disclosure also provide a storage medium for computer-readable storage.
  • the storage medium stores One or more programs can be executed by one or more processors to implement any defect detection method as provided in this disclosure.
  • Figure 1 is a schematic flow chart of a defect detection method provided by an embodiment of the present disclosure
  • Figure 2 is a schematic flowchart of the sub-steps of the defect detection method in Figure 1;
  • Figure 3 is a schematic flowchart of the sub-steps of the defect detection method in Figure 2;
  • Figure 4 is a flow chart of the Halcon caliper measurement tool in the defect detection method provided by the embodiment of the present disclosure
  • Figure 5 is a schematic diagram of the alignment of the target image to be tested and the standard template object in the defect detection method provided by the embodiment of the present disclosure
  • Figure 6 is a schematic diagram of a software interface detection applied to power module defect detection using a defect detection method provided by an embodiment of the present disclosure.
  • FIG. 7 is a schematic structural block diagram of a defect detection device provided by an embodiment of the present disclosure.
  • surface defects of different products have different definitions and types. Generally speaking, surface defects are areas with uneven local physical or chemical properties on the surface of the product; such as scratches, spots, and holes on the metal surface, color differences, and indentations on the paper surface. Inclusions, damage, stains, etc. on non-metallic surfaces such as glass. Surface defects of a product not only affect its appearance and comfort, indirectly leading to a decline in sales, but may also have a negative impact on its performance. For this reason, many manufacturing companies attach great importance to the detection of surface defects in products.
  • Embodiments of the present disclosure provide a defect detection method, defect detection equipment and storage medium.
  • the defect detection method can be applied to mobile terminals, which can be electronic devices such as mobile phones, tablet computers, notebook computers, desktop computers, personal digital assistants, and wearable devices.
  • the defect detection method provided by the embodiment of the present disclosure will be introduced in detail with reference to the scene in FIG. 1 . It should be noted that the scenario in Figure 1 is only used to explain the defect detection method provided by the embodiment of the present disclosure, but does not constitute a limitation on the application scenarios of the defect detection method provided by the embodiment of the present disclosure.
  • FIG. 1 is a schematic flowchart of a defect detection method provided by an embodiment of the present disclosure.
  • the defect detection method includes steps S101 to S104.
  • Step S101 Obtain a target image set of the target to be detected, and determine whether there is a suspected defective area in the target image set; where the suspected defective area is a defective area that may exist on the target to be detected, such as a scratch that may exist on the target to be detected. Defects such as marks and pits.
  • the industrial camera interface can be called, the industrial camera can be started through soft triggering, and sample images of the target to be detected can be collected.
  • the industrial camera can be any industrial camera with normal imaging.
  • the resolution of the image may be 2592x1944, using an industrial camera such as a Basler industrial camera.
  • the target to be detected may be a device on a production line, such as a power module on a power supply production line.
  • the time interval for taking pictures by the industrial camera can be set according to the transmission speed of the production line.
  • the interval can be 2 seconds.
  • step S101 some image acquisition techniques may be used to determine whether there is a suspected defective area in the target image in the target image set.
  • Step S101 may include: sub-steps S1011 to sub-step S1012.
  • Sub-step S1011 Use multiple target images to obtain a standard template image, where the standard template image is formed from multiple target images through mixed Gaussian background modeling.
  • target images can be selected from the target image set, and of course, several target images can also be collected from the production line through industrial cameras.
  • mixed Gaussian background modeling is a background representation method based on the statistical information of pixel samples, using statistical information such as the probability density of a large number of sample values of pixels over a long period of time (such as the number of modes, the mean and standard deviation of each mode ) represents the background, and then uses statistical difference to determine the target pixel, which can model complex dynamic backgrounds.
  • the color information between pixels can be considered to be independent of each other, so the processing of each pixel is independent of each other.
  • the change of its value in the sequence image can be regarded as a random process that continuously generates pixel values. That is, a Gaussian distribution is used to describe the color presentation rule of each pixel.
  • each pixel of the image is modeled by the superposition of multiple Gaussian distributions with different weights. Each Gaussian distribution corresponds to a state that may produce the color of the pixel, and the weight of each Gaussian distribution and distribution parameters are updated over time.
  • x N represents the observation data of the Nth random variable
  • the process of obtaining standard template images through Gaussian mixture modeling is mainly as follows.
  • Preprocessing methods can include: correcting the brightness, contrast, hue, etc. of the target image. Preprocessing the target image can make the results of subsequent detection of defective areas more accurate.
  • some parameters such as variance, mean, and weight in the mixed Gaussian model are first initialized, and the data required for modeling are obtained through these parameters.
  • the variance can be set larger and the weight set smaller, so that the parameter value can be continuously narrowed and updated, and as many pixels as possible can be included in the mixed Gaussian model, so that the mixed Gaussian model can comprehensively identify the characteristics of each target image. defective area.
  • five mixed Gaussian models can be used to characterize the characteristics of each pixel in the target image, and the collected gray value of each pixel of the current target image is used as the mean value.
  • the variance can be set to 15 and the weight can be set to 0.001.
  • the next target image is then fed into the Gaussian mixture model to update the Gaussian mixture model. That is to say, each pixel in the next target image is matched with the mixed Gaussian model. If successful, the point is determined to be a background point, otherwise it is a foreground point. Based on the judgment of background and foreground, the background and foreground are divided. Foreground, resulting in a standard template image.
  • Sub-step S1012 Align the remaining target images in the target image set with the standard template image, and locate the suspected defective areas on the remaining target images in the target image set.
  • aligning the target image with the standard template image is helpful for identifying suspected defective areas on the target image.
  • sub-step S1012 may include: sub-steps S10121-S10123.
  • Sub-step S10121 The remaining target images in the target image set (that is, all the target objects to be measured) can be aligned with the standard template image respectively, and the remaining target images in the target image set can be calculated as a difference with the standard template image respectively, to obtain difference image.
  • aligning the target image to be tested with the standard template object is helpful to find all suspected defective areas of the target image to be tested.
  • Halcon German company developed a very complete machine vision algorithm package.
  • the specific alignment process can include:
  • Figure 4 is a flow chart of the Halcon caliper measurement tool in the defect detection method provided by the embodiment of the present disclosure.
  • a measuring caliper is set on the target image to be measured, and edge points on the target image to be measured are detected one by one.
  • After extracting the edge points obtain the plane grayscale data of the edge points, smooth the plane grayscale data, and finally perform derivation calculation on the edge points and fit the edges to obtain the target contour.
  • Figure 5 is a schematic diagram of the alignment of the target image to be tested and the standard template object in the defect detection method provided by the embodiment of the present disclosure; in the target outline area, two straight lines are constructed, namely Line 1 and Line 2. The intersection of straight line 1 and straight line 2 is the "X" point in the upper left corner of Figure 5.
  • the difference between the remaining target images in the target image set and the standard template image respectively also includes: using Gaussian pyramid to optimize the remaining target images in the target image set and the standard template image respectively, where, The remaining target images in the target image set are respectively aligned with the standard template images. Create a Gaussian pyramid for the target image and calculate it from high to low to facilitate subsequent identification of suspected defective areas.
  • Gaussian pyramid is a kind of multi-scale expression in images, and is mainly used for image segmentation.
  • Gaussian pyramid obtains a series of downsampled images through Gaussian smoothing and subsampling. That is to say, the Kth layer Gaussian pyramid can obtain K+1 layer Gaussian images through smoothing and subsampling.
  • the Gaussian pyramid contains a series of low-pass filters whose cutoff frequency can gradually increase from the upper layer to the next layer by a factor of 2, so the Gaussian pyramid can span a large frequency range. The higher the level, the smaller the image and the resolution The lower the rate. Among them, K is a positive integer.
  • Sub-step S10122 Binarize the difference image to obtain a binarized image, and perform threshold segmentation on the binarized image to obtain a segmented image.
  • Halcon's own operator can be used to perform threshold segmentation on the binary image.
  • Sub-step S10123 Filter the segmented image, and extract the outline of the filtered segmented image to obtain the suspected defective area.
  • the segmented images may be further filtered through opening operations.
  • the main steps of the opening operation can include two steps: corrosion processing and expansion processing.
  • the segmented image is first corroded, for example, the edges of the segmented image can be reduced, which can enhance defects. degree of regional differentiation. Then the corroded segmented image is expanded. The dilation process can significantly enhance the probability of identifying defective areas.
  • a preset contour extraction operator can be used to extract all the contours of the filtered segmented image, thereby locating all suspected defective areas in the image to be tested.
  • Step S102 Obtain target images containing suspected defective areas to obtain a defective image set.
  • step S101 it can be determined which target images have suspected defective areas among the collected target images and which target images do not have suspected defective areas. If there is no suspected defective area in the target image, it means that the corresponding target product has no defects, so the target image without suspected defective area can be eliminated. The remaining target images all have suspected defect areas, so the remaining target images can be combined into a defect image set.
  • the embodiment of the present disclosure can implement the initial detection of the target image set of the target to be detected, and determine which target images contain suspected defective areas.
  • Step S103 Input the defect image set into the trained classification network model, identify the suspected defect area on the target image in the defect image set through the trained classification network model, and obtain the real defect area of the target image and the corresponding real defect type.
  • a classification network model can be obtained by training some lightweight network model structures.
  • the training process of the classification network model is as follows.
  • QT is a cross-platform C++ graphical user interface application development framework.
  • the classification network model can be trained based on the lightweight network model structure. For example, it can be trained based on the lightweight network MobileNet_v3. Initialize the weights of the lightweight network MobileNet_v3, and then set the number of iterations.
  • Figure 6 is a schematic diagram of the software interface detection applied to power module defect detection using the defect detection method provided by the embodiment of the present disclosure.
  • the lightweight network may include a first convolution layer group and a second convolution layer group that communicate in sequence.
  • the first convolution layer group in the lightweight network model is used to extract the defect features of the training data; the defect features are identified through the second convolution layer group in the lightweight network model to output the defects of the target image in the training data. area and corresponding defect type.
  • the first convolutional layer group may include multiple communicatively connected convolutional layers, and the convolutional kernel of each convolutional layer is 3x3. Lightweight network model.
  • the second convolutional layer group may include two convolutional layers with a convolution kernel of 1x1.
  • the lightweight network model extracts features through 3x3 convolution, with multiple convolutional layers in between. Finally, two 1x1 convolutional layers are used instead of fully connected to output the defect area and the corresponding defect category.
  • step 24 Use the test set prepared in advance to input the classification network model, and test and calculate the accuracy of the classification network model on the new test set. If the accuracy reaches the predetermined requirement, the classification network model can be saved as the final classification network model, otherwise, step 23) continues to be repeated for training.
  • the defect image set obtained in step S102 can be input into the classification network model trained in step 24), and then analyzed and compared with the defect image set to obtain an accurate defect area.
  • step 3 target images with precise defect areas and defect types are used as new training data, and the same operation as step 3) is performed to train the classification network model to achieve the effect of continuously iteratively optimizing the model, thus greatly improving the accuracy of the detection results. If the defective area located on the target image by the classification network model does not correspond to the label defined in step 21), the defective area located at that moment may not be the real defective area, so the target image with the defective area can be discarded. .
  • the embodiment of the present disclosure first obtains the suspected defective area of the target image, and then uses the classification network model to classify the suspected defective area. Since the suspected defective area is acquired in advance, it only needs to identify the suspected defective area to accurately identify the suspected defective area. The real defect areas and the corresponding defect types of each real defect area are detected. At this time, it is no longer necessary to obtain a large number of image samples of the target to be detected, and the defect area and type of defects of the target to be detected can still be accurately identified.
  • the defect image set into the trained classification network model may also include: obtaining a target image sample with defects of the target to be detected, and a type label corresponding to the type of defect, to obtain training data.
  • the defect detection method can be applied to the field of defect detection of power modules, and mainly includes the following solutions.
  • a power module image set i.e., target object set
  • 20 power module images can be selected from the power module image set.
  • Block images, or re-collect 20 power module images from the production line, and the 20 power module images are collected at different angles and different light intensities.
  • 20 power module images to construct a standard template image; use the Halcon tool to align the power module image with the standard template image, and then perform difference calculation, binarization, segmentation, filtering, and extraction of all suspected defective areas in sequence.
  • the defect detection method provided by the above embodiment can quickly detect the defective area of the power module without collecting a large number of power module image samples in advance to train the model, which is conducive to the promotion and application of the defect detection method.
  • FIG. 7 is a schematic structural block diagram of a defect detection device provided by an embodiment of the present disclosure.
  • the defect detection device 300 includes a processor 301 and a memory 302.
  • the processor 301 and the memory 302 are connected through a bus 303, which is, for example, an I2C (Inter-integrated Circuit) bus.
  • I2C Inter-integrated Circuit
  • the processor 301 is used to provide computing and control capabilities to support the operation of the entire defect detection equipment.
  • the processor 301 can be a central processing unit (Central Processing Unit, CPU).
  • the processor 301 can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC). ), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general processor may be a microprocessor or the processor may be any conventional processor.
  • the memory 302 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a USB disk, a mobile hard disk, or the like.
  • ROM read-only memory
  • the memory 302 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a USB disk, a mobile hard disk, or the like.
  • FIG. 7 is only a block diagram of a partial structure related to the embodiments of the present disclosure, and does not constitute a limitation on the defect detection equipment to which the embodiments of the present disclosure are applied.
  • Specific defect detection equipment may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
  • the processor 301 is used to run the computer program stored in the memory 302, and implement any defect detection method provided by the embodiments of the present disclosure when executing the computer program.
  • the processor 301 is used to run a computer program stored in the memory, and when executing the computer program, implement the steps of the defect detection method as described above, including the following: obtaining a target image of the target to be detected Set, determine whether there is a suspected defective area in the target image set; where the suspected defective area is the defective area that may exist on the target to be detected; obtain the target image with the suspected defective area to obtain the defective image set; input the defective image set
  • the trained classification network model is used to identify suspected defect areas on the target image in the defect image set through the trained classification network model, and obtain the real defect area of the target image and the corresponding real defect type.
  • the processor 301 when implemented, is used to implement the steps of the defect detection method as described above, including the following: obtaining a target image set of the target to be detected, and determining whether the target image in the target image set exists. Suspected defective area; where the suspected defective area is the defective area that may exist on the target to be detected; obtain the target image with the suspected defective area to obtain a defective image set; input the defective image set into the trained classification network model; through the trained The classification network model identifies the suspected defect areas on the target image in the defect image set, and obtains the real defect areas of the target image and the corresponding real defect types.
  • Embodiments of the present disclosure also provide a storage medium for computer-readable storage.
  • the storage medium stores one or more programs.
  • the one or more programs can be executed by one or more processors to implement the embodiments of the present disclosure. Any defect detection method provided in the instruction manual.
  • the storage medium may be an internal storage unit of the defect detection device of the aforementioned embodiment, such as a hard disk or memory of the defect detection device.
  • the storage medium can also be an external storage device of the defect detection equipment, such as a plug-in hard drive, a smart memory card (Smart Media Card, SMC), a secure digital (SD) card, or a flash memory card equipped on the defect detection equipment. (Flash Card) etc.
  • Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).
  • computer storage media includes volatile and nonvolatile media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. removable, removable and non-removable media.
  • Computer storage media include but are not limited to RAM, ROM, EEPROM, Flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical disk storage, magnetic cassettes, tapes, disk storage or other magnetic storage devices, or anything that can be used to store the desired information and can be accessed by a computer other media.
  • communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .
  • Embodiments of the present disclosure provide a defect detection method, defect detection equipment, and storage media, aiming to avoid collecting a large number of samples and to detect defects of each target.
  • Embodiments of the present disclosure provide a defect detection method, defect detection equipment and storage medium, which first obtains suspected defect areas and then uses a classification network model to classify the suspected defect areas. Since the suspected defect areas are obtained in advance, only the suspected defect areas need to be classified. By identifying the defect areas, the real defect areas and the corresponding defect types of each real defect area can be accurately detected. At this time, it is no longer necessary to obtain a large number of image samples of the target to be detected, and the defect area and type of defects of the target to be detected can still be accurately identified.

Abstract

本公开实施例提供一种缺陷检测方法、缺陷检测设备及存储介质，属于人工智能技术领域，该缺陷检测方法包括：获取待检测目标的目标图像集，确定目标图像集中的目标图像是否存在疑似缺陷区域；其中，疑似缺陷区域为待检测目标上可能存在的缺陷区域；获取存在疑似缺陷区域的目标图像，得到缺陷图像集；将缺陷图像集输入训练好的分类网络模型；通过训练好的分类网络模型识别缺陷图像集中目标图像上的疑似缺陷区域，得到目标图像的真实缺陷区域以及相应的真实缺陷类型。

Description

缺陷检测方法、缺陷检测设备及存储介质
相关申请的交叉引用
本公开要求享有2022年06月30日提交的名称为“缺陷检测方法、缺陷检测设备及存储介质”的中国专利申请CN202210762871.1的优先权，其全部内容通过引用并入本公开中。
技术领域
本公开涉及人工智能技术领域，尤其涉及一种缺陷检测方法、缺陷检测设备及存储介质。
背景技术
随着社会的不断进步，用户和生产企业对各项产品质量的要求越来越高。而在实际的产品生产过程中，产品表面产生缺陷往往是不可避免的。产品的表面缺陷不仅影响自身的美观和舒适度，间接导致销量下降，而且也可能给自身的使用性能带来不良影响。为此，很多生产企业非常重视产品的表面缺陷检测。一些情形下采用基于深度学习模型的缺陷检方法，但是该种方法存在需要前期收集大量的样本的技术问题。
发明内容
本公开实施例提供了一种缺陷检测方法、缺陷检测设备及存储介质。
第一方面，本公开实施例提供一种缺陷检测方法，包括：获取待检测目标的目标图像集，确定目标图像集中的目标图像是否存在疑似缺陷区域；其中，疑似缺陷区域为待检测目标上可能存在的缺陷区域；获取存在疑似缺陷区域的目标图像，得到缺陷图像集；将缺陷图像集输入训练好的分类网络模型；通过训练好的分类网络模型识别缺陷图像集中目标图像上的疑似缺陷区域，得到目标图像的真实缺陷区域以及相应的真实缺陷类型。
第二方面，本公开实施例还提供一种缺陷检测设备，缺陷检测设备包括处理器、存储器、存储在存储器上并可被处理器执行的计算机程序以及用于实现处理器和存储器之间的连接通信的数据总线，其中所述计算机程序被处理器执行时，实现如本公开说明书提供的任一项缺陷检测方法。
第三方面，本公开实施例还提供一种存储介质，用于计算机可读存储，存储介质存储有 一个或者多个程序，一个或者多个程序可被一个或者多个处理器执行，以实现如本公开说明书提供的任一项缺陷检测方法。
附图说明
为了更清楚地说明本公开实施例技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种缺陷检测方法的流程示意图；
图2为图1中的缺陷检测方法的子步骤流程示意图；
图3为图2中的缺陷检测方法的子步骤流程示意图；
图4为本公开实施例提供的缺陷检测方法中Halcon卡尺测量工具流程图；
图5为本公开实施例提供的缺陷检测方法中待测目标图像和标准模板对象对齐示意图；
图6为本公开实施例提供缺陷检测方法的应用于电源模块缺陷检测的软件界面检测示意图；以及
图7为本公开实施例提供的一种缺陷检测设备的结构示意框图。
具体实施方式
下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
附图中所示的流程图仅是示例说明，不是必须包括所有的内容和操作/步骤，也不是必须按所描述的顺序执行。例如，有的操作/步骤还可以分解、组合或部分合并，因此实际执行的顺序有可能根据实际情况改变。
应当理解，在此本公开说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本公开。如在本公开说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。
随着社会的不断进步，用户和生产企业对各项产品质量的要求越来越高。许多用户以及 生产企业，不仅要求产品的使用性能达标甚至远超达标条件，还要求产品具有良好的外观。良好的外观即良好的表面质量，而在实际的产品生产过程中，产品表面产生缺陷往往是不可避免的。
不同产品的表面缺陷有着不同的定义和类型，一般而言，表面缺陷是产品表面局部物理或化学性质不均匀的区域；如金属表面的划痕、斑点、孔洞，纸张表面的色差、压痕，玻璃等非金属表面的夹杂、破损、污点等。产品的表面缺陷不仅影响自身的美观和舒适度，间接导致销量下降，而且也可能给自身的使用性能带来不良影响。为此，很多生产企业非常重视产品的表面缺陷检测。
传统的产品表面缺陷检测方法是人工抽检。但是人工抽检，存在抽检率低、准确性不高、实时性差、效率低、劳动强度大、受人工经验和主观因素的影响大等缺陷。所以现在有一些生产企业采用基于深度学习模型的缺陷检方法，但是该种方法需要前期收集大量的样本，通过大量的样本来训练深度学习模型，才能保证检测的准确率。
前期收集大量的样本会耗费人力、物力，而且有些生产行业的有效样本较少，难以大量收集，这就给采用基于深度学习模型的缺陷检方法的落实带来了一定的困难。
本公开实施例提供一种缺陷检测方法、缺陷检测设备及存储介质。其中，该缺陷检测方法可应用于移动终端中，该移动终端可以手机、平板电脑、笔记本电脑、台式电脑、个人数字助理和穿戴式设备等电子设备。
下面结合附图，对本公开的一些实施例作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
以下，将结合图1中的场景对本公开的实施例提供的缺陷检测方法进行详细介绍。需知，图1中的场景仅用于解释本公开实施例提供的缺陷检测方法，但并不构成对本公开实施例提供的缺陷检测方法应用场景的限定。
请参照图1，图1为本公开实施例提供的一种缺陷检测方法的流程示意图，该缺陷检测方法包括步骤S101至步骤S104。
步骤S101、获取待检测目标的目标图像集，确定目标图像集中的目标图像是否存在疑似缺陷区域；其中，疑似缺陷区域为待检测目标上可能存在的缺陷区域，例如待检测目标上可能存在的划痕、凹坑等缺陷。
在本公开实施例中，可以调用工业相机接口，通过软触发的方式启动工业相机，对待检测目标进行样本图像的采集。当然，该工业相机只要是任何成像正常的工业相机即可。
在一示例性实施例中，图像的分辨率可以为2592x1944，工业相机比如Basler工业相机等。
在一示例性实施例中，待检测目标可以为生产线上的器件，例如可以为电源生产线上的电源模块。
此外，因为生产线要运输待检测目标，所以一般情况下生产线运动的。这种情况下，就需要设置相机的拍照时间，才能够保证采集到生产线上所有的待检测目标的图像。所以，在本公开实施例中，可以根据产线的传输速度，设置工业相机拍照的时间间隔，例如，间隔可以为2s。
在步骤S101中，可以通过一些图像采集技术，来确定目标图像集中的目标图像是否存在疑似缺陷区域。
在一示例性实施例中，请参照图2，图2为图1中的缺陷检测方法的子步骤流程示意图，步骤S101可以包括：子步骤S1011至子步骤S1012。
子步骤S1011、利用多个目标图像，获取标准模板图像，其中，标准模板图像由多个目标图像通过混合高斯背景建模形成。
可以在目标图像集中选择若干个目标图像，当然也可以另外再通过工业相机从生产线上采集若干个目标图像。
需要强调的是，若干个目标图像需要在不同角度、不同光线强度下采集得到的。
可以理解的是，混合高斯背景建模是基于像素样本统计信息的背景表示方法，利用像素在较长时间内大量样本值的概率密度等统计信息(如模式数量、每个模式的均值和标准差)表示背景，然后使用统计差分进行目标像素判断，可以对复杂动态背景进行建模。
在混合高斯背景模型中，可以认为像素之间的颜色信息互不相关，所以对各像素点的处理都是相互独立的。对于目标图像中的每一个像素点，其值在序列图像中的变化可看作是不断产生像素值的随机过程，即用高斯分布来描述每个像素点的颜色呈现规律。对于多峰高斯分布模型，图像的每一个像素点按不同权值的多个高斯分布的叠加来建模，每种高斯分布对应一个可能产生像素点所呈现颜色的状态，各个高斯分布的权值和分布参数随时间更新。
综上，当处理彩色图像时，假定图像像素点R、G、B三色通道相互独立并具有相同的方差。对于随机变量X的观测数据集{x1,x2,...xN}，xt＝(rt,gt,bt)为t时刻像素的采样点，则单个采样点xt是服从混合高斯分布概率密度函数的。
其中，xN表示第N个随机变量X的观测数据，N为正整数，rt,gt,bt分别表示t时刻R、G、B三色通道上像素点。
而通过混合高斯建模获取标准模板图像的过程主要如下所述。
11)首先在不同角度、不同光线强度下，采集若干张合格的待检测目标的样本图像，优选为20张，并对20张目标图像进行预处理。
预处理方式可以包括：对目标图像亮度、对比度、色调等进行校正，对目标图像进行预处理，可以使后续检测缺陷区域的结果更加准确。
12)对预处理后的目标对象进行混合高斯建模，建立一个普适性高的标准模板图像。
具体的混合高斯建模过程中，首先对混合高斯模型中的方差、均值、权值等一些参数初始化，并通过这些参数求出建模所需的数据。此外，可以将方差设置大些，权值设置小些，这样可以不断缩小范围更新参数值，将尽可能多的像素包含到混合高斯模型里面，从而使混合高斯模型能够全面地识别各个目标图像的缺陷区域。
本公开实施例中，首先在混合高斯模型初始化阶段，可以使用5个混合高斯模型来表征目标图像中各个像素点的特征，将采集到的当前目标图像的每个像素的灰度值作为均值，将方差可以设置为15，权值可以设置为0.001。
然后将下一个目标图像输入混合高斯模型，以更新混合高斯模型。也就是说，将下一个目标图像中的每个像素点与混合高斯模型匹配，如果成功则判定该点为背景点，否则为前景点，根据背景、前景的判断，也即划分出了背景与前景，从而得到标准模板图像。
子步骤S1012、将目标图像集中剩余的目标图像分别与标准模板图像相对齐，定位目标图像集中剩余的目标图像上的疑似缺陷区域。
一般来说，将目标图像分别与标准模板图像相对齐，有利于识别目标图像上的疑似缺陷区域。
在本实施例中，请参照图3，图3为图2中的缺陷检测方法的子步骤流程示意图；子步骤S1012可以包括：子步骤S10121-S10123。
子步骤S10121、可以将目标图像集中剩余的目标图像(也即所有的待测目标对象)分别与标准模板图像相对齐，对目标图像集中剩余的目标图像分别与标准模板图像进行差值计算，得到差值图像。
值得一提的是，将待测目标图像和标准模板对象对齐，有利于找出待测目标图像的所有疑似缺陷区域。为了方便将待测目标图像和标准模板对象对齐，本公开实施例的方案中，可 以使用Halcon的测量卡尺将待测目标图像和标准模板对象对齐。其中，Halcon德国公司开发的一套很完善的机器视觉算法包。
具体的对齐流程可以包括：
13)请参照图4，图4为本公开实施例提供的缺陷检测方法中Halcon卡尺测量工具流程图。首先利用一些小的边缘检测矩形，然后在待测目标图像上设置测量卡尺，逐一检测出待测目标图像上的边缘点。提取边缘点后，获取边缘点的平面灰度数据，将平面灰度数据进行平滑处理，最终对边缘点进行求导计算，拟合边缘求出目标轮廓。
14)请参照图5，图5为本公开实施例提供的缺陷检测方法中待测目标图像和标准模板对象对齐示意图；在目标轮廓区域内，构建两条直线，分别为直线1、直线2。直线1与直线2的交点，即图5中左上角的“X”点处。
15)最后使用Halcon的测量卡尺，将待测目标图像和标准模板图像进行交点点、直线线对位，也即将两条直线相交的交点、两条直线对齐，即可使待测目标图像与标准模板图像精确地对齐。
要强调的是，在对目标图像集中剩余的目标图像分别与标准模板图像进行差值计算之前，还包括：利用高斯金字塔，将目标图像集中剩余的目标图像分别与标准模板图像进行优化，其中，目标图像集中剩余的目标图像分别与标准模板图像相对齐。对目标图像建立高斯金字塔，由高到低进行计算，方便进行后续识别疑似缺陷区域的速度。
可以理解的是，高斯金字塔是图像中多尺度表达的一种，最主要用于图像的分割。高斯金字塔是通过高斯平滑和亚采样获得一些列下采样图像，也就是说第K层高斯金字塔通过平滑、亚采样就可以获得K+1层高斯图像。高斯金字塔包含了一系列低通滤波器，其截止频率可以以因子2从上一层到下一层逐渐增加，所以高斯金字塔可以跨越很大的频率范围，层级越高，则图像越小，分辨率越低。其中，K为正整数。
子步骤S10122、将差值图像进行二值化处理，得到二值化图像，并对二值化图像进行阈值分割，得到分割图像。
在本公开的一实施例中，可以使用Halcon自带的算子对二值化图像进行阈值分割。
子步骤S10123、过滤分割图像，并提取过滤后的分割图像的轮廓，得到疑似缺陷区域。
在本公开的一实施例中，可以通过开运算对分割图像进一步过滤。开运算主要步骤可以包括腐蚀处理、膨胀处理两个步骤。
其中，先对分割图像进行腐蚀处理，例如可以为缩减分割图像边缘，这样可以增强缺陷 区域的区分程度。接着再对腐蚀后的分割图像进行膨胀处理。膨胀处理能够明显增强缺陷区域被识别的概率。
为了方便提取过滤后的分割图像的轮廓，可以使用预设的轮廓提取算子，从而提取出过滤后的分割图像的所有轮廓，也就定位出待测图像中所有的疑似缺陷区域。
步骤S102、获取存在疑似缺陷区域的目标图像，得到缺陷图像集。
通过步骤S101可以确定所采集的目标图像集中，哪些目标图像上存在疑似缺陷区域，哪些目标图像上不存在疑似缺陷区域。目标图像不存在疑似缺陷区域，也就说明相应的目标产品没有瑕疵，所以可以剔除不存在疑似缺陷区域的目标图像。剩下的目标图像，都存在了疑似缺陷区域，所以可以将剩下的目标图像组合成缺陷图像集。
综上，本公开实施例通过步骤S101、步骤S102，即可实现对待检测目标的目标图像集进行初次检测，确定出哪些目标图像中存在疑似缺陷区域。
步骤S103、将缺陷图像集输入训练好的分类网络模型，通过训练好的分类网络模型识别缺陷图像集中目标图像上的疑似缺陷区域，得到目标图像的真实缺陷区域以及相应的真实缺陷类型。
可以通过对一些轻量网络模型结构进行训练，从而得到分类网络模型。在一示例性实施例中，以待检测目标为电源模块为例，分类网络模型的训练过程如下所述。
21)用户可以通过QT调用工业相机采集生产线上的电源模块图像，并且根据缺陷类型制作标签，将采集的电源模块图像、相应的标签作为训练数据。其中，QT是跨平台C++图形用户界面应用程序开发框架。
假设电源模块主要的缺陷有四种，例如为刮痕、缺失元器件、凹陷、元器件位置不在预设范围内四种。相应的，用户可以制作4种对应缺陷类型的标签，分别为{defect1、defect2、defect3、defect4}，从而收集到训练数据。
22)分类网络模型可以基于轻量网络模型结构训练得到，例如，可以基于轻量级网络MobileNet_v3训练得到。初始化轻量级网络MobileNet_v3的权重，然后设置迭代次数。
23)将训练数据输入到轻量级网络MobileNet_v3中进行训练，当迭代完之后，导出分类网络模型。
对电源模块进行实际检测时，运行的软件界面可以参照图6所示，图6为本公开实施例提供缺陷检测方法的应用于电源模块缺陷检测的软件界面检测示意图。
使用轻量级网络训练分类模型，可以提高缺陷区域识别过程的实时性。而在本公开一实 施例中，轻量级网络可以包括依次通信的第一卷积层组、第二卷积层组。其中，利用轻量级网络模型中第一卷积层组提取训练数据的缺陷特征；通过轻量级网络模型中第二卷积层组对缺陷特征进行识别，以输出训练数据中目标图像的缺陷区域以及相应的缺陷类型。
其中，第一卷积层组可以包括多个相通信连接的卷积层，每个卷积层的卷积核为3x3。轻量级网络模型。第二卷积层组可以包括两个卷积层，其卷积核为1x1。
综上，轻量级网络模型通过3x3的卷积，提取特征，中间经过多个卷积层。最后通过两个1x1的卷积层，代替全连接，输出缺陷区域以及相应的缺陷类别。
24)利用事先准备的测试集，输入分类网络模型，测试计算该分类网络模型在新的测试集上准确率。如果准确率达到预定要求，则可以将该分类网络模型保存为最后的分类网络模型，否则继续重复步骤23)进行训练。
可以将步骤S102得到的缺陷图像集输入步骤24)训练出来的分类网络模型，然后再与缺陷图像集进行分析对比，得到精确的缺陷区域。
同时将具有精确缺陷区域的目标图像以及缺陷类型，作为新的训练数据，执行步骤3)相同的操作，训练分类网络模型，达到不断迭代优化模型的效果，从而大大提高检测结果的准确率。如果该分类网络模型在目标图像上定位的缺陷区域，与步骤21)中所定义的标签无法对应，则该时刻所定位的缺陷区域可能并非真实缺陷区域，所以可以抛弃具有该缺陷区域的目标图像。
综上，本公开实施例通过先获取目标图像的疑似缺陷区域，再利用分类网络模型对疑似缺陷区域进行分类，因事先获取了疑似缺陷区域，只需要对疑似缺陷区域进行识别，就能够精确的检测出真实缺陷区域以及每个真实缺陷区域相应的缺陷类型。此时不再需要获取大量的待检测目标的图像样本，依旧能够准确识别出待检测目标缺陷区域以及缺陷的类型。
在本公开一实施例中，在将缺陷图像集输入训练好的分类网络模型之前，还可以包括：获取具有待检测目标的缺陷的目标图像样本，以及与缺陷的种类相对应的类型标签，得到训练数据。
将训练数据输入预设网络模型进行迭代训练，以得到分类网络模型。
本公开的一实施例中，缺陷检测方法可以应用于电源模块的缺陷检测领域，主要包括如下所述的方案。
31)通过工业相机采集生产线上的电源模块图像，作为电源模块图像集(也即目标对象集)，确定电源模块图像是否存在疑似缺陷区域；可以从电源模块图像集中选取20张电源模 块图像，或者重新从生产线上采集20张电源模块图像，且20张电源模块图像是在不同角度、不同光线强度下采集得到的。然后利用20张电源模块图像构建标准模板图像；利用Halcon工具，将电源模块图像与标准模板图像对齐，然后依次进行差值计算、二值化处理、分割处理、过滤、提取所有的疑似缺陷区域。
32)剔除不存在疑似缺陷区域的电源模块图像，剩余的电源模块图像组成电源模块缺陷图像集。
33)将组成电源模块缺陷图像集输入训练好的分类网络模型，通过分类网络模型输出电源模块缺陷区域以及相应的缺陷类型。
上述实施例提供的缺陷检测方法，能够快速检测出电源模块的缺陷区域，而且不需要事先采集大量的电源模块图像样本用来训练模型，有利于缺陷检测方法的推广应用。
请参阅图7，图7为本公开实施例提供的一种缺陷检测设备的结构示意框图。如图7所示，缺陷检测设备300包括处理器301和存储器302，处理器301和存储器302通过总线303连接，该总线比如为I2C(Inter-integrated Circuit)总线。
在一示例性实施例中，处理器301用于提供计算和控制能力，支撑整个缺陷检测设备的运行。处理器301可以是中央处理单元(Central Processing Unit，CPU)，该处理器301还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。其中，通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
在一示例性实施例中，存储器302可以是Flash芯片、只读存储器(ROM，Read-Only Memory)磁盘、光盘、U盘或移动硬盘等。
本领域技术人员可以理解，图7中示出的结构，仅仅是与本公开实施例方案相关的部分结构的框图，并不构成对本公开实施例方案所应用于其上的缺陷检测设备的限定，具体的缺陷检测设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
其中，处理器301用于运行存储在存储器302中的计算机程序，并在执行计算机程序时实现本公开实施例提供的任意一种的缺陷检测方法。
在本公开的一实施例中，处理器301用于运行存储在存储器中的计算机程序，并在执行计算机程序时实现如上所述的缺陷检测方法的步骤，包括如下：获取待检测目标的目标图像 集，确定目标图像集中的目标图像是否存在疑似缺陷区域；其中，疑似缺陷区域为待检测目标上可能存在的缺陷区域；获取存在疑似缺陷区域的目标图像，得到缺陷图像集；将缺陷图像集输入训练好的分类网络模型；通过训练好的分类网络模型识别缺陷图像集中目标图像上的疑似缺陷区域，得到目标图像的真实缺陷区域以及相应的真实缺陷类型。
在本公开的一实施例中，处理器301在实现时，用于实现如上所述的缺陷检测方法的步骤，包括如下：获取待检测目标的目标图像集，确定目标图像集中的目标图像是否存在疑似缺陷区域；其中，疑似缺陷区域为待检测目标上可能存在的缺陷区域；获取存在疑似缺陷区域的目标图像，得到缺陷图像集；将缺陷图像集输入训练好的分类网络模型；通过训练好的分类网络模型识别缺陷图像集中目标图像上的疑似缺陷区域，得到目标图像的真实缺陷区域以及相应的真实缺陷类型。
需要说明的是，所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的缺陷检测设备的具体工作过程，可以参考前述缺陷检测方法实施例中的对应过程，在此不再赘述。
本公开实施例还提供一种存储介质，用于计算机可读存储，存储介质存储有一个或者多个程序，一个或者多个程序可被一个或者多个处理器执行，以实现如本公开实施例说明书提供的任一项缺陷检测方法。
其中，存储介质可以是前述实施例的缺陷检测设备的内部存储单元，例如缺陷检测设备的硬盘或内存。存储介质也可以是所述缺陷检测设备的外部存储设备，例如缺陷检测设备上配备的插接式硬盘，智能存储卡(Smart Media Card，SMC)，安全数字(Secure Digital，SD)卡，闪存卡(Flash Card)等。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施例中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、 闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。
本公开实施例提供了一种缺陷检测方法、缺陷检测设备及存储介质，旨在避免收集大量的样本，也能够对各个目标的缺陷进行检测。本公开实施例提供一种缺陷检测方法、缺陷检测设备及存储介质，其通过先获取疑似缺陷区域，再利用分类网络模型对疑似缺陷区域进行分类，因事先获取了疑似缺陷区域，只需要对疑似缺陷区域进行识别，就能够精确的检测出真实缺陷区域以及每个真实缺陷区域相应的缺陷类型。此时不再需要获取大量的待检测目标的图像样本，依旧能够准确识别出待检测目标缺陷区域以及缺陷的类型。
应当理解，在本公开说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。
上述本公开实施例序号仅仅为了描述，不代表实施例的优劣。以上所述，仅为本公开的具体实施例，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以权利要求的保护范围为准。

Claims (10)

  1. 一种缺陷检测方法，包括：
    获取待检测目标的目标图像集，确定所述目标图像集中的目标图像是否存在疑似缺陷区域；其中，所述疑似缺陷区域为所述待检测目标上可能存在的缺陷区域；
    获取存在所述疑似缺陷区域的目标图像，得到缺陷图像集；
    将所述缺陷图像集输入训练好的分类网络模型；
    通过所述训练好的分类网络模型识别所述缺陷图像集中目标图像上的疑似缺陷区域，得到所述目标图像的真实缺陷区域以及相应的真实缺陷类型。
  2. 根据权利要求1所述的缺陷检测方法，其中，所述确定所述目标图像集中的目标图像是否存在疑似缺陷区域，包括：
    利用所述目标图像集中的多个目标图像，获取标准模板图像，其中，所述标准模板图像由所述多个目标图像通过混合高斯背景建模形成；
    将所述目标图像集中剩余的目标图像分别与所述标准模板图像相对齐，定位所述目标图像集中剩余的目标图像上的疑似缺陷区域。
  3. 根据权利要求2所述的缺陷检测方法，其中，所述将所述目标图像集中剩余的目标图像分别与所述标准模板图像相对齐，定位所述目标图像集中剩余的目标图像上的疑似缺陷区域，包括：
    将所述目标图像集中剩余的目标图像分别与所述标准模板图像相对齐，对所述目标图像集中剩余的目标图像分别与所述标准模板图像进行差值计算，得到差值图像；
    将所述差值图像进行二值化处理得到二值化图像，并对所述二值化图像进行阈值分割，得到分割图像；
    过滤所述分割图像，并提取所述过滤后的分割图像的轮廓，得到疑似缺陷区域。
  4. 根据权利要求3所述的缺陷检测方法，其中，所述过滤所述分割图像，并提取所述过滤后的分割图像的轮廓，包括：
    通过开运算过滤所述分割图像；
    利用预设的轮廓提取算子提取所述过滤后的分割图像的轮廓。
  5. 根据权利要求3所述的缺陷检测方法，其中，在对所述目标图像集中剩余的目标图像 分别与所述标准模板图像进行差值计算之前，还包括：
    利用高斯金字塔，将所述目标图像集中剩余的目标图像分别与所述标准模板图像进行优化，其中，所述目标图像集中剩余的目标图像分别与所述标准模板图像相对齐。
  6. 根据权利要求1所述的缺陷检测方法，其中，将所述缺陷图像集输入训练好的分类网络模型之前，还包括：
    获取具有所述待检测目标的缺陷的目标图像样本，以及与所述缺陷的种类相对应的类型标签，得到训练数据；
    将所述训练数据输入预设网络模型进行迭代训练，以得到所述分类网络模型。
  7. 根据权利要求6所述的缺陷检测方法，其中，所述预设网络模型包括依次相连的第一卷积层组、第二卷积层组；所述将所述训练数据输入预设的分类网络进行迭代训练，包括：
    利用所述轻量级网络模型中第一卷积层组提取所述训练数据的缺陷特征；
    通过所述轻量级网络模型中第二卷积层组对所述缺陷特征进行识别，以输出所述训练数据中目标图像的缺陷区域以及相应的缺陷类型。
  8. 根据权利要求1-7任一项所述的缺陷检测方法，其中，所述方法还包括：
    将所述真实缺陷区域以及所述相应的真实缺陷类型输入至所述分类网络模型中进行训练，以优化所述分类网络模型。
  9. 一种缺陷检测设备，包括处理器、存储器、存储在所述存储器上并可被所述处理器执行的计算机程序以及用于实现所述处理器和所述存储器之间的连接通信的数据总线，其中所述计算机程序被所述处理器执行时，实现如权利要求1至8中任一项所述的缺陷检测方法的步骤。
  10. 一种存储介质，用于计算机可读存储，所述存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现权利要求1至8中任一项所述的缺陷检测方法的步骤。
PCT/CN2023/103337 2022-06-30 2023-06-28 缺陷检测方法、缺陷检测设备及存储介质 WO2024002187A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210762871.1A CN117392042A (zh) 2022-06-30 2022-06-30 缺陷检测方法、缺陷检测设备及存储介质
CN202210762871.1 2022-06-30

Publications (1)

Publication Number Publication Date
WO2024002187A1 true WO2024002187A1 (zh) 2024-01-04

Family

ID=89383124

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/103337 WO2024002187A1 (zh) 2022-06-30 2023-06-28 缺陷检测方法、缺陷检测设备及存储介质

Country Status (2)

Country Link
CN (1) CN117392042A (zh)
WO (1) WO2024002187A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117576105A (zh) * 2024-01-17 2024-02-20 高科建材（咸阳）管道科技有限公司 基于人工智能的管道生产控制方法及系统
CN117745715A (zh) * 2024-02-06 2024-03-22 中科院南京耐尔思光电仪器有限公司 基于人工智能的大口径望远镜镜片缺陷检测方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3401672A1 (en) * 2016-01-08 2018-11-14 SCREEN Holdings Co., Ltd. Flaw detection device and flaw detection method
CN111640091A (zh) * 2020-05-14 2020-09-08 阿丘机器人科技(苏州)有限公司 产品缺陷的检测方法及计算机存储介质
CN112288727A (zh) * 2020-10-30 2021-01-29 创新奇智(重庆)科技有限公司 磁环表面缺陷检测方法、装置、电子设备及存储介质
CN113658121A (zh) * 2021-08-09 2021-11-16 国能榆林能源有限责任公司 一种线路缺陷检测方法、系统及计算机存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3401672A1 (en) * 2016-01-08 2018-11-14 SCREEN Holdings Co., Ltd. Flaw detection device and flaw detection method
CN111640091A (zh) * 2020-05-14 2020-09-08 阿丘机器人科技(苏州)有限公司 产品缺陷的检测方法及计算机存储介质
CN112288727A (zh) * 2020-10-30 2021-01-29 创新奇智(重庆)科技有限公司 磁环表面缺陷检测方法、装置、电子设备及存储介质
CN113658121A (zh) * 2021-08-09 2021-11-16 国能榆林能源有限责任公司 一种线路缺陷检测方法、系统及计算机存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117576105A (zh) * 2024-01-17 2024-02-20 高科建材（咸阳）管道科技有限公司 基于人工智能的管道生产控制方法及系统
CN117576105B (zh) * 2024-01-17 2024-03-29 高科建材（咸阳）管道科技有限公司 基于人工智能的管道生产控制方法及系统
CN117745715A (zh) * 2024-02-06 2024-03-22 中科院南京耐尔思光电仪器有限公司 基于人工智能的大口径望远镜镜片缺陷检测方法
CN117745715B (zh) * 2024-02-06 2024-04-23 中科院南京耐尔思光电仪器有限公司 基于人工智能的大口径望远镜镜片缺陷检测方法

Also Published As

Publication number Publication date
CN117392042A (zh) 2024-01-12

Similar Documents

Publication Publication Date Title
CN106875381B (zh) 一种基于深度学习的手机外壳缺陷检测方法
CN112581463B (zh) 图像缺陷的检测方法、装置、电子设备、存储介质及产品
WO2023077404A1 (zh) 缺陷检测方法、装置和系统
CN108074231B (zh) 一种基于卷积神经网络的磁片表面缺陷检测方法
WO2024002187A1 (zh) 缺陷检测方法、缺陷检测设备及存储介质
CN111693534B (zh) 表面缺陷的检测方法、模型训练方法、装置、设备和介质
CN110544231B (zh) 基于背景标准化和集中补偿算法的锂电池电极表面缺陷检测方法
CN111815564B (zh) 一种检测丝锭的方法、装置及丝锭分拣系统
TWI765442B (zh) 瑕疵等級判定的方法及存儲介質
CN111126393A (zh) 车辆外观改装判断方法、装置、计算机设备及存储介质
US20220076404A1 (en) Defect management apparatus, method and non-transitory computer readable medium
CN115880520A (zh) 一种基于模板匹配和缺陷分割的缺陷等级分类方法及系统
CN115937101A (zh) 质量检测方法、装置、设备及存储介质
CN115690670A (zh) 一种晶圆瑕疵智能识别方法和系统
CN115471476A (zh) 一种部件缺陷检测方法、装置、设备及介质
CN116071315A (zh) 一种基于机器视觉的产品可视缺陷检测方法及系统
CN112014407A (zh) 一种集成电路晶圆表面缺陷检测的方法
WO2021046726A1 (zh) 用于检测机械设备零部件的方法和装置
TWI694250B (zh) 表面缺陷偵測系統及其方法
CN113705564B (zh) 一种指针式仪表识别读数方法
TW201512649A (zh) 偵測晶片影像瑕疵方法及其系統與電腦程式產品
CN114359161A (zh) 一种缺陷检测方法、装置、设备和存储介质
CN113744252A (zh) 用于标记和检测缺陷的方法、设备、存储介质和程序产品
Kuo et al. Automated inspection of micro-defect recognition system for color filter
CN110930393A (zh) 一种基于机器视觉的芯片料管计数方法、装置及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23830358

Country of ref document: EP

Kind code of ref document: A1