WO2022121531A1 - 一种产品缺陷检测方法和装置 - Google Patents

一种产品缺陷检测方法和装置 Download PDF

Info

Publication number
WO2022121531A1
WO2022121531A1 PCT/CN2021/125590 CN2021125590W WO2022121531A1 WO 2022121531 A1 WO2022121531 A1 WO 2022121531A1 CN 2021125590 W CN2021125590 W CN 2021125590W WO 2022121531 A1 WO2022121531 A1 WO 2022121531A1
Authority
WO
WIPO (PCT)
Prior art keywords
defect
product
network
classification
image
Prior art date
Application number
PCT/CN2021/125590
Other languages
English (en)
French (fr)
Inventor
张文超
冯扬扬
刘杰
张一凡
Original Assignee
歌尔股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 歌尔股份有限公司 filed Critical 歌尔股份有限公司
Publication of WO2022121531A1 publication Critical patent/WO2022121531A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/001—Industrial image inspection using an image reference approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Definitions

  • the present application relates to the technical field of computer vision, and in particular, to a method and device for detecting product defects.
  • the embodiments of the present application provide a product defect detection method and device, which can realize efficient product defect detection, and the dependent model can be trained by relying on a small number of training samples.
  • an embodiment of the present application provides a product defect detection method, including: generating a mask image of a product image by using a Unet network; determining product features according to the distribution of specified pixel values in the mask image; setting product features corresponding to If the matching is successful, there is a defect in the product.
  • an embodiment of the present application further provides a product defect detection device, the device comprising:
  • the feature unit is used to determine product features according to the distribution of specified pixel values in the mask image.
  • a setting unit for setting defect conditions corresponding to product characteristics A setting unit for setting defect conditions corresponding to product characteristics.
  • the matching unit is used to match product features with defect conditions. If the matching is successful, there is a defect in the product.
  • embodiments of the present application further provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, the executable instructions, when executed, cause the processor to execute the above product defect detection method.
  • embodiments of the present application further provide a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, The equipment performs the product defect detection method as above.
  • the above-mentioned at least one technical solution adopted in the embodiment of the present application can achieve the following beneficial effects: applying a lightweight Unet network to a product detection scene to generate a mask, and can complete the simulation of image defect areas in the case of a small number of training samples of dozens of images Combined with the image extraction task, the generated mask can more accurately describe the information about defects, and can effectively detect product defects without being able to obtain a large number of training samples.
  • FIG. 1 shows a schematic flowchart of a product defect detection method according to an embodiment of the present application
  • FIG. 2 shows a schematic structural diagram of a Unet network according to an embodiment of the present application
  • FIG. 3 shows a schematic structural diagram of a classification network according to an embodiment of the present application
  • FIG. 4 shows a schematic structural diagram of a product defect detection device according to an embodiment of the present application
  • FIG. 5 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • the technical idea of this application is that, considering that there are many types of product defects, it is difficult to extract complex high-dimensional features with a small number of training samples, and the Unet network is used to generate the mask of the product image to reduce the complex information that is not easy to learn at this stage. , to obtain a binarized image that can still contain defect-related information, so that whether there is a defect can be easily detected according to the mask.
  • FIG. 1 shows a schematic flowchart of a product defect detection method according to an embodiment of the present application. As shown in Figure 1, the method includes:
  • Step S110 using the Unet network to generate a mask image of the product image.
  • the Unet network is a computer vision network mainly used in the medical field.
  • the reason for using the network in the embodiments of the present application is that its network structure is relatively shallow, so that only a few data sets of training samples can be used;
  • In the medical analysis scheme it has something in common with product defect detection, which can be diagnosed (detected) by analyzing geometric features such as the size of the lesion (defect).
  • FIG. 2 shows a schematic structural diagram of a Unet network according to an embodiment of the present application. It can be seen from FIG. 2 that the Unet network is characterized by a symmetrical U-shaped network, which is also the reason for the name "Unet”.
  • the first half of the Unet network performs feature extraction on the product image through a 3 ⁇ 3 convolution kernel.
  • the number of the first two convolution kernels is 64. After two additional convolution operations, the number of convolution kernels will become the original.
  • Each two convolution kernels will be matched with a 2 ⁇ 2 max pooling operation. After each max pooling, the length and width of the feature map will be halved; the number of convolution kernels in the second half is from 1024 starts to drop. After each convolution operation, the number of convolution kernels is reduced by half. With upsampling, the length and width of the feature map will double for each upsampling. And a new feature fusion method of copy and crop is added in the second half.
  • the feature map collected in the upper half is cropped to the same size as the feature map on the right, and then channel-level splicing is performed, that is, the features obtained by splicing.
  • the size of the map, the length and width are the size of the feature map on the right, and the number of channels is the sum of the number of channels of the two feature maps to be spliced.
  • the size and number of convolution kernels may be appropriately adjusted according to actual requirements.
  • a mask can be obtained by inputting the product image into Unet as shown in Figure 2.
  • the mask can be a black and white image, or a grayscale image with a grayscale value of 1 and 0. Since the proportion of defective parts in a product is usually small, generally, a small proportion of pixel values in the mask can be considered suspicious. Graphical representation of product defects.
  • Step S120 Determine product features according to the distribution of specified pixel values in the mask image. For example, a portion with a grayscale value of 1 can be regarded as a suspected defect.
  • Step S130 setting defect conditions corresponding to product features.
  • Defect conditions corresponding to product features can be set according to the actual manifestations of product defects, so as to match product features and defect conditions.
  • step S140 the product features are matched with the defect conditions. If the matching is successful, there is a defect in the product.
  • the method shown in Figure 1 uses the Unet network to not directly detect product defects, but obtains a mask image mask that can describe the product, and uses the distribution of the specified pixel values in the mask as the product feature.
  • the product features are further matched with defect conditions to determine whether the product is defective.
  • the beneficial effect of this technical solution is that the light-weight Unet network is applied to the product detection scene to generate a mask, which can complete the task of fitting and extracting image defect areas in the case of a small number of training samples of dozens of images.
  • the generated mask The information about defects can be described more accurately, and the detection of product defects can be carried out effectively without being able to obtain a large number of training samples.
  • determining the product features according to the distribution of specified pixel values in the mask image includes: determining a positioning frame that can cover the specified pixel values connected in the mask image, and using the geometric features of the positioning frame as the product feature; the defect condition is: the feature value of the geometric feature is greater than the preset threshold.
  • the positioning frame may be a circle, an ellipse, or the like, and is generally set as a rectangle, for example, obtained by calculating the minimum circumscribed rectangle for all specified pixel values in the mask image.
  • the geometric features and their corresponding preset thresholds can be determined according to the actual manifestations of defects. For example, for contamination defects, the geometric feature can be set as area, if the area is large enough, it is considered a defect; for scratch defects, the geometric feature can be set as length, if the length is long enough, it is considered a defect, etc. .
  • the position where the gray value is a specified value (for example, 1) can be saved as an index, and the length and width of the minimum circumscribed rectangle of the connected defect position can be calculated to determine whether the length and width meet the threshold requirements. , if either the length or the width satisfies the preset threshold, the product corresponding to the image is judged to be a defective product.
  • the preset threshold is considered to be defective.
  • the preset threshold in the defect feature condition is determined according to the size of the defect in each training sample image, so that the product features of a preset proportion of the training sample images can match the defect feature condition. match.
  • the preset thresholds can be determined by the size of the defects in the training sample images to reduce manual intervention.
  • setting the length threshold to 10 pixels can only match 80% of the training sample images, and if it is set to 7 pixels, it can match 90% of the training sample images, then set 7 pixels as the length threshold.
  • the Unet network is trained in the following manner: according to the defect annotation information of the training sample image, a first mask image is generated for the training sample image as the sample real value, and the specified value of the first mask image is The pixel value corresponds to the defect annotation information; the training sample image is input into the Unet network, and the second mask image output by the Unet network is obtained as the sample prediction value; the training loss value is calculated according to the sample real value and the sample prediction value, and the training loss The value is used to optimize the Unet network.
  • labeling tools such as labelme can be used to label the training sample images first, and the json annotation file can be obtained as defect annotation information, and then a mask of the same size as the training sample image can be generated according to the json annotation file as the real value of the sample.
  • the grayscale value is 1, and the grayscale value at other locations is 0. From this, the mask of each training sample image is obtained. Then each training sample image is input to the Unet network, and the output mask is obtained as the sample prediction value. Finally, by calculating the training loss value, the parameters of Unet are optimized through back propagation and other methods.
  • calculating the training loss value according to the sample real value and the sample predicted value includes: substituting the sample real value and the sample predicted value into the Dice-coefficient loss function to calculate the training loss value.
  • the training loss value is calculated using the Dice-coefficient loss function instead of the original Unet's loss function.
  • the Dice-coefficient loss function Dice-coefficient is more effective for the class imbalance problem, and can quickly measure the similarity between the generated mask image and the original mask image.
  • the formula for the Dice-coefficient loss function is: Where X is the mathematical representation of the true value of the sample, and Y is the mathematical representation of the predicted value of the sample.
  • the above method further includes: building a classification network based on the DenseNet network, the classification network includes an initial module, a backbone network and a classification module; if it is determined that there is a defect in the product, inputting the mask image into the classification network to obtain a classification Defect classification results output by the network.
  • DenseNet The "Dense” in DenseNet is because it proposes a more aggressive dense connection mechanism: it establishes a dense connection between all the previous layers and the latter layers. The input of the layer comes from the output of all the previous layers, and its name is also derived from this. DenseNet improves backpropagation of gradients due to dense connections, making the network easier to train. Also, its network structure is shallow and suitable for datasets with few training samples.
  • a classification network may also be built based on Inception, ResNet, or the like.
  • inputting the mask image into the classification network, and obtaining the defect classification result output by the classification network includes: using an initial module to obtain an initial feature map of the mask image, and the initial module includes a convolution layer and a maximum Pooling layer; use the backbone network to densify the initial feature map to obtain a dense feature map; the backbone network includes cascaded DenseX modules, convolution layers and average pooling layers, and the DenseX module includes X cascaded Dense modules, X is a positive integer; the classification module is used to output the defect classification result determined according to the dense feature map, and the classification module includes a fully connected layer and a softmax layer.
  • FIG. 3 shows a schematic structural diagram of a classification network according to an embodiment of the present application.
  • the classification network can be considered to be obtained by simplifying the network structure of DensNet121.
  • the new network structure is not complicated, but it is very effective.
  • the classification network includes an initial module, a backbone network and a classification module.
  • the initial module consists of convolution (conv) with stride 2 first, followed by max pooling with stride 2 (max pool).
  • the size of the above two-step convolution kernel and pooling block can be determined by experiment and parameter adjustment. According to the experiment, a 7 ⁇ 7 convolution kernel and a 3 ⁇ 3 maximum pooling block can be preferably used.
  • the backbone of the classification network uses four DenseX (Dense3, Dense6, Dense8 and Dense12) modules.
  • DenseX DenseX
  • the size of the feature map remains unchanged, in order to achieve the uniform size of the feature map in each Dense block (that is, the length and width of the feature map in the Dense block remain unchanged)
  • convenient Channel-level splicing is performed inside the DenseX module to achieve parameter reuse.
  • the Dense module uses a 1 ⁇ 1 convolution operation to reduce the input features after channel-level concatenation (concat)
  • This operation can not only reduce the dimensionality and reduce the amount of calculation, but also fuse the features of each channel. Therefore, a 1 ⁇ 1 convolution is directly interspersed within each Dense module and each DenseX module.
  • the last pooling layer is global average pooling (global avg pool); finally the inference results are output through the fully connected layer (fc) and the softmax layer.
  • the classification network uses a loss function formed by a combination of cross-entropy and L2 regularization term during training.
  • the initial learning rate may be set to 0.1, and a piecewise constant decays the learning rate; in some embodiments, in the above method, the classification network uses a Momentum optimizer to optimize parameters during training, and the Momentum value is preferably 0.9.
  • the solution of the present invention can meet the needs of the production line, detect defective products quickly, can effectively improve the work efficiency of the production line, and provide a better solution for subsequent comparisons.
  • Training of deep product defect detection models provides training samples.
  • the embodiment of the present application further provides a product defect detection device, which is used to implement any one of the above product defect detection methods.
  • FIG. 4 shows a schematic structural diagram of a product defect detection apparatus according to an embodiment of the present application.
  • the product defect detection device 400 includes:
  • the generating unit 410 is used for generating the mask image of the product image by using the Unet network.
  • the feature unit 420 is configured to determine product features according to the distribution of specified pixel values in the mask image.
  • the setting unit 430 is used for setting defect conditions corresponding to product features.
  • the matching unit 440 is configured to match the product features with the defect conditions. If the matching is successful, there is a defect in the product.
  • the feature unit 420 is configured to determine a positioning frame that can cover the specified pixel values connected in the mask image, and use the geometric feature of the positioning frame as a product feature; the defect condition is: the feature of the geometric feature The value is greater than the preset threshold.
  • the preset threshold in the defect feature condition is determined according to the size of the defect in each training sample image, so that the product features of a preset proportion of training sample images can match the defect feature condition.
  • the apparatus further includes a training unit for obtaining the Unet network by training in the following manner: generating a first mask image for the training sample image as the sample real value according to the defect annotation information of the training sample image, and the first mask image The specified pixel value of the image corresponds to the defect annotation information; the training sample image is input into the Unet network, and the second mask image output by the Unet network is obtained as the sample predicted value; the training loss value is calculated according to the sample real value and the sample predicted value, The optimization of the Unet network is performed according to the training loss value.
  • the training unit is configured to calculate the training loss value by substituting the actual value of the sample and the predicted value of the sample into the Dice-coefficient loss function.
  • the apparatus further includes: a classification unit for constructing a classification network based on a DenseNet network, the classification network includes an initial module, a backbone network and a classification module; if it is determined that there is a defect in the product, the mask image is input to the classification network , the defect classification results output by the classification network are obtained.
  • a classification unit for constructing a classification network based on a DenseNet network, the classification network includes an initial module, a backbone network and a classification module; if it is determined that there is a defect in the product, the mask image is input to the classification network , the defect classification results output by the classification network are obtained.
  • the classification unit is configured to obtain an initial feature map of the mask image by using an initial module, where the initial module includes a convolution layer and a maximum pooling layer; the backbone network is used to densify the initial feature map. , to obtain a dense feature map; the backbone network includes a cascaded DenseX module, a convolutional layer and an average pooling layer, the DenseX module includes X cascaded Dense modules, X is a positive integer; the classification module is used to output the output determined according to the dense feature map. Defect classification results, the classification module includes a fully connected layer and a softmax layer.
  • the classification network uses a loss function formed by a combination of cross-entropy and L2 regularization term during training.
  • the classification network uses the Momentum optimizer to perform parameter optimization during training.
  • the above-mentioned product defect detection device can implement each step of the product defect detection method provided in the foregoing embodiments, and the relevant explanations about the product defect detection method are all applicable to the product defect detection device, and are not repeated here.
  • FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory.
  • the memory may include memory, such as high-speed random-access memory (Random-Access Memory, RAM), or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
  • RAM Random-Access Memory
  • non-volatile memory such as at least one disk memory.
  • the electronic equipment may also include hardware required for other services.
  • the processor, network interface and memory can be connected to each other through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Component Interconnect Standard) bus. Industry Standard Architecture, extended industry standard structure) bus, etc.
  • the bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one bidirectional arrow is shown in FIG. 5, but it does not mean that there is only one bus or one type of bus.
  • the program may include program code, and the program code includes computer operation instructions.
  • the memory may include memory and non-volatile memory and provide instructions and data to the processor.
  • the processor reads the corresponding computer program from the non-volatile memory into the memory and runs it, forming a product defect detection device on a logical level.
  • the processor executes the program stored in the memory, and is specifically used to perform the following operations:
  • the above-mentioned method performed by the product defect detection apparatus disclosed in the embodiment shown in FIG. 1 of the present application may be applied to a processor, or implemented by a processor.
  • a processor may be an integrated circuit chip with signal processing capabilities.
  • each step of the above-mentioned method can be completed by a hardware integrated logic circuit in a processor or an instruction in the form of software.
  • the above-mentioned processor can be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processor, DSP), dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor.
  • the software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art.
  • the storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.
  • the electronic device can also execute the method executed by the product defect detection apparatus in FIG. 1 , and realize the functions of the product defect detection apparatus in the embodiment shown in FIG. 1 , which will not be repeated in this embodiment of the present application.
  • the embodiments of the present application also provide a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and the one or more programs include instructions, and the instructions are executed by an electronic device including multiple application programs.
  • the electronic device can be made to execute the method executed by the product defect detection apparatus in the embodiment shown in FIG. 1 , and is specifically used to execute:
  • the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.
  • a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • processors CPUs
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • Memory may include forms of non-persistent memory, random access memory (RAM) and/or non-volatile memory in computer readable media, such as read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
  • RAM random access memory
  • ROM read only memory
  • flash RAM flash memory
  • Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology.
  • Information may be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
  • computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

Abstract

一种产品缺陷检测方法和装置。其中，方法包括：利用Unet网络生成产品图像的掩模图像；根据掩模图像中指定像素值的分布情况确定产品特征；设置与产品特征相对应的缺陷条件；将产品特征与缺陷条件进行匹配，若匹配成功，则产品中存在缺陷。该方法将轻量的Unet网络应用于产品检测场景来生成mask，能够在几十张图的少量训练样本情况下完成图像缺陷区域的拟合与图像提取任务，生成的mask能够较为准确地描述有关缺陷的信息，可以在未能够获得大量训练样本的情况下有效地进行产品缺陷的检测。

Description

一种产品缺陷检测方法和装置 技术领域
本申请涉及计算机视觉技术领域，尤其涉及一种产品缺陷检测方法和装置。
发明背景
在产品制造过程中，由于工艺不稳定、厂房环境变化、使用原材料不良等因素的影响，生产出的产品会存在各种形态各异的缺陷、利用计算机视觉技术可以实现对人工检测缺陷方式的替代，但是大多数计算机视觉技术所依赖的产品缺陷检测模型要依赖于大量有标注的训练样本，而这依然需要较多的人工参与，成本高，效率低。
发明内容
本申请实施例提供了一种产品缺陷检测方法和装置，可以实现高效的产品缺陷检测，并且依赖的模型可依赖少量训练样本即可训练完成。
本申请实施例采用下述技术方案：
第一方面，本申请实施例提供一种产品缺陷检测方法，包括：利用Unet网络生成产品图像的掩模图像；根据掩模图像中指定像素值的分布情况确定产品特征；设置与产品特征相对应的缺陷条件；将产品特征与缺陷条件进行匹配，若匹配成功，则产品中存在缺陷。
第二方面，本申请实施例还提供一种产品缺陷检测装置，该装置包括：
生成单元，用于利用Unet网络生成产品图像的掩模图像。
特征单元，用于根据掩模图像中指定像素值的分布情况确定产品特征。
设置单元，用于设置与产品特征相对应的缺陷条件。
匹配单元，用于将产品特征与缺陷条件进行匹配，若匹配成功，则产品中存在缺陷。
第三方面，本申请实施例还提供一种电子设备，包括：处理器；以及被安排成存储计算机可执行指令的存储器，可执行指令在被执行时使处理器执行如 上的产品缺陷检测方法。
第四方面，本申请实施例还提供一种计算机可读存储介质，计算机可读存储介质存储一个或多个程序，一个或多个程序当被包括多个应用程序的电子设备执行时，使得电子设备执行如上的产品缺陷检测方法。
本申请实施例采用的上述至少一个技术方案能够达到以下有益效果：将轻量的Unet网络应用于产品检测场景来生成mask，能够在几十张图的少量训练样本情况下完成图像缺陷区域的拟合与图像提取任务，生成的mask能够较为准确地描述有关缺陷的信息，可以在未能够获得大量训练样本的情况下有效地进行产品缺陷的检测。
附图简要说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1示出了根据本申请一个实施例的产品缺陷检测方法的流程示意图；
图2示出了根据本申请一个实施例的Unet网络的结构示意图；
图3示出了根据本申请一个实施例的分类网络的结构示意图；
图4示出了根据本申请一个实施例的产品缺陷检测装置的结构示意图；
图5示出了根据本申请一个实施例的电子设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请具体实施例及相应的附图对本申请技术方案进行清楚、完整地描述。显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的技术构思在于，考虑到产品缺陷的类型众多，凭借少量的训练样本很难提取出复杂高维特征的情况，利用Unet网络生成产品图像的mask，减少 不易在此阶段学习到的复杂信息，得到仍能够包含缺陷相关信息的二值化图像，从而根据mask能够方便地进行是否存在缺陷的检测。
以下结合附图，详细说明本申请各实施例提供的技术方案。
图1示出了根据本申请一个实施例的产品缺陷检测方法的流程示意图。如图1所示，该方法包括：
步骤S110，利用Unet网络生成产品图像的掩模图像。
Unet网络是一种主要应用于医学领域的计算机视觉网络，本申请实施例使用该网络的原因在于，其网络结构较浅，能够使只有用少量训练样本的数据集；并且，在一些使用Unet进行医学分析的方案中，与产品缺陷检测有着共通之处，都可以通过对病灶(缺陷)大小等几何特征进行分析来进行诊断(检测)。
图2示出了根据本申请一个实施例的Unet网络的结构示意图，由图2可以看出，Unet网络的特点是网络呈对称的U形，这也是“Unet”得名的原因。
Unet网络的前半部分通过3×3的卷积核对产品图像进行特征提取，前两个卷积核的个数为64，以后每增加两次卷积操作，卷积核的个数都会变为原来的2倍，每两个卷积核会配合一个2×2的最大池化操作，每经过一次最大池化，特征图的长度和宽度均会减半；后半部分卷积核的个数从1024开始下降，每经过一次卷积操作，卷积核的个数就下降一半，配合上采样，每经过一次上采样，特征图的长度和宽度都会增加一倍。并且在后半部分增加了一种新的拼接(copy and crop)的特征融合方式，将上半段采集的特征图裁剪到与右侧特征图一样大小然后进行通道级拼接，即拼接得到的特征图的大小，长宽是右侧特征图的大小，通道数是两个待拼接的特征图的通道数之和。
需要说明的是，在其他实施例中可以根据实际需求对卷积核的大小、个数等进行适当调整。
将产品图像输入到如图2所示的Unet之中，就可以得到一张mask。Mask可以是黑白两色的黑白图像，或者灰度值包含1和0的灰度图像，由于产品中缺陷部分的比例通常较小，一般地，可以将mask中较少比例的像素值认为是疑似产品缺陷的图像表示。
步骤S120，根据掩模图像中指定像素值的分布情况确定产品特征。例如，可以将灰度值为1的部分认为是疑似缺陷。
步骤S130，设置与产品特征相对应的缺陷条件。
可以根据产品缺陷的实际表现形式来设置与产品特征相对应的缺陷条件，从而将产品特征与缺陷条件进行匹配。
步骤S140，将产品特征与缺陷条件进行匹配，若匹配成功，则产品中存在缺陷。
可见，图1所示的方法，采用Unet网络并不直接进行产品缺陷的检测，而是得到一张能够描述产品的掩模图像mask，将该mask中指定像素值的分布情况作为产品特征，再进一步将产品特征与缺陷条件进行匹配来判断产品是否存在缺陷。该技术方案的有益效果在于，将轻量的Unet网络应用于产品检测场景来生成mask，能够在几十张图的少量训练样本情况下完成图像缺陷区域的拟合与图像提取任务，生成的mask能够较为准确地描述有关缺陷的信息，可以在未能够获得大量训练样本的情况下有效地进行产品缺陷的检测。
在一些实施例中，上述方法中，根据掩模图像中指定像素值的分布情况确定产品特征包括：确定能够涵盖掩模图像中连通的指定像素值的定位框，将定位框的几何特征作为产品特征；缺陷条件为：几何特征的特征值大于预设阈值。
具体地，定位框可以是圆形、椭圆形等，一般地设置为矩形，例如通过对掩模图像中所有指定像素值求最小外接矩形得到。
几何特征及其对应的预设阈值可以是根据缺陷的实际表现形式来确定的。例如，对于污损缺陷，可以将几何特征设置为面积，若面积足够大，则认为是缺陷；对于划痕缺陷，可以将几何特征设置为长度，若长度足够长，则认为是缺陷，等等。
例如，可以通过遍历像素值的方式，保存灰度值为指定值(例如为1)的位置作为索引，通过计算连通的缺陷位置的最小外接矩形的长与宽，判断长与宽是否满足阈值要求，如果长或宽有一个满足预设阈值，则判断该图像所对应的产品为缺陷产品。
当然，对于不需要检测出是何种缺陷的情况，由于各种缺陷的表现形式不同，可以将多种几何特征及其对应的预设阈值配合使用，如果有一类几何特征的特征值大于对应的预设阈值，则认为是存在缺陷。
在一些实施例中，上述方法中，缺陷特征条件中的预设阈值是根据各训练样本图像中的缺陷大小确定的，从而使预设比例数量的训练样本图像的产品特征能够与缺陷特征条件相匹配。
为了避免人为设置预设阈值使得鲁棒性较差，可以通过训练样本图像中的缺陷大小来确定预设阈值，减少人工干预。
例如，将长度阈值设置成10像素，只可以匹配到80％的训练样本图像，而如果设置为7像素，就能匹配到90％的训练样本图像，此时就将7像素设置为长度阈值。
在一些实施例中，上述方法中，Unet网络是通过如下方式训练的：根据训练样本图像的缺陷标注信息，为训练样本图像生成第一掩模图像作为样本真实值，第一掩模图像的指定像素值与缺陷标注信息相对应；将训练样本图像输入到Unet网络中，得到Unet网络输出的第二掩模图像作为样本预测值；根据样本真实值和样本预测值计算训练损失值，根据训练损失值进行Unet网络的优化。
例如，可以利用labelme等标注工具先进行训练样本图像的标注，得到json标注文件作为缺陷标注信息，然后根据json标注文件生成与训练样本图像相同大小的mask作为样本真实值，其中mask的缺陷区域的灰度值为1，其他位置的灰度值为0。由此，得到了每张训练样本图像的mask。然后将各训练样本图像输入到Unet网络，得到输出的mask作为样本预测值。最后通过计算训练损失值，通过反向传播等方式进行Unet的参数优化。
在一些实施例中，上述方法中，根据样本真实值和样本预测值计算训练损失值包括：将样本真实值和样本预测值代入Dice-coefficient损失函数计算训练损失值。
在这些实施例中，使用Dice-coefficient损失函数，而非原始Unet的损失函数来计算训练损失值。原因在于，相比原始Unet的损失函数，Dice-coefficient 损失函数Dice-coefficient对于类别不均衡问题效果更优，并且能较快的度量出生成的mask图与原始mask图之间的相似度。Dice-coefficient损失函数的公式为：
Figure PCTCN2021125590-appb-000001
其中X为样本真实值的数学表示，Y为样本预测值的数学表示。
若不关注缺陷的类型，只希望确定产品是否存在缺陷，或者在生产初期尽快地收集缺陷图像，方便为后续较深的深度学习网络提供样本，那么在确定产品中存在缺陷后，就可以结束检测。如果希望进一步检测缺陷类型，那么还可以进行以下的处理。
在一些实施例中，上述方法还包括：基于DenseNet网络搭建分类网络，分类网络包括初始模块、主干网络和分类模块；若确定产品中存在缺陷，则将掩模图像输入到分类网络中，得到分类网络输出的缺陷分类结果。
DenseNet中的“Dense”是因为它提出了一个更激进的密集连接机制：它建立的是前面所有层与后面层的密集连接(dense connection)，简单来说，DenseNet所使用的模块DenseX中每一层的输入均来自前面所有层的输出，它的名称也是由此而来。由于密集连接方式，DenseNet提升了梯度的反向传播，使得网络更容易训练。同样，其网络结构较浅，适用于少量训练样本的数据集。
当然，在其他实施例中，也可以基于Inception、ResNet等搭建分类网络。
在一些实施例中，上述方法中，将掩模图像输入到分类网络中，得到分类网络输出的缺陷分类结果包括：利用初始模块得到掩模图像的初始特征图，初始模块包括卷积层和最大池化层；利用主干网络对初始特征图进行稠密化处理，得到稠密特征图；主干网络包括级联的DenseX模块、卷积层和平均池化层，DenseX模块包括X个级联的Dense模块，X为正整数；利用分类模块输出根据稠密特征图确定的缺陷分类结果，分类模块包括全连接层和softmax层。
例如，图3示出了根据本申请一个实施例的分类网络的结构示意图。该分类网络可以认为是对DensNet121的网络结构进行了精简得到的，新的网络结构不复杂，却很有效。
如图3所示，分类网络包括初始模块、主干网络和分类模块。初始模块包括先经过步长为2的卷积(conv)，然后经过步长为2的最大池化(max pool)。
以上两步卷积核与池化块的大小可以通过实验与调参确定，根据实验可以优选使用7×7的卷积核与3×3的最大池化块。
分类网络的主干部分选用了4个DenseX(Dense3，Dense6，Dense8和Dense12)模块。经过每个DenseX模块内部的时候特征图的大小保持不变，是为了实现各个Dense模块(dense block)内的特征图的尺寸统一(即Dense模块内特征图的长、宽保持不变)，方便在DenseX模块内部进行通道级拼接，实现参数重用。每经过一个2×2的平均池化(avg pool)，特征图的长度和宽带分别减半；Dense模块使用1×1的卷积操作，是为了在通道级拼接(concat)之后减少输入的特征图的深度，该操作既能降维减少计算量，又能融合各个通道的特征，因此在每个Dense模块内部和每个DenseX模块直接都会穿插使用1×1的卷积。最后一个池化层为全局平均池化(global avg pool)；最后通过全连接层(fc)和softmax层以输出推理结果。
在一些实施例中，上述方法中，分类网络在训练时使用交叉熵和L2正则化项组合形成的损失函数。初始学习率可以设置为0.1，采用分段常数衰减学习率；在一些实施例中，上述方法中，分类网络在训练时利用Momentum优化器进行参数优化，Momentum值优选为0.9。
经过实验，在产线的生产初期，缺陷样本图像数量不足的情况下，本发明方案可以满足产线需求，较快地检出有缺陷的产品，能够有效提高生产线的工作效率，并为后续较深的产品缺陷检测模型的训练提供训练样本。
本申请实施例还提供一种产品缺陷检测装置，用于实现如上任一的产品缺陷检测方法。
例如，图4示出了根据本申请一个实施例的产品缺陷检测装置的结构示意图。如图4所示，产品缺陷检测装置400包括：
生成单元410，用于利用Unet网络生成产品图像的掩模图像。
特征单元420，用于根据掩模图像中指定像素值的分布情况确定产品特征。
设置单元430，用于设置与产品特征相对应的缺陷条件。
匹配单元440，用于将产品特征与缺陷条件进行匹配，若匹配成功，则产品中存在缺陷。
在一些实施例中，上述装置中，特征单元420，用于确定能够涵盖掩模图像中连通的指定像素值的定位框，将定位框的几何特征作为产品特征；缺陷条件为：几何特征的特征值大于预设阈值。
在一些实施例中，缺陷特征条件中的预设阈值是根据各训练样本图像中的缺陷大小确定的，从而使预设比例数量的训练样本图像的产品特征能够与缺陷特征条件相匹配。
在一些实施例中，装置还包括训练单元，用于通过如下方式训练得到Unet网络：根据训练样本图像的缺陷标注信息，为训练样本图像生成第一掩模图像作为样本真实值，第一掩模图像的指定像素值与缺陷标注信息相对应；将训练样本图像输入到Unet网络中，得到Unet网络输出的第二掩模图像作为样本预测值；根据样本真实值和样本预测值计算训练损失值，根据训练损失值进行Unet网络的优化。
在一些实施例中，上述装置中，训练单元，用于将样本真实值和样本预测值代入Dice-coefficient损失函数计算训练损失值。
在一些实施例中，装置还包括：分类单元，用于基于DenseNet网络搭建分类网络，分类网络包括初始模块、主干网络和分类模块；若确定产品中存在缺陷，则将掩模图像输入到分类网络中，得到分类网络输出的缺陷分类结果。
在一些实施例中，上述装置中，分类单元，用于利用初始模块得到掩模图像的初始特征图，初始模块包括卷积层和最大池化层；利用主干网络对初始特征图进行稠密化处理，得到稠密特征图；主干网络包括级联的DenseX模块、卷积层和平均池化层，DenseX模块包括X个级联的Dense模块，X为正整数；利用分类模块输出根据稠密特征图确定的缺陷分类结果，分类模块包括全连接层和softmax层。
在一些实施例中，上述装置中，分类网络在训练时使用交叉熵和L2正则化 项组合形成的损失函数。
在一些实施例中，上述装置中，分类网络在训练时利用Momentum优化器进行参数优化。
能够理解，上述产品缺陷检测装置，能够实现前述实施例中提供的产品缺陷检测方法的各个步骤，关于产品缺陷检测方法的相关阐释均适用于产品缺陷检测装置，此处不再赘述。
图5是本申请的一个实施例电子设备的结构示意图。请参考图5，在硬件层面，该电子设备包括处理器，可选地还包括内部总线、网络接口、存储器。其中，存储器可能包含内存，例如高速随机存取存储器(Random-Access Memory，RAM)，也可能还包括非易失性存储器(non-volatile memory)，例如至少1个磁盘存储器等。当然，该电子设备还可能包括其他业务所需要的硬件。
处理器、网络接口和存储器可以通过内部总线相互连接，该内部总线可以是ISA(Industry Standard Architecture，工业标准体系结构)总线、PCI(Peripheral Component Interconnect，外设部件互连标准)总线或EISA(Extended Industry Standard Architecture，扩展工业标准结构)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图5中仅用一个双向箭头表示，但并不表示仅有一根总线或一种类型的总线。
存储器，用于存放程序。具体地，程序可以包括程序代码，程序代码包括计算机操作指令。存储器可以包括内存和非易失性存储器，并向处理器提供指令和数据。
处理器从非易失性存储器中读取对应的计算机程序到内存中然后运行，在逻辑层面上形成产品缺陷检测装置。处理器，执行存储器所存放的程序，并具体用于执行以下操作：
利用Unet网络生成产品图像的掩模图像；根据掩模图像中指定像素值的分布情况确定产品特征；设置与产品特征相对应的缺陷条件；将产品特征与缺陷条件进行匹配，若匹配成功，则产品中存在缺陷。
上述如本申请图1所示实施例揭示的产品缺陷检测装置执行的方法可以应 用于处理器中，或者由处理器实现。处理器可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，CPU)、网络处理器(Network Processor，NP)等；还可以是数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。
该电子设备还可执行图1中产品缺陷检测装置执行的方法，并实现产品缺陷检测装置在图1所示实施例的功能，本申请实施例在此不再赘述。
本申请实施例还提出了一种计算机可读存储介质，该计算机可读存储介质存储一个或多个程序，该一个或多个程序包括指令，该指令当被包括多个应用程序的电子设备执行时，能够使该电子设备执行图1所示实施例中产品缺陷检测装置执行的方法，并具体用于执行：
利用Unet网络生成产品图像的掩模图像；根据掩模图像中指定像素值的分布情况确定产品特征；设置与产品特征相对应的缺陷条件；将产品特征与缺陷条件进行匹配，若匹配成功，则产品中存在缺陷。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、 CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在一个典型的配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、 数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
以上所述仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。

Claims (14)

  1. 一种产品缺陷检测方法，包括：
    利用Unet网络生成产品图像的掩模图像；
    根据所述掩模图像中指定像素值的分布情况确定产品特征；
    设置与所述产品特征相对应的缺陷条件；
    将所述产品特征与缺陷条件进行匹配，若匹配成功，则所述产品中存在缺陷。
  2. 如权利要求1所述的方法，其中，所述根据所述掩模图像中指定像素值的分布情况确定产品特征包括：
    确定能够涵盖所述掩模图像中连通的指定像素值的定位框，将所述定位框的几何特征作为产品特征；
    所述缺陷条件为：所述几何特征的特征值大于预设阈值。
  3. 如权利要求2所述的方法，其中，所述缺陷特征条件中的预设阈值是根据各训练样本图像中的缺陷大小确定的，从而使预设比例数量的训练样本图像的产品特征能够与所述缺陷特征条件相匹配。
  4. 如权利要求1所述的方法，其中，所述Unet网络是通过如下方式训练的：
    根据训练样本图像的缺陷标注信息，为训练样本图像生成第一掩模图像作为样本真实值，所述第一掩模图像的所述指定像素值与所述缺陷标注信息相对应；
    将所述训练样本图像输入到所述Unet网络中，得到所述Unet网络输出的第二掩模图像作为样本预测值；
    根据所述样本真实值和所述样本预测值计算训练损失值，根据所述训练损失值进行所述Unet网络的优化。
  5. 如权利要求4所述的方法，其中，所述根据所述样本真实值和所述样本预测值计算训练损失值包括：
    将所述样本真实值和所述样本预测值代入Dice-coefficient损失函数计算训练损失值。
  6. 如权利要求1所述的方法，其中，所述方法还包括：
    基于DenseNet网络搭建分类网络，所述分类网络包括初始模块、主干网络和分类模块；
    若确定所述产品中存在缺陷，则将所述掩模图像输入到所述分类网络中，得到所述分类网络输出的缺陷分类结果。
  7. 如权利要求6所述的方法，其中，所述将所述掩模图像输入到分类网络中，得到所述分类网络输出的缺陷分类结果包括：
    利用所述初始模块得到掩模图像的初始特征图，所述初始模块包括卷积层和最大池化层；
    利用所述主干网络对所述初始特征图进行稠密化处理，得到稠密特征图；所述主干网络包括级联的DenseX模块、卷积层和平均池化层，所述DenseX模块包括X个级联的Dense模块，X为正整数；
    利用所述分类模块输出根据所述稠密特征图确定的缺陷分类结果，所述分类模块包括全连接层和softmax层。
  8. 如权利要求7所述的方法，其中，所述分类网络在训练时使用交叉熵和L2正则化项组合形成的损失函数。
  9. 如权利要求7所述的方法，其中，所述分类网络在训练时利用Momentum优化器进行参数优化。
  10. 一种产品缺陷检测装置，其中，所述产品缺陷检测装置包括：
    生成单元，用于利用Unet网络生成产品图像的掩模图像。
    特征单元，用于根据掩模图像中指定像素值的分布情况确定产品特征。
    设置单元，用于设置与产品特征相对应的缺陷条件。
    匹配单元，用于将产品特征与缺陷条件进行匹配，若匹配成功，则产品中存在缺陷。
  11. 如权利要求10所述的装置，其中，
    特征单元，用于确定能够涵盖掩模图像中连通的指定像素值的定位框，将定位框的几何特征作为产品特征；缺陷条件为：几何特征的特征值大于预设阈值。
  12. 如权利要求10所述的装置，其中，
    所述装置还包括训练单元，用于通过如下方式训练得到Unet网络：根据训练样本图像的缺陷标注信息，为训练样本图像生成第一掩模图像作为样本真实值，第一掩模图像的指定像素值与缺陷标注信息相对应；将训练样本图像输入到Unet网络中，得到Unet网络输出的第二掩模图像作为样本预测值；根据样本真实值和样本预测值计算训练损失值，根据训练损失值进行Unet网络的优化。
  13. 如权利要求10所述的装置，其中，所述装置还包括分类单元，用于基于DenseNet网络搭建分类网络，分类网络包括初始模块、主干网络和分类模块；若确定产品中存在缺陷，则将掩模图像输入到分类网络中，得到分类网络输出的缺陷分类结果。
  14. 一种电子设备，包括：处理器；以及被安排成存储计算机可执行指令的存储器，可执行指令在被执行时使处理器执行下述的产品缺陷检测方法：
    利用Unet网络生成产品图像的掩模图像；根据掩模图像中指定像素值的分布情况确定产品特征；设置与产品特征相对应的缺陷条件；将产品特征与缺陷条件进行匹配，若匹配成功，则产品中存在缺陷。
PCT/CN2021/125590 2020-12-09 2021-10-22 一种产品缺陷检测方法和装置 WO2022121531A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011449711.9 2020-12-09
CN202011449711.9A CN112634209A (zh) 2020-12-09 2020-12-09 一种产品缺陷检测方法和装置

Publications (1)

Publication Number Publication Date
WO2022121531A1 true WO2022121531A1 (zh) 2022-06-16

Family

ID=75309706

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/125590 WO2022121531A1 (zh) 2020-12-09 2021-10-22 一种产品缺陷检测方法和装置

Country Status (2)

Country Link
CN (1) CN112634209A (zh)
WO (1) WO2022121531A1 (zh)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115346091A (zh) * 2022-10-14 2022-11-15 深圳精智达技术股份有限公司 一种Mura缺陷图像数据集的生成方法和生成装置
CN115797877A (zh) * 2023-02-13 2023-03-14 山东鸿德电力科技有限公司 一种电力传输设备的智能监控方法、系统及介质
CN116030038A (zh) * 2023-02-23 2023-04-28 季华实验室 基于缺陷生成的无监督oled缺陷检测方法
CN116188440A (zh) * 2023-02-28 2023-05-30 聊城市红日机械配件厂 一种轴承保持架的生产分析优化方法、设备及介质
CN116245846A (zh) * 2023-03-08 2023-06-09 华院计算技术(上海)股份有限公司 带钢的缺陷检测方法及装置、存储介质、计算设备
CN116384253A (zh) * 2023-04-20 2023-07-04 中国石油大学(北京) 管道漏磁检测缺陷深度预测模型建立、预测方法及装置
CN116630242A (zh) * 2023-04-28 2023-08-22 广东励图空间信息技术有限公司 一种基于实例分割的管道缺陷评估方法及装置
CN117496003A (zh) * 2023-11-01 2024-02-02 合肥高斯智能科技有限公司 一种工业元件的缺陷图像生成方法
CN117557157A (zh) * 2023-11-30 2024-02-13 中国标准化研究院 一种基于卷积神经网络的质量考核评价方法
WO2024044947A1 (zh) * 2022-08-30 2024-03-07 宁德时代新能源科技股份有限公司 缺陷检测的方法、装置和计算机可读存储介质
CN117671431A (zh) * 2024-01-29 2024-03-08 杭州安脉盛智能技术有限公司 一种工业缺陷图像生成方法、装置、设备及存储介质
CN117710366A (zh) * 2024-02-02 2024-03-15 杭州百子尖科技股份有限公司 用于保温杯的质检方法、装置和存储介质
CN117710366B (zh) * 2024-02-02 2024-05-14 杭州百子尖科技股份有限公司 用于保温杯的质检方法、装置和存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112634209A (zh) * 2020-12-09 2021-04-09 歌尔股份有限公司 一种产品缺陷检测方法和装置
CN113674203A (zh) * 2021-07-14 2021-11-19 歌尔股份有限公司 缺陷检测模型训练方法、装置和缺陷检测方法、装置
CN113689383B (zh) * 2021-07-27 2024-04-16 南京旭锐软件科技有限公司 一种图像处理方法、装置、设备及存储介质
CN113658167B (zh) * 2021-08-24 2024-03-26 凌云光技术股份有限公司 一种含缺陷的训练图像生成方法及装置

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109242849A (zh) * 2018-09-26 2019-01-18 上海联影智能医疗科技有限公司 医学图像处理方法、装置、系统和存储介质
US20190073770A1 (en) * 2017-09-06 2019-03-07 International Business Machines Corporation Disease detection algorithms trainable with small number of positive samples
US20190370972A1 (en) * 2018-06-04 2019-12-05 University Of Central Florida Research Foundation, Inc. Capsules for image analysis
CN111179278A (zh) * 2019-12-16 2020-05-19 精英数智科技股份有限公司 一种图像检测的方法、装置、设备和存储介质
CN111179253A (zh) * 2019-12-30 2020-05-19 歌尔股份有限公司 一种产品缺陷检测方法、装置与系统
CN111598084A (zh) * 2020-05-11 2020-08-28 北京阿丘机器人科技有限公司 缺陷分割网络训练方法、装置、设备及可读存储介质
CN111814867A (zh) * 2020-07-03 2020-10-23 浙江大华技术股份有限公司 缺陷检测模型的训练方法、缺陷检测方法及相关装置
CN112634209A (zh) * 2020-12-09 2021-04-09 歌尔股份有限公司 一种产品缺陷检测方法和装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105588845B (zh) * 2016-01-04 2018-07-03 江苏科技大学 一种焊接缺陷特征参数提取方法
CN109886960A (zh) * 2019-03-27 2019-06-14 中建材凯盛机器人（上海）有限公司 基于机器视觉的玻璃边缘缺陷检测的方法
CN112001841A (zh) * 2020-07-14 2020-11-27 歌尔股份有限公司 图像待检测区域提取方法、装置和产品缺陷检测系统

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190073770A1 (en) * 2017-09-06 2019-03-07 International Business Machines Corporation Disease detection algorithms trainable with small number of positive samples
US20190370972A1 (en) * 2018-06-04 2019-12-05 University Of Central Florida Research Foundation, Inc. Capsules for image analysis
CN109242849A (zh) * 2018-09-26 2019-01-18 上海联影智能医疗科技有限公司 医学图像处理方法、装置、系统和存储介质
CN111179278A (zh) * 2019-12-16 2020-05-19 精英数智科技股份有限公司 一种图像检测的方法、装置、设备和存储介质
CN111179253A (zh) * 2019-12-30 2020-05-19 歌尔股份有限公司 一种产品缺陷检测方法、装置与系统
CN111598084A (zh) * 2020-05-11 2020-08-28 北京阿丘机器人科技有限公司 缺陷分割网络训练方法、装置、设备及可读存储介质
CN111814867A (zh) * 2020-07-03 2020-10-23 浙江大华技术股份有限公司 缺陷检测模型的训练方法、缺陷检测方法及相关装置
CN112634209A (zh) * 2020-12-09 2021-04-09 歌尔股份有限公司 一种产品缺陷检测方法和装置

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024044947A1 (zh) * 2022-08-30 2024-03-07 宁德时代新能源科技股份有限公司 缺陷检测的方法、装置和计算机可读存储介质
CN115346091B (zh) * 2022-10-14 2023-01-31 深圳精智达技术股份有限公司 一种Mura缺陷图像数据集的生成方法和生成装置
CN115346091A (zh) * 2022-10-14 2022-11-15 深圳精智达技术股份有限公司 一种Mura缺陷图像数据集的生成方法和生成装置
CN115797877A (zh) * 2023-02-13 2023-03-14 山东鸿德电力科技有限公司 一种电力传输设备的智能监控方法、系统及介质
CN116030038A (zh) * 2023-02-23 2023-04-28 季华实验室 基于缺陷生成的无监督oled缺陷检测方法
CN116188440A (zh) * 2023-02-28 2023-05-30 聊城市红日机械配件厂 一种轴承保持架的生产分析优化方法、设备及介质
CN116188440B (zh) * 2023-02-28 2023-08-29 聊城市红日机械配件厂 一种轴承保持架的生产分析优化方法、设备及介质
CN116245846A (zh) * 2023-03-08 2023-06-09 华院计算技术(上海)股份有限公司 带钢的缺陷检测方法及装置、存储介质、计算设备
CN116245846B (zh) * 2023-03-08 2023-11-21 华院计算技术(上海)股份有限公司 带钢的缺陷检测方法及装置、存储介质、计算设备
CN116384253A (zh) * 2023-04-20 2023-07-04 中国石油大学(北京) 管道漏磁检测缺陷深度预测模型建立、预测方法及装置
CN116384253B (zh) * 2023-04-20 2024-04-05 中国石油大学(北京) 管道漏磁检测缺陷深度预测模型建立、预测方法及装置
CN116630242B (zh) * 2023-04-28 2024-01-12 广东励图空间信息技术有限公司 一种基于实例分割的管道缺陷评估方法及装置
CN116630242A (zh) * 2023-04-28 2023-08-22 广东励图空间信息技术有限公司 一种基于实例分割的管道缺陷评估方法及装置
CN117496003A (zh) * 2023-11-01 2024-02-02 合肥高斯智能科技有限公司 一种工业元件的缺陷图像生成方法
CN117496003B (zh) * 2023-11-01 2024-05-14 合肥高斯智能科技有限公司 一种工业元件的缺陷图像生成方法
CN117557157A (zh) * 2023-11-30 2024-02-13 中国标准化研究院 一种基于卷积神经网络的质量考核评价方法
CN117557157B (zh) * 2023-11-30 2024-04-05 中国标准化研究院 一种基于卷积神经网络的质量考核评价方法
CN117671431A (zh) * 2024-01-29 2024-03-08 杭州安脉盛智能技术有限公司 一种工业缺陷图像生成方法、装置、设备及存储介质
CN117671431B (zh) * 2024-01-29 2024-05-07 杭州安脉盛智能技术有限公司 一种工业缺陷图像生成方法、装置、设备及存储介质
CN117710366A (zh) * 2024-02-02 2024-03-15 杭州百子尖科技股份有限公司 用于保温杯的质检方法、装置和存储介质
CN117710366B (zh) * 2024-02-02 2024-05-14 杭州百子尖科技股份有限公司 用于保温杯的质检方法、装置和存储介质

Also Published As

Publication number Publication date
CN112634209A (zh) 2021-04-09

Similar Documents

Publication Publication Date Title
WO2022121531A1 (zh) 一种产品缺陷检测方法和装置
WO2023077404A1 (zh) 缺陷检测方法、装置和系统
JP7138753B2 (ja) 画像欠陥検出方法、装置、電子機器、記憶媒体および製品
TW202040511A (zh) 資料處理方法、瑕疵的檢測方法、計算設備及儲存媒體
CN116342600B (zh) 胸腺瘤组织病理学图像中细胞核的分割方法
WO2023116632A1 (zh) 基于时空记忆信息的视频实例分割方法和分割装置
WO2022116720A1 (zh) 目标检测方法、装置和电子设备
CN113240623B (zh) 一种路面病害检测方法及装置
TWI765442B (zh) 瑕疵等級判定的方法及存儲介質
CN110599453A (zh) 一种基于图像融合的面板缺陷检测方法、装置及设备终端
US20210390282A1 (en) Training data increment method, electronic apparatus and computer-readable medium
Liu et al. An expert knowledge-empowered CNN approach for welding radiographic image recognition
Yang et al. A scratch detection method based on deep learning and image segmentation
CN109784145B (zh) 基于深度图的目标检测方法及存储介质
WO2024016686A1 (zh) 角点检测的方法和装置
CN116486166A (zh) 一种基于边缘计算的输电线异物识别检测方法
CN116468702A (zh) 黄褐斑评估方法、装置、电子设备及计算机可读存储介质
CN113780136B (zh) 基于时空纹理识别的VOCs气体泄漏检测方法、系统及设备
CN113378847B (zh) 字符分割方法、系统、计算机设备和存储介质
CN115423765A (zh) 一种基于模板图像的晶粒缺陷量化分割方法
CN114964206A (zh) 一种单目视觉里程计目标位姿检测方法
Kee et al. Cracks identification using mask region-based denoised deformable convolutional network
CN111192264A (zh) 一种基于改进非局部均值Faster-RCNN算法的纸张缺陷自动检测方法
TWI777696B (zh) 缺陷檢測方法以及使用該方法的缺陷檢測系統
Ma et al. Chip Surface Defect Recognition based on Improved Faster R-CNN

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21902238

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21902238

Country of ref document: EP

Kind code of ref document: A1