WO2021135300A1 - 一种产品缺陷检测方法、装置与系统 - Google Patents

一种产品缺陷检测方法、装置与系统 Download PDF

Info

Publication number
WO2021135300A1
WO2021135300A1 PCT/CN2020/111313 CN2020111313W WO2021135300A1 WO 2021135300 A1 WO2021135300 A1 WO 2021135300A1 CN 2020111313 W CN2020111313 W CN 2020111313W WO 2021135300 A1 WO2021135300 A1 WO 2021135300A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
product
image block
candidate
block
Prior art date
Application number
PCT/CN2020/111313
Other languages
English (en)
French (fr)
Inventor
刘杰
田继锋
谢馥励
邸顺然
张一凡
Original Assignee
歌尔股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 歌尔股份有限公司 filed Critical 歌尔股份有限公司
Priority to US17/309,407 priority Critical patent/US11741593B2/en
Publication of WO2021135300A1 publication Critical patent/WO2021135300A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component

Definitions

  • the invention relates to a product defect detection method, device and system.
  • the purpose of the embodiments of the present invention is to provide a product defect detection method, device and system.
  • an embodiment of the present invention provides a product defect detection method, including:
  • Collect sample images of products extract candidate image blocks that may have product defects from the sample images, and extract the preset shape features corresponding to the candidate image blocks and the texture characteristics corresponding to the candidate image blocks;
  • an embodiment of the present invention provides a product defect detection device, including:
  • the preprocessing unit is used to collect sample images of products, extract candidate image blocks that may have product defects from the sample images, and extract the preset shape features corresponding to the candidate image blocks and the texture features corresponding to the candidate image blocks; use the preset shape
  • the feature trains the first-level classifier to obtain the first-level classifier that can further filter out the target image blocks that may have product defects from the candidate image blocks; use the texture features to train the second-level classifier to get the correct Second-level classifier that identifies product defects;
  • the defect detection unit is used to input the preset shape features of the candidate image blocks extracted from the product image into the first-level classifier during product defect detection, and then input the obtained texture features of the target image block into the first classifier.
  • the secondary classifier detects defects in the product.
  • an embodiment of the present invention provides a product defect detection system, including: a memory and a processor;
  • Memory storing computer executable instructions
  • the processor when executed, cause the processor to execute the product defect detection method.
  • an embodiment of the present invention provides a computer-readable storage medium, and one or more computer programs are stored on the computer-readable storage medium, and the product defect detection method is implemented when the one or more computer programs are executed.
  • the present invention has at least the following technical effects: the present invention trains a first-level classifier and a second-level classifier through sample images in advance, and when detecting product defects, it extracts candidate images that may have product defects from product images
  • the block completes the first screening of image blocks, and uses the first-level classifier to screen the candidate image blocks of the product image for the second time.
  • the rough classification of the image blocks is completed, and then Use the second-level classifier to screen the target image block for the third time, and complete the fine classification operation of the image block through the third screening.
  • the target with product defects can be screened out.
  • the image block can quickly and accurately detect the defect of the product based on the target image block with the product defect.
  • FIG. 1 is a block diagram of the hardware configuration of a product defect detection system shown in an embodiment of the present invention
  • FIG. 2 is a flowchart of a product defect detection method shown in an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a training process of a second-level classifier according to an embodiment of the present invention
  • FIG. 4 is a schematic diagram of a defect detection framework shown in an embodiment of the present invention.
  • FIG. 5 is a structural block diagram of a product defect detection device shown in an embodiment of the present invention.
  • Fig. 6 is a structural block diagram of a product defect detection system according to an embodiment of the present invention.
  • FIG. 1 is a block diagram of the hardware configuration of a product defect detection system 100 according to an embodiment of the present invention.
  • the product defect detection system 100 includes an image acquisition device 1000 and a product defect detection device 2000.
  • the image collection device 1000 is used to collect product images and provide the collected product images to the product defect detection device 2000.
  • the image acquisition device 1000 may be any imaging device capable of taking pictures, such as a camera.
  • the product defect detection device 2000 may be any electronic device, such as a PC, a notebook computer, a server, and so on.
  • the product defect detection device 2000 may include a processor 2100, a memory 2200, an interface device 2300, a communication device 2400, a display device 2500, an input device 2600, a speaker 2700, a microphone 2800, and so on.
  • the processor 2100 may be a mobile version processor.
  • the memory 2200 includes, for example, ROM (Read Only Memory), RAM (Random Access Memory), nonvolatile memory such as a hard disk, and the like.
  • the interface device 2300 includes, for example, a USB interface, a headphone interface, and the like.
  • the communication device 2400 can perform wired or wireless communication, for example, and the communication device 2400 may include short-range communication devices, such as based on Hilink protocol, WiFi (IEEE 802.11 protocol), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, Any device that performs short-range wireless communication with a short-range wireless communication protocol such as LiFi.
  • the communication device 2400 may also include a remote communication device, for example, any device that performs WLAN, GPRS, 2G/3G/4G/5G remote communication.
  • the display device 2500 is, for example, a liquid crystal display screen, a touch screen, etc.
  • the display device 2500 is used to display product images collected by the image collection device 1000.
  • the input device 2600 may include, for example, a touch screen, a keyboard, and the like. The user can input/output voice information through the speaker 2700 and the microphone 2800.
  • the memory 2200 of the product defect detection apparatus 2000 is used to store instructions for controlling the processor 2100 to operate to at least execute the product defect detection method according to any embodiment of the present invention.
  • Technical personnel can design instructions according to the disclosed scheme of the present invention. How the instruction controls the processor to operate is well known in the art, so it will not be described in detail here.
  • the present invention may only involve some of them.
  • the product defect detection device 2000 only involves the memory 2200, the processor 2100, and the display device 2500.
  • the image capture device 1000 is used to capture product images and provide them to the product defect detection device 2000, and the product defect detection device 2000 implements the product defect detection method according to any embodiment of the present invention based on the image.
  • FIG. 1 only shows one image acquisition device 1000 and one product defect detection device 2000, it does not mean to limit the respective quantities.
  • the product defect detection system 100 may include multiple image acquisition devices 1000 and/or Product defect detection device 2000.
  • the overall idea of the embodiments of this application is to first generate image blocks that may have product defects based on the image of the screen-like product, and roughly classify these image blocks based on the shape characteristics of the image blocks , Get the image blocks that may have defects, and then extract the texture features of the image blocks that may have defects, and finally classify them based on the texture features using a classifier to identify the defects included in the screen product.
  • a wire mesh product is taken as an example to illustrate specific technical details, but the technical solution of this embodiment is not limited to the detection scene of the wire mesh product.
  • FIG. 2 is a flowchart of a product defect detection method shown in an embodiment of the present invention. As shown in FIG. 2, the method in this embodiment includes:
  • S2100 Collect a sample image of the product, extract candidate image blocks that may have product defects from the sample image, and extract a preset shape feature corresponding to the candidate image block and a texture feature corresponding to the candidate image block.
  • This step completes the first screening of image blocks by extracting candidate image blocks that may have product defects from the sample images.
  • multiple initial image blocks can be detected by performing area detection on the sample image of the product. For example, in a sample image with a size of 512*512 pixels, for a screen-like product, if the pixel size corresponding to the product grid is 100*100, then about 36 initial image blocks can be obtained through area detection.
  • this step can be based on the feature that the normal image block and the image block where the product defect is located have a large difference in area, and extract from the initial image block based on the area feature of the image block Candidate image blocks are extracted, and the extracted candidate image blocks include image blocks that may have defects.
  • S2200 Use preset shape features to train the first-level classifier to obtain a first-level classifier that can further filter out target image blocks that may have product defects from the candidate image blocks.
  • this embodiment trains the first-level classifier based on the preset shape features of the candidate image blocks to perform rough classification operations to screen out target image blocks with a higher probability of product defects during product defect detection, thereby reducing subsequent The number of image blocks that need to be processed for the fine classification operation reduces the overall time-consuming of defect detection.
  • the rough classification operation includes the first image block screening to extract candidate image blocks that may have product defects from the sample image and the second image block screening to classify the candidate image blocks using the first-level classifier, and the fine classification operation Including the third image block screening that uses the second-level classifier to classify the target image block.
  • S2300 Use texture features to train the second-level classifier to obtain a second-level classifier that can correctly identify product defects.
  • the first-level classifier is trained based on the shape features of the candidate image blocks, and the target image blocks separated by the first-level classifier may include image blocks without product defects. Therefore, this step uses the texture features of the candidate image block to train the second-level classifier, so that the trained second-level classifier can accurately classify the target image block for product defects when classifying the target image block.
  • the image blocks and the image blocks without product defects are classified, so as to screen out the target image blocks with product defects, and complete the detection of product defects.
  • S2400 When performing product defect detection, input the preset shape features of the candidate image block extracted from the product image into the first-level classifier, and then input the obtained texture feature of the target image block to the second-level classifier, Detect defects in the product.
  • the first-level classifier and the second-level classifier are trained in advance through sample images.
  • the first image block of the image block is completed by extracting candidate image blocks that may have product defects from the product image.
  • Second screening and use the first-level classifier to screen the candidate image blocks of the product image for the second time, complete the rough classification operation of the image blocks through the first screening and the second screening, and then use the second-level classifier to
  • the target image block is screened for the third time, and the fine classification operation of the image block is completed through the third screening.
  • the target image block with product defects can be screened out.
  • the target image block of the product defect can quickly and accurately detect the defect of the product.
  • the embodiment of the present application also provides a product defect detection method.
  • the sample image includes multiple positive sample images, one negative sample image, and one label image, and the positive sample image is an unqualified product image with product defects.
  • the negative sample image is a qualified product image without product defects
  • the pixel value of the position where the product defect is marked is the first value (for example, 255)
  • the pixel value of the non-defect position is the second value (for example, 0)
  • the binary image is used to the first value (for example, 255)
  • extracting candidate image blocks that may have product defects from the sample image includes S2110 to S2130:
  • S2110 Acquire initial image blocks included in each positive sample image and negative sample image.
  • the initial image block can be extracted by the following method:
  • Perform region detection on the binarized image for example, use the region detection function regionprops in the image detection tool skimage.measure to perform region detection on the binarized image to obtain the initial image blocks included in each positive sample image and negative sample image.
  • S2120 Screen initial image blocks in the negative sample image whose image block area is greater than a first preset area value, and use the screened initial image blocks as the candidate image blocks extracted from the negative sample image.
  • the first preset area value can be set according to product defect characteristics. For example, if the grid area where the damage defect of a certain screen product is located is generally greater than 100 pixels, then the initial image block with an area greater than 100 pixels detected from the negative sample image can be used as the candidate image of the negative sample image block, there is shown a block negative candidate image with the sample image P N.
  • S2130 Screen the initial image blocks in each positive sample image that intersect with the position of the image block on the labeled image and whose intersection area is greater than a second preset value, and use the screened initial image blocks as candidate image blocks extracted from the positive sample image .
  • the second preset value can be set according to actual needs.
  • the larger the second preset value the more likely the candidate image block selected from the initial image block of the positive sample image will have product defects.
  • the selected candidate image The smaller the number of blocks.
  • the smaller the second preset value is, the less likely the candidate image blocks screened from the initial image blocks of the positive sample image will have product defects, and the greater the number of screened candidate image blocks.
  • the initial image block that intersects with the image block where the defect on the marked image is located, and the intersection area is greater than 50% of the initial image block area is used as the candidate image block of the positive sample image.
  • P represents the candidate image block of the positive sample image.
  • the first screening of image blocks is completed.
  • the following scheme 1 can be used to extract the preset shape features corresponding to the candidate image blocks, and use the extracted preset shapes
  • the feature trains the first-level classifier; and extracts the texture features corresponding to the candidate image blocks through the following scheme two, and uses the extracted texture features to train the second-level classifier.
  • the preset shape features corresponding to the candidate image blocks are extracted through the following steps S11 to S12, and the first-level classifier is trained using the extracted preset shape features.
  • S11 Extract one or more of the four features of the area area of the candidate image block, the area of the smallest circumscribed rectangle, the eccentricity and the area ratio as a preset shape feature.
  • the eccentricity ratio is the ratio of the focal length of the ellipse with the same second moment to the length of the major axis of the candidate image block. That is, the candidate image block is equivalent to an ellipse with the same second-order moment, and the ratio of the focal length of the ellipse to the length of the major axis is calculated, and the ratio is the eccentricity of the candidate image block. Since the shape of each candidate image block may be different, the second moment of the ellipse equivalent to each candidate image block may also be different, and the corresponding eccentricity of each candidate image block may also be different. Assuming that there are 50 candidate image blocks, each candidate image block can be equivalent to an ellipse, and the 50 ellipses may have different second-order moments.
  • the area ratio is the ratio of the number of pixels of the candidate image block to the number of pixels of the smallest bounding rectangle of the candidate image block.
  • a candidate program in the image block is an image block represented by P P P N and to screen out the positive sample image from the candidate image block P P represents an example, a preferred embodiment is in the area of P P, P P is the smallest area of the circumscribed rectangle, and the eccentricity P P P P wherein the area ratio of the four preset shape as P wherein P candidate image block.
  • the classifier performs adversarial training, and optimizes the classifier parameters of the first-level classifier based on the adversarial training result until the adversarial training result meets the preset value. For example, the adversarial training is stopped when the classification accuracy rate of the first-level classifier reaches 95%, and the classifier parameters at this time are used as the final parameters of the first-level classifier.
  • the first-level classifiers include, but are not limited to, SVM (Support Vector Machine) classifiers, K-nearest neighbor classifiers, decision tree classifiers (including XGBClassifier), and so on. Since XGBClassifier has the characteristics of fast running speed and high accuracy, this embodiment can use XGBClassifier as the first-level classifier.
  • SVM Serial Vector Machine
  • K-nearest neighbor classifiers decision tree classifiers (including XGBClassifier)
  • XGBClassifier has the characteristics of fast running speed and high accuracy
  • this embodiment can use XGBClassifier as the first-level classifier.
  • the gbtree tree model can be used as the base classifier.
  • the texture features corresponding to the candidate image blocks are extracted through the following steps S21 to S22, and the second-level classifier is trained using the extracted texture features.
  • the size of the minimum circumscribed rectangle of each candidate image block is also different.
  • the size of the smallest circumscribed rectangular frame of each candidate image block is adjusted to obtain the circumscribed rectangular frame of each candidate image block after adjustment, and the circumscribed rectangular frame of each candidate image block after adjustment has the same size.
  • the image area of the sample image where the circumscribed rectangle of each candidate image block is located is detected, and the LBP feature of the image area is extracted as the texture feature of the candidate image block.
  • the smallest bounding rectangles of the three candidate image blocks patch1, patch2, and patch3 are box1', box2', and box3', respectively. Adjust the sizes of the smallest circumscribed rectangular boxes box1', box2', and box3' to the sizes of box1, box2, and box3, respectively, so that the adjusted circumscribed rectangular boxes box1, box2, and box3 have the same size.
  • the image region I1 of the sample image Image where the circumscribed rectangular box box1 of the candidate image block patch1 is located is detected, and the LBP feature of the region image I1 is extracted as the texture feature of the candidate image block patch1.
  • the image area I2 (or I3) of the sample image Image where the circumscribed rectangular frame box2 (or box3) of the candidate image block patch2 (or patch3) is located is detected, and the image area I2 (or I3) is extracted.
  • the LBP feature of I3) is the texture feature of the candidate image block patch2 (or patch3).
  • S22 Set the initial parameters of the second-level classifier, and use the texture features of the candidate image blocks extracted from the negative sample image and the texture features of the candidate image blocks extracted from the positive sample image to confront the second-level classifier Training, optimize the classifier parameters of the second-level classifier based on the adversarial training result until the adversarial training result meets the preset value. For example, the confrontation training is stopped when the classification accuracy rate reaches 95%.
  • the second-level classifier can be implemented by an SVM classifier.
  • the process of training the second-level classifier includes:
  • the candidate image blocks of the sample image are extracted through steps S2110 to S2130, that is, the image blocks represented by P N and P P are obtained. Then the smallest circumscribed rectangle of each candidate image block is detected, and the size of the smallest circumscribed rectangle is adjusted so that the size of the circumscribed rectangle of each candidate image block is the same. Then calculate the LBP feature of the image area of the sample image where the circumscribed rectangle of each candidate image block is located. Finally, the LBP feature extracted from the candidate image block of the positive sample image and the LBP feature extracted from the candidate image block of the negative sample image are input into the SVM classifier for confrontation training, and the classification result of the confrontation training is obtained.
  • the SVM classifier can use a linear kernel function, and the error term penalty coefficient can be set to 1000 to enable probability estimation.
  • This embodiment also provides a product defect detection method.
  • the collected product image is input to the first-level classifier, and then the obtained classification result is input to the second-level classifier, and it is detected that the product is present in the product.
  • the defects further include S2410 ⁇ S2450:
  • S2410 Set one or more first-level classifiers, input the preset shape features of the candidate image blocks extracted from the product image into one or more first-level classifiers, and detect the target image block.
  • the preset shape features of each candidate image block can be sequentially input to the first-level classifier for classification processing. That is, first input the preset shape feature of a candidate image block into the first-level classifier, and after the first-level classifier finishes classifying the candidate image block, then input the preset shape feature of the second candidate image block into the first-level classifier.
  • the first-level classifier thus using the first-level classifier to complete the classification processing of all candidate image blocks.
  • the preset shape features of multiple candidate image blocks can be input to the corresponding first-level classifiers, and multiple first-level classifiers can be used to parallelize multiple candidate image blocks.
  • Classification processing For example, if ten candidate image blocks are extracted from a product image, ten first-level classifiers can be set, and the preset shape features of these ten candidate image blocks are input into the corresponding ten first-level classifiers. Ten first-level classifiers perform parallel classification processing on these ten candidate image blocks.
  • S2420 Set a plurality of second-level classifiers, input the obtained texture features of the target image block into corresponding second-level classifiers, and determine whether the product is in the product according to the multiple classification results output by each second classifier Whether there are defects.
  • the number of second-level classifiers can be set according to the number of target image blocks. For example, when there are ten target image blocks, ten second-level classifiers can be set.
  • a preferred solution is to perform the size of the minimum bounding rectangle of each target image block before inputting the obtained target image block to the second-level classifier Adjust to obtain the circumscribed rectangular frame of each target image block, and the circumscribed rectangular frame of each target image block has the same size.
  • the second-level classifier since the texture feature of the candidate image block is used as the training data, the size of the smallest bounding rectangle of the candidate image block is adjusted during the training phase.
  • the texture feature of the target image is used as the detection data during the product defect detection, the minimum circumscribed rectangular frame of the target image block is adjusted in size during the detection stage.
  • the classification results of all target image blocks are the first value, for example, when the classification results of all target image blocks are 0, it is determined that the product is a qualified product and does not have defects.
  • the classification result of one or more target image blocks is the second value, for example, when the classification result of one target image block is 1, it is determined that the product is defective.
  • a defect detection framework can be built based on the first-level classifier and the second-level classifier, and the defect detection framework includes a rough classification network and a fine classification network.
  • the rough classification network includes at least one first branch, and the first branch includes at least a shape feature extraction structure for performing preset shape feature extraction and a first-level classifier.
  • the fine classification network includes a judgment structure for logically judging the classification results and multiple second branches.
  • Each second branch includes a size adjustment structure for adjusting the size of the circumscribed rectangle, and a texture for extracting texture features.
  • Feature extraction structure and second-level classifier are examples of the fine classification network.
  • the rough classification network includes a first branch and n second branches.
  • the first branch uses the shape feature extraction structure to receive the candidate image block and perform preset shape feature extraction on the received candidate image block, and input the extracted preset shape feature to the first-level classifier for classification to detect the Whether the candidate image block can be classified as the target image block.
  • the first branch inputs the selected multiple target image blocks into the corresponding respective size adjustment structures. Assuming that the first branch filters out n target image blocks, then the first target image block is input to the first numbered 1 in Fig. 4
  • the size adjustment structure of the second branch (second branch-1) the second target image block is input into the size adjustment structure of the second branch (second branch-2) labeled 2 in FIG.
  • the target image blocks are input to the size adjustment structure of the second branch (second branch-n) labeled n in Figure 4, and the n size adjustment structures are used in parallel processing to determine the minimum bounding rectangle of the target image block.
  • the size is adjusted so that each target image block has a circumscribed rectangular frame of the same size.
  • the adjusted circumscribed rectangular frame is input to the corresponding texture feature extraction structure, and the multiple texture feature extraction structures detect the target area in the product image in parallel based on the received circumscribed rectangular frame.
  • the texture features of the target area are extracted and input to the corresponding second-level classifier.
  • the second-level classifier is used to classify the target image block. As shown in Figure 4, the second-level classifier includes multiple SVM classifiers, each The classification result of the SVM classifier includes 1 and 0. 1 indicates that the target image block has a product defect, and 0 indicates that the target image block does not have a product defect.
  • the judgment structure judges that the product is a qualified product without defects; when there is at least one classification result of the target image block in the n target image blocks When it is 1, the judgment structure judges that the product is unqualified and has product defects.
  • the number of target image blocks shown in FIG. 4 is the same as the number of second branches in the fine classification network, but in some embodiments, the number of second branches in the fine classification network may also be the same as the number of target image blocks.
  • the number of blocks is different, for example, the number of second branches is more than the number of target image blocks.
  • the product defect detection method of this embodiment is more effective for the broken defects of screen products. With only a small number of samples, the corresponding first-level classifier and second-level classifier can be quickly trained. Therefore, it can be used in the production line. In the early stage of production, it meets the needs of the production line and effectively improves the inspection efficiency of the production line.
  • FIG. 5 is a structural block diagram of a product defect detection device shown in an embodiment of the present invention. As shown in FIG. 5, the device in this embodiment includes:
  • the preprocessing unit is used to collect sample images of products, extract candidate image blocks that may have product defects from the sample images, and extract the preset shape features corresponding to the candidate image blocks and the texture features corresponding to the candidate image blocks; use the preset shape
  • the feature trains the first-level classifier to obtain the first-level classifier that can further filter out the target image blocks that may have product defects from the candidate image blocks; use the texture features to train the second-level classifier to get the correct Second-level classifier that identifies product defects;
  • the defect detection unit is used to input the preset shape features of the candidate image blocks extracted from the product image into the first-level classifier during product defect detection, and then input the obtained texture features of the target image block into the first classifier.
  • the secondary classifier detects defects in the product.
  • the sample image includes multiple positive sample images, one negative sample image, and one annotation image.
  • the preprocessing unit includes a first processing module configured to obtain the initial image blocks included in each positive sample image and the negative sample image; the area of the image block in the negative sample image is screened to be larger than the first preset The initial image block of the area value, the selected initial image block is used as the candidate image block extracted from the negative sample image; the position of each positive sample image that intersects the image block position on the labeled image is selected and the intersection area is greater than the second preset value The initial image block is selected as the candidate image block extracted from the positive sample image.
  • the first processing module specifically performs image binarization processing on each positive sample image and negative sample image to obtain a binarized image corresponding to the positive sample image and a binarized image corresponding to the negative sample image; Area detection is performed on the transformed image, and the initial image blocks included in each positive sample image and negative sample image are obtained.
  • the preprocessing unit includes a second processing module for extracting one or more of the four features of the area of the candidate image block, the area of the smallest circumscribed rectangle, and the eccentricity to the area ratio.
  • the preset shape feature of the candidate image block wherein, the eccentricity is equivalent to the ratio of the focal length of the candidate image block to the length of the major axis of an ellipse with the same second moment, and the area ratio is the number of pixels of the candidate image block and The ratio of the number of pixels of the smallest circumscribed rectangle of the candidate image block.
  • the preprocessing unit includes a third processing module for block detection of the image area of the sample image where the smallest circumscribed rectangular frame of the candidate image block is located; extracting the local binary pattern LBP feature of the image area , Regard the LBP feature as the texture feature.
  • the defect detection unit includes: one or more first-level classifiers, texture feature extraction modules, multiple second-level classifiers, and judgment modules; wherein,
  • the one or more first-level classifiers are used to classify the candidate image blocks according to the received preset shape features of the candidate image blocks extracted from the product image, and obtain the target image separated from the candidate image blocks Piece.
  • the texture feature extraction module is used to detect the target area of the product image where the smallest circumscribed rectangular frame of the target image block is located, extract the LBP feature of the target area, and input the LBP feature as the texture feature of the target image block into the second-level classifier.
  • the multiple second-level classifiers are used to classify each target image block according to the received texture feature of the target image block to obtain a classification result of each target image block.
  • the judgment module is used to judge whether there are defects in the product according to the classification result of each target image block; specifically, it is used to determine that the product is qualified when the classification results of all target image blocks are the first value The product has no defect; when the classification result of one or more target image blocks is the second value, it is determined that the product has a defect.
  • the defect detection unit further includes a size adjustment module, which is used to adjust the size of the smallest circumscribed rectangular frame of each target image block to obtain the circumscribed rectangular frame of each target image block.
  • the size of the circumscribed rectangle of the image block is the same.
  • the texture feature extraction module is used to detect the target area of the product image where the circumscribed rectangular frame of the target image block is located, extract the LBP feature of the target area, and input the LBP feature as the texture feature of the target image block into the second-level classifier .
  • Fig. 6 is a structural block diagram of a product defect detection system according to an embodiment of the present invention.
  • the virtual reality system includes a processor, and optionally an internal bus, a network interface, and a memory.
  • the memory may include memory, such as high-speed random access memory (Random-Access Memory, RAM), or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
  • RAM Random-Access Memory
  • non-volatile memory such as at least one disk memory.
  • the processor, network interface, and memory can be connected to each other through an internal bus.
  • the internal bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnection standard) bus, or an EISA (Extended) bus. Industry Standard Architecture, extended industry standard structure) bus, etc.
  • the bus can be divided into an address bus, a data bus, a control bus, and so on. For ease of presentation, only one bidirectional arrow is used to indicate in FIG. 6, but it does not mean that there is only one bus or one type of bus.
  • Memory used to store programs.
  • the program may include program code, and the program code includes computer executable instructions.
  • the memory may include memory and non-volatile memory, and provide instructions and data to the processor.
  • the processor reads the corresponding computer program from the non-volatile memory to the memory and then runs it to form a product defect detection device on a logical level.
  • the processor executes the program stored in the memory to implement the product defect detection method described above.
  • the method executed by the product defect detection apparatus disclosed in the embodiment shown in FIG. 6 of this specification can be applied to a processor or implemented by the processor.
  • the processor may be an integrated circuit chip with signal processing capabilities.
  • the steps of the product defect detection method described above can be completed by hardware integrated logic circuits in the processor or instructions in the form of software.
  • the above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (DSP), a dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • CPU central processing unit
  • NP Network Processor
  • DSP digital signal processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of this specification can be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of this specification can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above-mentioned product defect detection method in combination with its hardware.
  • the present invention also provides a computer-readable storage medium.
  • the computer-readable storage medium stores one or more computer programs, and the one or more computer programs include instructions that, when executed by a processor, can implement the product defect detection method described above.
  • words such as “first” and “second” are used to distinguish the same or similar items with basically the same function and effect. Personnel can understand that the words “first” and “second” do not limit the quantity and order of execution.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Pathology (AREA)
  • Quality & Reliability (AREA)
  • Immunology (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Signal Processing (AREA)
  • Biochemistry (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种产品缺陷检测方法、装置与系统。该方法包括：采集产品的样本图像，从样本图像中提取可能存在产品缺陷的候选图像块，并提取候选图像块对应的预设形状特征和候选图像块对应的纹理特征（S2100）；利用预设形状特征对第一级分类器进行训练，得到能够从候选图像块中进一步筛选出可能存在产品缺陷的目标图像块的第一级分类器（S2200）；利用纹理特征对第二级分类器进行训练，得到能够正确识别出产品缺陷的第二级分类器（S2300）；在进行产品缺陷检测时，将从产品图像中提取的候选图像块的预设形状特征输入第一级分类器后，将得到的目标图像块的纹理特征再输入至第二级分类器，检测出产品中存在的缺陷（S2400）。

Description

一种产品缺陷检测方法、装置与系统 技术领域
本发明涉及一种产品缺陷检测方法、装置与系统。
发明背景
在传统的精密制造行业，产品的缺陷检测一般通过人工检验来完成。在一般的制造工厂中，检验人力占比将近30％，由于人力需求大，招聘比较困难；而且人力检验的劳动强度大，容易因疲劳造成检验质量的波动。因此，检验质量稳定、效果一致、不受人为因素影响的机器自动化检测方案必将受传统精密制造行业的青睐。
发明内容
本发明实施例的目的在于提供了一种产品缺陷检测方法、装置与系统。
一方面，本发明实施例提供了一种产品缺陷检测方法，包括：
采集产品的样本图像，从样本图像中提取可能存在产品缺陷的候选图像块，并提取候选图像块对应的预设形状特征和候选图像块对应的纹理特征；
利用预设形状特征对第一级分类器进行训练，得到能够从候选图像块中进一步筛选出可能存在产品缺陷的目标图像块的第一级分类器；
利用纹理特征对第二级分类器进行训练，得到能够正确识别出产品缺陷的第二级分类器；
在进行产品缺陷检测时，将从产品图像中提取的候选图像块的预设形状特征输入第一级分类器后，将得到的目标图像块的纹理特征再输入至第二级分类器，检测出产品中存在的缺陷。
再一方面，本发明实施例提供了一种产品缺陷检测装置，包括：
预处理单元，用于采集产品的样本图像，从样本图像中提取可能存在产品缺陷的候选图像块，并提取候选图像块对应的预设形状特征和候选图像块对应的纹理特征；利用预设形状特征对第一级分类器进行训练，得到能够从候选图像块中进一步筛选出可能存在产品缺陷的目标图像块的第一级分类器；利用纹理特征对第二级分类器进行训练，得到能够正确识别出产品缺陷的第二级分类器；
缺陷检测单元，用于在在进行产品缺陷检测时，将从产品图像中提取的候 选图像块的预设形状特征输入第一级分类器后，将得到的目标图像块的纹理特征再输入至第二级分类器，检测出产品中存在的缺陷。
又一方面，本发明实施例提供了一种产品缺陷检测系统，包括：存储器和处理器；
存储器，存储计算机可执行指令；
处理器，计算机可执行指令在被执行时使处理器执行产品缺陷检测方法。
又一方面，本发明实施例提供了一种计算机可读存储介质，计算机可读存储介质上存储有一个或多个计算机程序，一个或多个计算机程序被执行时实现产品缺陷检测方法。
本发明至少取得以下技术效果：本发明预先通过样本图像训练出第一级分类器和第二级分类器，在对产品缺陷进行检测时，通过从产品图像中提取出可能存在产品缺陷的候选图像块完成图像块的第一次筛选，并利用第一级分类器对产品图像的候选图像块进行第二次筛选，通过第一筛选和第二次筛选完成对图像块的粗分类操作，之后再利用第二级分类器对目标图像块进行第三次筛选，通过第三次筛选完成对图像块的细分类操作，通过上述粗分类操作与细分类操作的结合即可筛选出存在产品缺陷的目标图像块，即可根据存在产品缺陷的目标图像块快速准确的检测出产品存在的缺陷。
附图简要说明
为了更清楚地说明本发明实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍。应当理解，以下附图仅示出了本发明的某些实施例，因此不应被看作是对范围的限定。对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1为本发明实施例示出的产品缺陷检测系统的硬件配置的框图；
图2为本发明实施例示出的产品缺陷检测方法流程图；
图3为本发明实施例示出的第二级分类器的训练过程示意图；
图4为本发明实施例示出的缺陷检测框架示意图；
图5为本发明实施例示出的产品缺陷检测装置的结构框图；
图6为本发明实施例示出的产品缺陷检测系统的结构框图。
具体实施方式
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
<实施例一>
图1是根据本发明实施例的产品缺陷检测系统100的硬件配置的框图。
如图1所示，产品缺陷检测系统100包括图像采集装置1000和产品缺陷检测装置2000。
图像采集装置1000用于采集产品图像，并将采集到的产品图像提供至产品缺陷检测装置2000。
该图像采集装置1000可以是能够进行拍照的任意成像设备，例如摄像头等。
产品缺陷检测装置2000可以是任意的电子设备，例如PC机、笔记本电脑、服务器等。
在本实施例中，参照图1所示，产品缺陷检测装置2000可以包括处理器2100、存储器2200、接口装置2300、通信装置2400、显示装置2500、输入装置2600、扬声器2700、麦克风2800等等。
处理器2100可以是移动版处理器。存储器2200例如包括ROM(只读存储器)、RAM(随机存取存储器)、诸如硬盘的非易失性存储器等。接口装置2300例如包括USB接口、耳机接口等。通信装置2400例如能够进行有线或无线通信，通信装置2400可以包括短距离通信装置，例如是基于Hilink协议、WiFi(IEEE 802.11协议)、Mesh、蓝牙、ZigBee、Thread、Z-Wave、NFC、UWB、LiFi等短距离无线通信协议进行短距离无线通信的任意装置，通信装置2400也可以包括远程通信装置，例如是进行WLAN、GPRS、2G/3G/4G/5G远程通信的任意装置。显示装置2500例如是液晶显示屏、触摸显示屏等，显示装置2500用于显示图 像采集装置1000采集的产品图像。输入装置2600例如可以包括触摸屏、键盘等。用户可以通过扬声器2700和麦克风2800输入/输出语音信息。
在该实施例中，产品缺陷检测装置2000的存储器2200用于存储指令，该指令用于控制处理器2100进行操作以至少执行根据本发明任意实施例的产品缺陷检测方法。技术人员可以根据本发明所公开方案设计指令。指令如何控制处理器进行操作，这是本领域公知，故在此不再详细描述。
尽管在图1中示出了产品缺陷检测装置2000的多个装置，但是，本发明可以仅涉及其中的部分装置，例如，产品缺陷检测装置2000只涉及存储器2200、处理器2100和显示装置2500。
在本实施例中，图像采集装置1000用于采集产品图像提供至产品缺陷检测装置2000，产品缺陷检测装置2000则基于该图像实施根据本发明任意实施例的产品缺陷检测方法。
应当理解的是，尽管图1仅示出一个图像采集装置1000和一个产品缺陷检测装置2000，但不意味着限制各自的数量，产品缺陷检测系统100中可以包含多个图像采集装置1000和/或产品缺陷检测装置2000。
<实施例二>
在精密产品的制造过程中，工艺的不稳定、机械定位精度不够高以及厂房内的环境因素等经常会使生产出来的产品具有各种形态的缺陷。针对工业制造中经常出现的丝网状产品，破损缺陷是其主要缺陷，如果出现丝网破损，会严重影响产品的性能。因此，破损类缺陷应尽可能做到零漏检，但是在产品的制造前期，不良样本通常很少，无法利用当前流行的基于深度学习的分类算法准确的检测出不良产品。
针对上述丝网状产品的破损缺陷检测问题，本申请实施例的整体构思为：先根据丝网状产品图像产生可能存在产品缺陷的图像块，基于图像块的形状特征对这些图像块进行粗分类，得到可能存在缺陷的图像块，再对这些可能存在缺陷的图像块提取纹理特征，最后基于纹理特征利用分类器对其进行细分类，以判别丝网状产品中包括的缺陷。本实施例以丝网状产品为例来说明具体的技术细节，但本实施例的技术方案不局限于丝网状产品的检测场景。
图2为本发明实施例示出的产品缺陷检测方法流程图，如图2所示，本实施例的方法包括：
S2100，采集产品的样本图像，从样本图像中提取可能存在产品缺陷的候选 图像块，并提取候选图像块对应的预设形状特征和候选图像块对应的纹理特征。
本步骤通过从样本图像中提取可能存在产品缺陷的候选图像块完成图像块的首次筛选。本实施例通过对产品的样本图像进行区域检测，可以检测到多个初始图像块。例如，512*512像素大小的样本图像中，对丝网状产品，若产品网格对应的像素大小为100*100，那么通过区域检测，可以得到约36个初始图像块。由于初始图像块中可能包括大量不存在缺陷的图像块，本步骤可以基于正常图像块与产品缺陷所在图像块在面积上有较大差异这一特征，基于图像块面积特征从初始图像块中提取出候选图像块，所提取出来的候选图像块中包括可能存在缺陷的图像块。
S2200，利用预设形状特征对第一级分类器进行训练，得到能够从候选图像块中进一步筛选出可能存在产品缺陷的目标图像块的第一级分类器。
由于候选图像块是基于面积特征从初始图像块筛选而来，因此候选图像块中存在较多的不存在产品缺陷的图像块。基于此，本实施例基于候选图像块的预设形状特征训练出第一级分类器，以在产品缺陷检测时，通过执行粗分类操作筛选出存在产品缺陷概率比较大的目标图像块，减少后续细分类操作所需处理的图像块数量，降低缺陷检测的整体耗时。其中，粗分类操作包括从样本图像中提取可能存在产品缺陷的候选图像块的第一次图像块筛选和利用第一级分类器对候选图像块进行分类的第二次图像块筛选，细分类操作包括利用第二级分类器对目标图像块进行分类的第三次图像块筛选。
S2300，利用纹理特征对第二级分类器进行训练，得到能够正确识别出产品缺陷的第二级分类器。
本实施例是基于候选图像块的形状特征训练第一级分类器，通过第一级分类器分出的目标图像块中有可能包括不存在产品缺陷的图像块。因此，本步骤利用候选图像块的纹理特征对第二级分类器进行训练，这样训练好的第二级分类器在对目标图像块进行分类时，能够准确的将目标图像块中存在产品缺陷的图像块和不存在产品缺陷的图像块进行分类，从而筛选出存在产品缺陷的目标图像块，完成产品缺陷的检测。
S2400，在进行产品缺陷检测时，将从产品图像中提取的候选图像块的预设形状特征输入第一级分类器后，将得到的目标图像块的纹理特征再输入至第二级分类器，检测出产品中存在的缺陷。
本实施例预先通过样本图像训练出第一级分类器和第二级分类器，在对产 品缺陷进行检测时，通过从产品图像中提取出可能存在产品缺陷的候选图像块完成图像块的第一次筛选，并利用第一级分类器对产品图像的候选图像块进行第二次筛选，通过第一筛选和第二次筛选完成对图像块的粗分类操作，之后再利用第二级分类器对目标图像块进行第三次筛选，通过第三次筛选完成对图像块的细分类操作，通过上述粗分类操作与细分类操作的结合即可筛选出存在产品缺陷的目标图像块，即可根据存在产品缺陷的目标图像块快速准确的检测出产品存在的缺陷。
<实施例三>
本申请实施例还提供了一种产品缺陷检测方法，在本实施例中，样本图像包括多个正样本图像、一个负样本图像和一个标注图像，正样本图像为存在产品缺陷的不合格产品图像，负样本图像为不存在产品缺陷的合格产品图像，标注图像为产品缺陷所在位置的像素值为第一数值(例如为255)，非缺陷所在位置的像素值为第二数值(例如为0)的二值图像。
上述步骤S2100中从样本图像中提取可能存在产品缺陷的候选图像块包括S2110～S2130：
S2110，获取每个正样本图像与负样本图像包括的初始图像块。
在一个实施例中，可以通过下述方法提取初始图像块：
对每个正样本图像与负样本图像分别进行图像二值化处理，得到正样本图像对应的二值化图像和负样本图像对应的二值化图像；
对二值化图像进行区域检测，例如利用图像检测工具skimage.measure中的区域检测函数regionprops对二值化图像进行区域检测，得到每个正样本图像与负样本图像包括的初始图像块。
S2120，筛选负样本图像中图像块面积大于第一预设面积值的初始图像块，将筛选出的初始图像块作为从负样本图像中提取的所述候选图像块。
第一预设面积值可以根据产品缺陷特征进行设定。例如，某种丝网状产品的破损缺陷所在的网格面积一般大于100个像素点，那么可以将从负样本图像中检测到面积大于100个像素点的初始图像块作为负样本图像的候选图像块，这里用P N表示负样本图像的候选图像块。
S2130，筛选每个正样本图像中与标注图像上图像块位置相交且相交面积大于第二预设值的初始图像块，将筛选出的初始图像块作为从正样本图像中提取到的候选图像块。
第二预设值可以根据实际需求设定，第二预设值越大，从正样本图像的初始图像块中筛选出的候选图像块存在产品缺陷的可能性就越大，筛选出的候选图像块的数量越少。反之，第二预设值越小，从正样本图像的初始图像块中筛选的候选图像块存在产品缺陷的可能性就越小，筛选出的候选图像块的数量越多。
例如，对于正样本图像的初始图像块，将与标注图像上的缺陷所在图像块相交，且相交面积大于该初始图像块面积50％的初始图像块作为正样本图像的候选图像块，这里用P P表示正样本图像的候选图像块。
通过上述步骤S2110～S2130即完成了图像块的第一次筛选，在筛选出候选图像块之后，即可通过下述方案一提取候选图像块对应的预设形状特征，利用提取到的预设形状特征训练第一级分类器；以及通过下述方案二提取候选图像块对应的纹理特征，利用提取到的纹理特征训练第二级分类器。
方案一第一级分类器
通过下述步骤S11～S12提取候选图像块对应的预设形状特征，利用提取到的预设形状特征训练第一级分类器。
S11，提取候选图像块的区域面积、最小外接矩形面积、偏心率与面积比这四个特征中的一个或多个作为预设形状特征。
其中，偏心率为把候选图像块等价成具有相同二阶矩的椭圆的焦距与长轴长度的比值。即把候选图像块等价成具有相同二阶矩的椭圆，计算该椭圆的焦距与长轴长度的比值，该比值即为候选图像块的偏心率。由于每个候选图像块的形状可能不同，因此每个候选图像块等价成的椭圆的二阶矩也可能不同，相应的每个候选图像块对应的偏心率也可能不同。假设存在50个候选图像块，每个候选图像块都可以等价为一个椭圆，这50个椭圆可能具有不同的二阶矩。
面积比为候选图像块的像素数量与候选图像块的最小外接矩形的像素数量的比值。
方案一中的候选图像块是指P N与P P表示的图像块，以P P表示的从正样本图像中筛选出的候选图像块为例，一个优选方案是将P P的区域面积、P P的最小外接矩形面积、P P的偏心率和P P的面积比这四个特征作为P P候选图像块的预设形状特征。
S12，设定第一级分类器的初始参数，利用从负样本图像中提取出的候选图像块的预设形状特征与从正样本图像中提取的候选图像块的预设形状特征对第 一级分类器进行对抗训练，基于对抗训练结果优化第一级分类器的分类器参数，直至对抗训练结果符合预设值。例如，在第一级分类器的分类正确率达到95％时停止对抗训练，将此时的分类器参数作为该第一级分类器的最终参数。
第一级分类器包括但不限于SVM(支持向量机)分类器、K近邻分类器、决策树分类器(包含XGBClassifier)等。由于XGBClassifier具有运行速度快、准确率较高的特征，本实施例可以采用XGBClassifier作为第一级分类器。训练时，可以采用gbtree树模型作为基分类器，相应的，需要设置gbtree树模型的树深度初始值、迭代更新步长初始值、决策树个数初始值、惩罚项系数初始值等。
方案二第二级分类器
通过下述步骤S21～S22提取候选图像块对应的纹理特征，利用提取到的纹理特征训练第二级分类器。
S21，检测候选图像块(即P N与P P表示的图像块)的最小外接矩形框所在的样本图像的图像区域，提取图像区域的LBP(Local Binary Patterns，局部二值模式)特征，将LBP特征作为纹理特征。
在检测每个候选图像块的最小外接矩形时，由于每个候选图像块的形态不同，相应的每个候选图像块的最小外接矩形的大小也不相同，本步骤的一个优选方案是，将每个候选图像块的最小外接矩形框进行尺寸调整，得到调整后的每个候选图像块的外接矩形框，调整后的每个候选图像块的外接矩形框尺寸相同。在得到具有相同尺寸的外接矩形框之后，检测每个候选图像块的外接矩形所在的样本图像的图像区域，提取该图像区域的LBP特征作为该候选图像块的纹理特征。
例如，在得到候选图像块patch1、patch2与patch3之后，这三个候选图像块patch1、patch2与patch3的最小外接矩形框分别为box1’、box2’与box3’。调整最小外接矩形框box1’、box2’与box3’的尺寸分别为box1、box2与box3的尺寸，使得调整后的外接矩形框box1、box2与box3尺寸相同。这样，在提取LBP特征时，检测候选图像块patch1的外接矩形框box1所在的样本图像Image的图像区域I1，提取该区域图像I1的LBP特征为该候选图像块patch1的纹理特征。与候选图像块patch1的纹理特征提取方法类似，检测候选图像块patch2(或patch3)的外接矩形框box2(或box3)所在的样本图像Image的图像区域I2(或I3)，提取该区域图像I2(或I3)的LBP特征为该候选图像块patch2(或patch3) 的纹理特征。
S22，设定第二级分类器的初始参数，利用从负样本图像中提取出的候选图像块的纹理特征与从正样本图像中提取的候选图像块的纹理特征对第二级分类器进行对抗训练，基于对抗训练结果优化第二级分类器的分类器参数，直至对抗训练结果符合预设值。例如在分类正确率达到95％时停止对抗训练。其中，第二级分类器可以采用SVM分类器实现。
如图3所示，训练第二级分类器的过程包括：
先通过步骤S2110～S2130提取样本图像的候选图像块，即获得P N与P P表示的图像块。然后检测每个候选图像块的最小外接矩形，并调整最小外接矩形的尺寸，使每个候选图像块的外接矩形尺寸相同。接着计算每个候选图像块的外接矩形所在样本图像的图像区域的LBP特征。最后，将从正样本图像的候选图像块提取的LBP特征和从负样本图像的候选图像块提取的LBP特征输入至SVM分类器中进行对抗训练，获得对抗训练的分类结果。
其中，SVM分类器可以采用线性核函数，错误项惩罚系数可以设置为1000，启用概率估计。
<实施例四>
本实施例还提供一种产品缺陷检测方法。在本实施例中，上述步骤S2400中在进行产品缺陷检测时，将采集到的产品图像输入第一级分类器后，将得到的分类结果再输入至第二级分类器，检测出产品中存在的缺陷进一步包括S2410～S2450：
S2410，设置一个或多个第一级分类器，将从产品图像中提取到的候选图像块的预设形状特征输入一个或多个第一级分类器，检测得到目标图像块。
在设置一个第一级分类器时，可以依次将每个候选图像块的预设形状特征依次输入该第一级分类器进行分类处理。即先将一个候选图像块的预设形状特征输入该第一级分类器，在第一级分类器对该候选图像块分类结束后，再将第二个候选图像块的预设形状特征输入第一级分类器，由此利用第一级分类器完成对所有候选图像块的分类处理。
在设置多个第一级分类器时，可以分别将多个候选图像块的预设形状特征输入到相应的第一级分类器，利用多个第一级分类器对多个候选图像块进行并行分类处理。例如，从产品图像中提取到十个候选图像块，则可以设置十个第一级分类器，将这十个候选图像块预设形状特征输入到相应的十个第一级分类 器中，这十个第一级分类器对这十个候选图像块进行并行分类处理。
S2420，设置多个第二级分类器，将得到的目标图像块的纹理特征分别输入至对应的各第二级分类器，根据各第二分类器输出的多个分类结果，判断所述产品中是否存在缺陷。
本实施例可以根据目标图像块的数量设置第二级分类器的数量，例如有十个目标图像块时，可以设置十个第二级分类器。为了使多个第二级分类器对目标图像块进行并行处理，一个优选方案是，将得到的目标图像块输入至第二级分类器之前，将每个目标图像块的最小外接矩形框进行尺寸调整，得到每个目标图像块的外接矩形框，每个目标图像块的外接矩形框尺寸相同。
需要说明的是，本申请在对第二级分类器进行训练时，由于是将候选图像块的纹理特征作为训练数据，因此，在训练阶段是对候选图像块的最小外接矩形框进行尺寸调整。相应的，由于在产品缺陷检测时，是将目标图像的纹理特征作为检测数据，因此，在检测阶段是对目标图像块的最小外接矩形框进行尺寸调整。
得到尺寸相同外接矩形框之后，检测目标图像块的外接矩形框所在产品图像的目标区域，计算该目标区域的LBP特征作为该目标图像块的纹理特征，将该纹理特征输入第二级分类器进行分类处理。
当所有目标图像块的分类结果均为第一数值时，例如所有目标图像块的分类结果都为0时，确定该产品为合格产品，不存在缺陷。当存在一个或一个以上的目标图像块的分类结果为第二数值时，例如当存在一个目标图像块的分类结果为1时，确定该产品存在缺陷。
在本实施例的一个应用场景中，可以基于第一级分类器和第二级分类器搭建缺陷检测框架，该缺陷检测框架包括粗分类网络与细分类网络。
粗分类网络中包括至少一个第一分支，第一分支至少包括用于进行预设形状特征提取的形状特征提取结构和第一级分类器。
细分类网络中包括用于对分类结果进行逻辑判断的判断结构和多个第二分支，每个第二分支包括用于进行外接矩形框尺寸调整的尺寸调整结构、用于进行纹理特征提取的纹理特征提取结构和第二级分类器。
参考图4，在图4示出的缺陷检测框架中，粗分类网络中包括一个第一分支和n个第二分支。
其中，第一分支利用形状特征提取结构接收候选图像块并对接收到的候选 图像块进行预设形状特征提取，将提取到的预设形状特征输入给第一级分类器进行分类，以检测该候选图像块是否可以归类为目标图像块。
第一分支将筛选出的多个目标图像块输入相应的各个尺寸调整结构，假设第一分支筛选出n个目标图像块，则将第1个目标图像块输入到图4中标号为1的第二分支(第二分支-1)的尺寸调整结构，将第2个目标图像块输入到图4中标号为2的第二分支(第二分支-2)的尺寸调整结构，依此将第n个目标图像块输入到图4中标号为n的第二分支(第二分支-n)的尺寸调整结构，采用并行处理的方式利用这n个尺寸调整结构对目标图像块的最小外接矩形框的尺寸进行调整，使得每个目标图像块具有相同尺寸的外接矩形框。在尺寸调整结构完成尺寸调整后，将调整后的外接矩形框输入给相应的纹理特征提取结构，多个纹理特征提取结构基于接收到的外接矩形框采用并行方式在产品图像中检测出目标区域，提取目标区域的纹理特征并输入给相应的第二级分类器，利用第二级分类器对目标图像块进行分类，如图4所示，第二级分类器包括多个SVM分类器，每个SVM分类器的分类结果包括1与0，1表示该目标图像块存在产品缺陷，0表示该目标图像块不存在产品缺陷。
在得到对每个目标图像块的分类结果后，通过细分类网络的判断结构判断产品是否存在缺陷。
继续参考图4，当这n个目标图像块的分类结果都为0时，判断结构判断该产品为合格产品，不存在缺陷；当这n个目标图像块中存在至少一个目标图像块的分类结果为1时，判断结构判断该产品为不合格产品，存在产品缺陷。
需要说明的是，图4中示出的目标图像块的数量与细分类网络中第二分支的数量相同，但在一些本实施例中，细分类网络中第二分支的数量也可以与目标图像块的数量不同，例如第二分支的数量也行多于目标图像块的数量。
通过上述步骤S2410～S2420即可快速准确的检测出产品存在的缺陷。本实施例的产品缺陷检测方法对丝网类产品的破损缺陷比较有效，在只有少量样本的情况下即可快速训练出对应的第一级分类器与第二级分类器，因此，可以在生产线的生产初期，满足产线需求，有效提高生产线的检测效率。
<实施例五>
图5为本发明实施例示出的产品缺陷检测装置的结构框图，如图5所示，本实施例的装置包括：
预处理单元，用于采集产品的样本图像，从样本图像中提取可能存在产品 缺陷的候选图像块，并提取候选图像块对应的预设形状特征和候选图像块对应的纹理特征；利用预设形状特征对第一级分类器进行训练，得到能够从候选图像块中进一步筛选出可能存在产品缺陷的目标图像块的第一级分类器；利用纹理特征对第二级分类器进行训练，得到能够正确识别出产品缺陷的第二级分类器；
缺陷检测单元，用于在在进行产品缺陷检测时，将从产品图像中提取的候选图像块的预设形状特征输入第一级分类器后，将得到的目标图像块的纹理特征再输入至第二级分类器，检测出产品中存在的缺陷。
在一些实施例中，样本图像包括多个正样本图像、一个负样本图像和一个标注图像。相应的，预处理单元包括第一处理模块，该第一处理模块用于获取每个正样本图像与所述负样本图像包括的初始图像块；筛选负样本图像中图像块面积大于第一预设面积值的初始图像块，将筛选出的初始图像块作为从负样本图像中提取的候选图像块；筛选每个正样本图像中与标注图像上图像块位置相交且相交面积大于第二预设值的初始图像块，将筛选出的初始图像块作为从正样本图像中提取到的候选图像块。
其中，第一处理模块具体是对每个正样本图像与负样本图像分别进行图像二值化处理，得到正样本图像对应的二值化图像和负样本图像对应的二值化图像；对二值化图像进行区域检测，得到每个正样本图像与负样本图像包括的初始图像块。
在一些实施例中，预处理单元包括第二处理模块，该第二处理模块用于提取候选图像块的区域面积、最小外接矩形面积、偏心率与面积比这四个特征中的一个或多个作为所述候选图像块的预设形状特征；其中，偏心率为把候选图像块等价成具有相同二阶矩的椭圆的焦距与长轴长度的比值，面积比为候选图像块的像素数量与候选图像块的最小外接矩形的像素数量的比值。
在一些实施例中，预处理单元包括第三处理模块，该第三处理模用于块检测候选图像块的最小外接矩形框所在的样本图像的图像区域；提取图像区域的局部二值模式LBP特征，将LBP特征作为纹理特征。
在一些实施例中，缺陷检测单元包括：一个或多个第一级分类器、纹理特征提取模块、多个第二级分类器和判断模块；其中，
该一个或多个第一级分类器，用于根据接收到的从产品图像中提取到的候选图像块的预设形状特征对候选图像块进行分类，获得从候选图像块中分出的 目标图像块。
该纹理特征提取模块，用于检测目标图像块的最小外接矩形框所在的产品图像的目标区域，提取目标区域的LBP特征，将LBP特征作为目标图像块的纹理特征输入第二级分类器。
该多个第二级分类器，用于根据接收到的目标图像块的纹理特征对每个目标图像块进行分类，得到每个目标图像块的分类结果。
该判断模块，用于根据每个目标图像块的分类结果，判断所述产品中是否存在缺陷；具体是用于当所有目标图像块的分类结果均为第一数值时，确定所述产品为合格产品，不存在缺陷；当存在一个或一个以上的目标图像块的分类结果为第二数值时，确定所述产品存在缺陷。
在一些实施例中，缺陷检测单元还包括尺寸调整模块，该尺寸调整模块用于将每个目标图像块的最小外接矩形框进行尺寸调整，得到每个目标图像块的外接矩形框，每个目标图像块的外接矩形框尺寸相同。
相应的，该纹理特征提取模块，用于检测目标图像块的外接矩形框所在的产品图像的目标区域，提取目标区域的LBP特征，将LBP特征作为目标图像块的纹理特征输入第二级分类器。
本装置实施例中各单元和模块的具体执行方式可以参见本方法实施例的相关内容，在此不再赘述。
<实施例六>
图6为本发明实施例示出的产品缺陷检测系统的结构框图，如图6所示，在硬件层面，该虚拟现实系统包括处理器，可选地还包括内部总线、网络接口、存储器。其中，存储器可能包含内存，例如高速随机存取存储器(Random-Access Memory，RAM)，也可能还包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器等。
处理器、网络接口和存储器可以通过内部总线相互连接，该内部总线可以是ISA(Industry Standard Architecture，工业标准体系结构)总线、PCI(Peripheral Component Interconnect，外设部件互连标准)总线或EISA(Extended Industry Standard Architecture，扩展工业标准结构)总线等。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示，图6中仅用一个双向箭头表示，但并不表示仅有一根总线或一种类型的总线。
存储器，用于存放程序。具体地，程序可以包括程序代码，所述程序代码 包括计算机可执行指令。存储器可以包括内存和非易失性存储器，并向处理器提供指令和数据。
处理器从非易失性存储器中读取对应的计算机程序到内存中然后运行，在逻辑层面上形成产品缺陷检测装置。处理器，执行存储器所存放的程序实现如上文描述的产品缺陷检测方法。
上述如本说明书图6所示实施例揭示的产品缺陷检测装置执行的方法可以应用于处理器中，或者由处理器实现。处理器可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上文描述的产品缺陷检测方法的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，CPU)、网络处理器(Network Processor，NP)等；还可以是数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本说明书实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本说明书实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述产品缺陷检测方法的步骤。
本发明还提供了一种计算机可读存储介质。
该计算机可读存储介质存储一个或多个计算机程序，该一个或多个计算机程序包括指令，该指令当被处理器执行时，能够实现上文描述的产品缺陷检测方法。
为了便于清楚描述本发明实施例的技术方案，在发明的实施例中，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分，本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定。
以上所述，仅为本发明的具体实施方式，在本发明的上述教导下，本领域技术人员可以在上述实施例的基础上进行其他的改进或变形。本领域技术人员 应该明白，上述的具体描述只是更好的解释本发明的目的，本发明的保护范围应以权利要求的保护范围为准

Claims (15)

  1. 一种产品缺陷检测方法，其特征在于，包括：
    采集产品的样本图像，从样本图像中提取可能存在产品缺陷的候选图像块，并提取所述候选图像块对应的预设形状特征和所述候选图像块对应的纹理特征；
    利用所述预设形状特征对第一级分类器进行训练，得到能够从候选图像块中进一步筛选出可能存在产品缺陷的目标图像块的第一级分类器；
    利用所述纹理特征对第二级分类器进行训练，得到能够正确识别出产品缺陷的第二级分类器；
    在进行产品缺陷检测时，将从产品图像中提取的候选图像块的预设形状特征输入第一级分类器后，将得到的目标图像块的纹理特征再输入至第二级分类器，检测出产品中存在的缺陷。
  2. 根据权利要求1所述的方法，其特征在于，所述样本图像包括多个正样本图像、一个负样本图像和一个标注图像，所述从样本图像中提取可能存在产品缺陷的候选图像块，包括：
    获取每个正样本图像与所述负样本图像包括的初始图像块；
    筛选所述负样本图像中图像块面积大于第一预设面积值的初始图像块，将筛选出的初始图像块作为从负样本图像中提取的所述候选图像块；
    筛选每个正样本图像中与所述标注图像上图像块位置相交且相交面积大于第二预设值的初始图像块，将筛选出的初始图像块作为从所述正样本图像中提取到的所述候选图像块。
  3. 根据权利要求2所述的方法，其特征在于，对正样本图像的一初始图像块，设置所述第二预设值为该初始图像块面积的50％。
  4. 根据权利要求1所述的方法，其特征在于，提取所述候选图像块对应的预设形状特征，包括：
    提取所述候选图像块的区域面积、最小外接矩形面积、偏心率与面积比这四个特征中的一个或多个；
    其中，所述偏心率为把所述候选图像块等价成具有相同二阶矩的椭圆的焦距与长轴长度的比值，所述面积比为所述候选图像块的像素数量与所述候选图像块的最小外接矩形的像素数量的比值。
  5. 根据权利要求2所述的方法，其特征在于，所述利用所述预设形状特征 对第一级分类器进行训练，得到能够从候选图像块中进一步筛选出可能存在产品缺陷的目标图像块的第一级分类器，包括：
    设定第一级分类器的初始参数；
    利用从负样本图像中提取出的候选图像块的预设形状特征与从正样本图像中提取的候选图像块的预设形状特征对所述第一级分类器进行对抗训练；
    基于对抗训练结果优化所述第一级分类器的分类器参数，直至所述对抗训练结果符合预设值。
  6. 根据权利要求1所述的方法，其特征在于，提取所述候选图像块对应的纹理特征包括：
    检测所述候选图像块的最小外接矩形框所在的样本图像的图像区域；
    提取所述图像区域的局部二值模式LBP特征，将所述LBP特征作为所述纹理特征。
  7. 根据权利要求1所述的方法，其特征在于，在进行产品缺陷检测时，将从产品图像中提取的候选图像块的预设形状特征输入第一级分类器后，将得到的目标图像块的纹理特征再输入至第二级分类器，检测出产品中存在的缺陷，包括：
    设置一个或多个第一级分类器，将从所述产品图像中提取到的候选图像块的预设形状特征输入所述一个或多个第一级分类器，检测得到目标图像块；
    设置多个第二级分类器，将得到的目标图像块的纹理特征分别输入至对应的各第二级分类器，根据各第二分类器输出的多个分类结果，判断所述产品中是否存在缺陷。
  8. 根据权利要求1所述的方法，其特征在于，将得到的目标图像块输入至第二级分类器之前，所述方法还包括：
    将每个目标图像块的最小外接矩形框进行尺寸调整，得到调整后的每个目标图像块的外接矩形框，调整后的每个目标图像块的外接矩形框尺寸相同。
  9. 根据权利要求7所述的方法，其特征在于，所述根据各第二分类器输出的多个分类结果，判断所述产品中是否存在缺陷，包括：
    当所有目标图像块的分类结果均为第一数值时，确定所述产品为合格产品，不存在缺陷；当存在一个或一个以上的目标图像块的分类结果为第二数值时，确定所述产品存在缺陷。
  10. 根据权利要求2所述的方法，其特征在于，所述获取每个正样本图像与所述负样本图像包括的初始图像块，包括：
    对每个正样本图像与所述负样本图像分别进行图像二值化处理，得到所述正样本图像对应的二值化图像和所述负样本图像对应的二值化图像；
    对所述二值化图像进行区域检测，得到每个正样本图像与所述负样本图像包括的初始图像块。
  11. 一种产品缺陷检测装置，包括：
    预处理单元，用于采集产品的样本图像，从样本图像中提取可能存在产品缺陷的候选图像块，并提取所述候选图像块对应的预设形状特征和所述候选图像块对应的纹理特征；利用所述预设形状特征对第一级分类器进行训练，得到能够从候选图像块中进一步筛选出可能存在产品缺陷的目标图像块的第一级分类器；利用所述纹理特征对第二级分类器进行训练，得到能够正确识别出产品缺陷的第二级分类器；
    缺陷检测单元，用于在进行产品缺陷检测时，将从产品图像中提取的候选图像块的预设形状特征输入第一级分类器后，将得到的目标图像块的纹理特征再输入至第二级分类器，检测出产品中存在的缺陷。
  12. 根据权利要求11所述的装置，其中，
    样本图像包括多个正样本图像、一个负样本图像和一个标注图像，预处理单元包括第一处理模块，
    该第一处理模块，用于获取每个正样本图像与所述负样本图像包括的初始图像块；筛选负样本图像中图像块面积大于第一预设面积值的初始图像块，将筛选出的初始图像块作为从负样本图像中提取的候选图像块；筛选每个正样本图像中与标注图像上图像块位置相交且相交面积大于第二预设值的初始图像块，将筛选出的初始图像块作为从正样本图像中提取到的候选图像块。
  13. 根据权利要求11所述的装置，其中，
    预处理单元还包括第二处理模块，该第二处理模块用于提取候选图像块的区域面积、最小外接矩形面积、偏心率与面积比这四个特征中的一个或多个作为所述候选图像块的预设形状特征；其中，偏心率为把候选图像块等价成具有相同二阶矩的椭圆的焦距与长轴长度的比值，面积比为候选图像块的像素数量与候选图像块的最小外接矩形的像素数量的比值。
  14. 根据权利要求11所述的装置，其中，
    预处理单元还包括第三处理模块，该第三处理模块，用于检测候选图像块的最小外接矩形框所在的样本图像的图像区域；提取图像区域的局部二值模式LBP特征，将LBP特征作为纹理特征。
  15. 一种产品缺陷检测系统，其特征在于，包括：存储器和处理器；
    所述存储器，存储计算机可执行指令；
    所述处理器，所述计算机可执行指令在被执行时使所述处理器执行如权利要求1-10任一项所述的产品缺陷检测方法。
PCT/CN2020/111313 2019-12-30 2020-08-26 一种产品缺陷检测方法、装置与系统 WO2021135300A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/309,407 US11741593B2 (en) 2019-12-30 2020-08-26 Product defect detection method, device and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911397516.3A CN111161246B (zh) 2019-12-30 2019-12-30 一种产品缺陷检测方法、装置与系统
CN201911397516.3 2019-12-30

Publications (1)

Publication Number Publication Date
WO2021135300A1 true WO2021135300A1 (zh) 2021-07-08

Family

ID=70559217

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/111313 WO2021135300A1 (zh) 2019-12-30 2020-08-26 一种产品缺陷检测方法、装置与系统

Country Status (3)

Country Link
US (1) US11741593B2 (zh)
CN (1) CN111161246B (zh)
WO (1) WO2021135300A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808104A (zh) * 2021-09-16 2021-12-17 西安交通大学 一种基于分块的金属表面缺陷检测方法及系统
CN116994007A (zh) * 2023-09-26 2023-11-03 支付宝(杭州)信息技术有限公司 商品纹理检测处理方法及装置

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111161246B (zh) 2019-12-30 2024-05-14 歌尔股份有限公司 一种产品缺陷检测方法、装置与系统
US11360030B2 (en) * 2020-02-04 2022-06-14 Applied Materials Isreal Ltd Selecting a coreset of potential defects for estimating expected defects of interest
CN111783886B (zh) * 2020-06-30 2023-01-20 创新奇智(青岛)科技有限公司 一种识别产品缺陷的方法及装置
CN113971650A (zh) * 2020-07-22 2022-01-25 富泰华工业（深圳）有限公司 产品瑕疵检测方法、计算机装置及存储介质
CN112001902A (zh) * 2020-08-19 2020-11-27 上海商汤智能科技有限公司 缺陷检测方法和相关装置、设备、存储介质
CN111951255B (zh) * 2020-08-20 2024-06-18 华北电力大学 缺陷识别方法、装置、终端设备和可读存储介质
CN112232215B (zh) * 2020-10-16 2021-04-06 哈尔滨市科佳通用机电股份有限公司 一种铁路货车钩尾销托梁脱落故障检测方法
CN112270377B (zh) * 2020-11-11 2024-03-15 北京百度网讯科技有限公司 目标图像提取方法、神经网络训练方法和装置
CN112381798A (zh) * 2020-11-16 2021-02-19 广东电网有限责任公司肇庆供电局 一种输电线路缺陷识别方法和终端
CN113298166A (zh) * 2021-06-01 2021-08-24 中科晶源微电子技术（北京）有限公司 缺陷分类器和缺陷分类方法、装置、设备及存储介质
CN113505799B (zh) * 2021-06-30 2022-12-23 深圳市慧鲤科技有限公司 显著性检测方法及其模型的训练方法和装置、设备、介质
CN113781430B (zh) * 2021-09-09 2023-08-25 北京云屿科技有限公司 一种基于深度学习的手套表面缺陷检测方法及系统
CN113838025A (zh) * 2021-09-22 2021-12-24 联想(北京)有限公司 处理方法和处理装置
CN114758260B (zh) * 2022-06-15 2022-10-18 成都鹏业软件股份有限公司 工地安全防护网检测方法及系统
CN115131336B (zh) * 2022-07-22 2023-11-14 电子科技大学 一种显示屏四色背景画面下的暗线缺陷检测方法
CN115660596B (zh) * 2022-11-03 2023-07-04 创启科技(广州)有限公司 一种移动终端的数据交互方法及移动终端
CN115937169A (zh) * 2022-12-23 2023-04-07 广东创新科技职业学院 一种基于高分辨率和目标检测的虾苗计数方法、系统
CN115908988B (zh) * 2023-03-09 2023-05-30 苏州苏映视图像软件科技有限公司 一种缺陷检测模型生成方法、装置、设备以及存储介质
CN116109638B (zh) * 2023-04-13 2023-07-04 中铁四局集团有限公司 一种钢轨折断检测方法及系统
CN116342597B (zh) * 2023-05-29 2023-07-28 深圳市民达科技有限公司 一种汽车配件表面电镀加工缺陷检测方法和系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106340016A (zh) * 2016-08-31 2017-01-18 湖南品信生物工程有限公司 一种基于细胞显微镜图像的dna定量分析方法
CN109711345A (zh) * 2018-12-27 2019-05-03 南京林业大学 一种火焰图像识别方法、装置及其存储介质
CN109712135A (zh) * 2018-12-28 2019-05-03 南京林业大学 一种水果缺陷识别方法、装置及其存储介质
CN110060237A (zh) * 2019-03-29 2019-07-26 腾讯科技（深圳）有限公司 一种故障检测方法、装置、设备及系统
CN110458791A (zh) * 2018-05-04 2019-11-15 深圳市圆周率软件科技有限责任公司 质量缺陷检测方法和检测设备
CN110533023A (zh) * 2019-07-08 2019-12-03 天津商业大学 一种用于检测识别铁路货车异物的方法及装置
CN111161246A (zh) * 2019-12-30 2020-05-15 歌尔股份有限公司 一种产品缺陷检测方法、装置与系统

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4095860B2 (ja) * 2002-08-12 2008-06-04 株式会社日立ハイテクノロジーズ 欠陥検査方法及びその装置
US8457414B2 (en) * 2009-08-03 2013-06-04 National Instruments Corporation Detection of textural defects using a one class support vector machine
CN105118044B (zh) * 2015-06-16 2017-11-07 华南理工大学 一种轮形铸造产品缺陷自动检测方法
US20170069075A1 (en) * 2015-09-04 2017-03-09 Canon Kabushiki Kaisha Classifier generation apparatus, defective/non-defective determination method, and program
CN105930815B (zh) * 2016-05-04 2022-10-04 中国农业大学 一种水下生物检测方法和系统
CN106650830A (zh) * 2017-01-06 2017-05-10 西北工业大学 基于深层模型和浅层模型决策融合的肺结节ct图像自动分类方法
JP6358351B1 (ja) * 2017-03-21 2018-07-18 Ｊｆｅスチール株式会社 表面欠陥検査方法及び表面欠陥検査装置
US10453366B2 (en) * 2017-04-18 2019-10-22 Samsung Display Co., Ltd. System and method for white spot mura detection
CN107341499B (zh) * 2017-05-26 2021-01-05 昆明理工大学 一种基于无监督分割和elm的织物缺陷检测和分类方法
CN107392896B (zh) * 2017-07-14 2019-11-08 佛山市南海区广工大数控装备协同创新研究院 一种基于深度学习的木材缺陷检测方法和系统
CN108154510A (zh) * 2018-01-17 2018-06-12 深圳市亿图视觉自动化技术有限公司 产品表面缺陷检测方法、装置及计算机可读存储介质
CN108564124A (zh) * 2018-04-13 2018-09-21 山东农业大学 一种基于支持向量机的稻瘟病菌孢子显微图像检测识别方法
CN109859181A (zh) * 2019-01-29 2019-06-07 桂林电子科技大学 一种pcb焊点缺陷检测方法
CN109949305B (zh) * 2019-03-29 2021-09-28 北京百度网讯科技有限公司 产品表面缺陷检测方法、装置及计算机设备
CN110223269A (zh) * 2019-04-24 2019-09-10 深圳市派科斯科技有限公司 一种fpc缺陷检测方法和装置
CN110570396B (zh) * 2019-08-07 2022-03-15 华中科技大学 一种基于深度学习的工业产品缺陷检测方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106340016A (zh) * 2016-08-31 2017-01-18 湖南品信生物工程有限公司 一种基于细胞显微镜图像的dna定量分析方法
CN110458791A (zh) * 2018-05-04 2019-11-15 深圳市圆周率软件科技有限责任公司 质量缺陷检测方法和检测设备
CN109711345A (zh) * 2018-12-27 2019-05-03 南京林业大学 一种火焰图像识别方法、装置及其存储介质
CN109712135A (zh) * 2018-12-28 2019-05-03 南京林业大学 一种水果缺陷识别方法、装置及其存储介质
CN110060237A (zh) * 2019-03-29 2019-07-26 腾讯科技（深圳）有限公司 一种故障检测方法、装置、设备及系统
CN110533023A (zh) * 2019-07-08 2019-12-03 天津商业大学 一种用于检测识别铁路货车异物的方法及装置
CN111161246A (zh) * 2019-12-30 2020-05-15 歌尔股份有限公司 一种产品缺陷检测方法、装置与系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808104A (zh) * 2021-09-16 2021-12-17 西安交通大学 一种基于分块的金属表面缺陷检测方法及系统
CN113808104B (zh) * 2021-09-16 2024-04-02 西安交通大学 一种基于分块的金属表面缺陷检测方法及系统
CN116994007A (zh) * 2023-09-26 2023-11-03 支付宝(杭州)信息技术有限公司 商品纹理检测处理方法及装置
CN116994007B (zh) * 2023-09-26 2024-03-19 支付宝(杭州)信息技术有限公司 商品纹理检测处理方法及装置

Also Published As

Publication number Publication date
US20220309640A1 (en) 2022-09-29
US11741593B2 (en) 2023-08-29
CN111161246B (zh) 2024-05-14
CN111161246A (zh) 2020-05-15

Similar Documents

Publication Publication Date Title
WO2021135300A1 (zh) 一种产品缺陷检测方法、装置与系统
WO2021135302A1 (zh) 一种产品缺陷检测方法、装置与系统
WO2021135372A1 (zh) 一种产品缺陷检测方法、装置与系统
WO2021135331A1 (zh) 一种产品缺陷检测方法、装置与系统
CN110826416B (zh) 一种基于深度学习的卫浴陶瓷表面缺陷检测方法及装置
WO2022036919A1 (zh) 缺陷检测方法、装置、电子设备及计算机存储介质
US8750596B2 (en) System and method for identifying defects in a material
TWI642126B (zh) 半導體晶圓檢測系統及其方法
CN109308489B (zh) 一种元件电弧焊焊接质量检测方法
CN110889838A (zh) 一种织物疵点检测方法及装置
WO2024002187A1 (zh) 缺陷检测方法、缺陷检测设备及存储介质
CN205538710U (zh) 一种基于机器视觉的电感质量自动检测系统
CN109598298B (zh) 图像物体识别方法和系统
CN111310826A (zh) 样本集的标注异常检测方法、装置及电子设备
CN111242899A (zh) 基于图像的瑕疵检测方法及计算机可读存储介质
CN114897868A (zh) 极片缺陷识别及模型训练方法、装置及电子设备
TWI749714B (zh) 瑕疵檢測方法、瑕疵分類方法及其系統
CN113793322A (zh) 一种对磁性材料自动检测的方法、电子设备和存储介质
TWM550465U (zh) 半導體晶圓檢測系統
CN117011216A (zh) 一种缺陷检测方法、装置、电子设备及存储介质
CN114486647A (zh) 物料粒子的检测方法、装置、粒子检测装置、计算机设备
CN116958086B (zh) 具有增强特征融合能力的金属表面缺陷检测方法及系统
TWI777696B (zh) 缺陷檢測方法以及使用該方法的缺陷檢測系統
Murugan et al. Enhancing Textile Quality Assurance with TensorFlow: Detecting Fabric Anomalies
CN112730427B (zh) 一种基于机器视觉的产品表面缺陷检测方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20908564

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20908564

Country of ref document: EP

Kind code of ref document: A1