WO2022148109A1 - 产品缺陷检测方法、装置、设备及计算机可读存储介质 - Google Patents

产品缺陷检测方法、装置、设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2022148109A1
WO2022148109A1 PCT/CN2021/126935 CN2021126935W WO2022148109A1 WO 2022148109 A1 WO2022148109 A1 WO 2022148109A1 CN 2021126935 W CN2021126935 W CN 2021126935W WO 2022148109 A1 WO2022148109 A1 WO 2022148109A1
Authority
WO
WIPO (PCT)
Prior art keywords
product
anchor
defect
image
predicted
Prior art date
Application number
PCT/CN2021/126935
Other languages
English (en)
French (fr)
Inventor
于瑞涛
高巍
Original Assignee
歌尔股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 歌尔股份有限公司 filed Critical 歌尔股份有限公司
Publication of WO2022148109A1 publication Critical patent/WO2022148109A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/60—Rotation of a whole image or part thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/68—Analysis of geometric attributes of symmetry
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Definitions

  • the present invention relates to the technical field of artificial intelligence, and in particular, to a product defect detection method, apparatus, device, and computer-readable storage medium.
  • the main purpose of the present invention is to provide a product defect detection method, device, equipment and computer-readable storage medium, which aims to solve the technical problem that the current neural network algorithm has low detection accuracy for subtle defects in different product surface directions.
  • an embodiment of the present invention provides a product defect detection method, which includes:
  • Feature extraction is performed on the product image of the target product to obtain a feature map
  • the anchor boxes are classified to obtain a detection result of whether each anchor box contains a defect.
  • the embodiment of the present invention also provides a product defect detection device, including:
  • the extraction module is used to extract the feature map of the product image of the target product to obtain the feature map
  • a generation module is used to generate anchor frames of a first preset size, a preset aspect ratio and a preset rotation angle to each anchor point in the feature map;
  • the classification module is used to classify each anchor frame to obtain the detection result of whether each anchor frame contains defects.
  • an embodiment of the present invention also provides a product defect detection device, the product defect detection device includes: a memory, a processor, and a product defect detection program stored in the memory and running on the processor, and a product defect detection program.
  • the above product defect detection method is implemented when the program is executed by the processor.
  • the present invention also provides a computer-readable storage medium.
  • the computer-readable storage medium stores a product defect detection program, and when the product defect detection program is executed by the processor, implements the above product defect detection method.
  • a feature map is obtained by performing feature extraction on a product image of a target product, an anchor frame with a preset length-width ratio, a preset size and a preset rotation angle is generated for each anchor point in the feature map, and each anchor frame is The detection results of whether each anchor box contains defects are obtained by classification, so as to realize the detection of whether the target product contains defects.
  • the present invention also adds a preset rotation angle to generate an anchor frame with angle components, so that subtle defects in different directions in the product can be more fully located, thereby improving the product quality.
  • FIG. 1 is a schematic structural diagram of a hardware operating environment involved in an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of the first embodiment of the product defect detection method of the present invention.
  • FIG. 3 is a schematic diagram of an anchor frame involved in an embodiment of the product defect detection method of the present invention.
  • FIG. 4 is a schematic diagram of a process flow diagram of a diaphragm image defect detection involved in an embodiment of the product defect detection method of the present invention
  • FIG. 5 is a schematic diagram of functional modules of a preferred embodiment of the product defect detection device of the present invention.
  • FIG. 1 is a schematic diagram of a device structure of a hardware operating environment involved in an embodiment of the present invention.
  • the product defect detection device in the embodiment of the present invention may be devices such as a smart phone, a personal computer, and a server, which are not specifically limited herein.
  • the product defect detection device may include: a processor 1001 , such as a CPU, a network interface 1004 , a user interface 1003 , a memory 1005 , and a communication bus 1002 .
  • the communication bus 1002 is used to realize the connection and communication between these components.
  • the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may include a standard wired interface and a wireless interface (eg, a WI-FI interface).
  • the memory 1005 may be high-speed RAM memory, or may be non-volatile memory, such as disk memory.
  • the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .
  • the device structure shown in FIG. 1 does not constitute a limitation on the product defect detection device, and may include more or less components than the one shown, or combine some components, or arrange different components .
  • the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and a product defect detection program.
  • An operating system is a program that manages and controls device hardware and software resources, supports the operation of product defect detection programs, and other software or programs.
  • the user interface 1003 is mainly used for data communication with the client;
  • the network interface 1004 is mainly used to establish a communication connection with the server;
  • the processor 1001 can be used to call the product defect detection stored in the memory 1005 program, and do the following:
  • Feature extraction is performed on the product image of the target product to obtain a feature map
  • An anchor frame with a first preset size, a preset aspect ratio and a preset rotation angle is generated for each anchor point in the feature map;
  • the processor 1001 can also be used to call the product defect detection program stored in the memory 1005, and perform the following operations:
  • boundary regression calculation is performed on the positive anchor frames containing defects in each anchor frame, and the predicted offset corresponding to each positive anchor frame is obtained, wherein the predicted offset includes the predicted angle offset;
  • the region location of the predicted defect region in the product image is generated based on the positive anchor box and the corresponding predicted offset.
  • the processor 1001 can also be used to call the product defect detection program stored in the memory 1005, and perform the following operations:
  • the predicted defect area is classified to obtain the defect type of the predicted defect area.
  • the processor 1001 can also be used to call the product defect detection program stored in the memory 1005, and perform the following operations: :
  • the processor 1001 can also be used to call the product defect detection program stored in the memory 1005, and perform the following operations:
  • the target product is an axisymmetric product, select the preset rotation angle from 0 to 180 degrees;
  • the processor 1001 can also be used to call the product defect detection program stored in the memory 1005, and perform the following operations:
  • the steps of classifying each anchor frame to obtain a detection result of whether each anchor frame contains defects include:
  • the step of performing feature extraction on the product image of the target product to obtain a feature map includes:
  • the product image is obtained by filling the area except the product area in the preliminary image with a solid color
  • feature extraction is performed to obtain a feature map.
  • FIG. 2 is a schematic flowchart of the first embodiment of the product defect detection method of the present invention.
  • This embodiment of the present invention provides an embodiment of a product defect detection method. It should be noted that although a logical sequence is shown in the flowchart, in some cases, the shown or described steps.
  • the execution body of each embodiment of the product defect detection method of the present invention may be devices such as a smart phone, a personal computer, and a server. For ease of description, the execution body is omitted in the following embodiments for description.
  • the product defect detection method includes:
  • Step S10 performing feature extraction on the product image of the target product to obtain a feature map
  • some defects on the surface of the product may be caused by factors such as price, technology, equipment, raw materials, etc., so it is necessary to detect the defects on the surface of the product, and repair or eliminate the defective product, To ensure product quality.
  • the products in this embodiment may be products with various surfaces that may have defects, such as headphone diaphragms, ceramic products, and the like.
  • the product that needs to be inspected for defects can be used as the target product, and a camera can be used to capture an image containing the area to be inspected, that is, the product image.
  • Feature extraction is performed on the product image to obtain a feature map.
  • commonly used feature extraction network models can be used for feature extraction operations.
  • the convolution layer in Mask RCNN or Faster RCNN can be used for feature extraction.
  • Mask RCNN and Faster RCNN are both instance segmentation (Instance segmentation) algorithms .
  • preprocessing operations such as size scaling or cropping can be performed on the product image to meet the data structure requirements of the input data of the feature extraction network model, and the preprocessed image is input into the feature extraction network model.
  • successive processing to obtain feature maps There are generally multiple feature maps, and the processing steps for each feature map are similar, so in the following embodiments, a feature map may be used as an example for description.
  • step S10 includes:
  • Step S101 collect a preliminary image of the target product, and locate the product area in the preliminary image
  • Step S102 performing solid color filling on the area other than the product area in the preliminary image to obtain a product image
  • Step S103 After scaling the product image to a second preset size, feature extraction is performed to obtain a feature map.
  • the image of the target product captured by the camera may be used as a preliminary image.
  • the product area in the preliminary image is located, and the area other than the product area in the preliminary image is filled with solid color, and the obtained image is used as the product image.
  • the PMAlign tool in VisionPro a computer-based vision software
  • a solid color fill can be filled with black or white, etc.
  • the product image can be scaled to a preset size (hereinafter referred to as the second preset size for distinction), and then feature extraction is performed on the scaled image to obtain a feature map.
  • the second preset size may be set in advance as required, and may be set according to the size of the product. For example, for the headphone diaphragm, the second preset size may be set to 2048*2048.
  • a camera can be set at a defect detection station of the production line, and when the product is transported or placed at the station, the camera captures an image of the product, and the inspection equipment and the camera are connected through wired or wireless communication. , obtain product images from cameras, and perform defect detection on product images or offline.
  • Step S20 generating anchor frames with a first preset size, a preset length-width ratio and a preset rotation angle for each anchor point in the feature map;
  • each point in the feature map can be used as an anchor point, and a preset size (hereinafter referred to as the first preset size to distinguish), preset aspect ratio and preset rotation can be generated for each anchor point.
  • Anchor for the angle.
  • the preset size, preset aspect ratio and preset rotation angle can be set in advance according to needs, and multiple preset size, preset aspect ratio and preset rotation angle can be set respectively, then, set the size, preset aspect ratio and preset rotation angle.
  • the three parameters of scale and rotation angle are arranged and combined to obtain multiple parameter groups. For an anchor point, an anchor frame is generated according to each parameter group, and multiple anchor frames corresponding to the anchor point are obtained.
  • the quantization rotation step size corresponding to each anchor frame can be the same.
  • the quantization rotation step size used by the anchor box of , and the quantization rotation step size used by different anchor boxes is different.
  • an anchor frame with a reference direction can be generated according to the scale and size in the parameter group, and then the anchor frame can be rotated according to the rotation angle in the parameter group to obtain the anchor frame corresponding to the parameter group.
  • the preset size has three sizes of 2, 4 and 8
  • the preset aspect ratio has three aspect ratios of 1:3, 1:5 and 1:7
  • the preset rotation angle has 0 degree, 45 degree
  • FIG 3 in the feature map of M*N size, taking an anchor frame with an aspect ratio of 1:3 and a size of 2 as an example, four rotation angles of 0 degrees, 45 degrees, 90 degrees and 135 degrees are generated in total. anchor box.
  • the anchor frame generation method can be generated according to the anchor frame generation method of RPN (Region Proposal Networks) in Mask RCNN or Faster RCNN.
  • RPN Registered Proposal Networks
  • the rotation angle is increased, that is, the original An anchor frame with a certain size and a certain proportion is also rotated according to a certain rotation angle, so as to obtain an anchor frame with a direction component.
  • each generated anchor frame is an anchor frame with an angular component
  • anchor frames with different rotation angles can locate defects in different directions, so that defects in various directions can be more accurately identified.
  • a slender anchor frame in multiple directions can be generated by setting a ratio with a large difference in length and width and multiple rotation angles, and these defects can be identified to increase defect detection. Accuracy.
  • Step S30 Classify each anchor frame to obtain a detection result of whether each anchor frame contains a defect.
  • each anchor frame is classified to obtain a detection result of whether each anchor frame contains defects.
  • a conventional binary classification algorithm can be used to classify each anchor frame to obtain whether the included detection result is included, or each anchor frame can be input into a second classifier for classification, and the second classifier outputs the anchor frame Whether to contain the detection result of the defect.
  • the binary classifier can be implemented by a commonly used classifier, such as softmax.
  • the product image of the target product may be a training sample for training the network model.
  • the defect areas in the product images may be manually or machine marked to obtain the real defect areas in the product images.
  • the classification loss function can use the commonly used binary classification loss function.
  • the detection results of each anchor frame After the detection results of each anchor frame are obtained, it can be determined whether the target product has defects according to the detection results. Specifically, if the detection result of one of the anchor frames contains defects, it can be determined that the target product is defective, and if all the anchor frame detection results are not defective, it can be determined that the target product does not exist.
  • a threshold can be set; when the number of anchor frames containing defects is less than the threshold, it can be determined that the target product does not have defects, and the target product is sent to the next station for continuation Processing or packaging; when the number of anchor frames containing defects is not greater than the threshold, the defect of the target product can be determined, and the position of the anchor frame can be output for the production personnel to check whether the position of the anchor frame is defective.
  • a feature map is obtained by performing feature extraction on the product image of the target product, an anchor frame with a preset aspect ratio, preset size and preset rotation angle is generated for each anchor point in the feature map, and each anchor point in the feature map is generated.
  • the frame is classified to obtain the detection results of whether each anchor frame contains defects, so as to realize the detection of whether the target product contains defects.
  • a preset rotation angle is also added, and an anchor frame with an angle component is generated, so that subtle defects in different directions in the product can be more fully located, thereby improving the performance of the product.
  • step S30 the method further includes:
  • Step S40 performing boundary regression calculation on positive anchor frames containing defects in each anchor frame according to the detection result, to obtain a predicted offset corresponding to each positive anchor frame, wherein the predicted offset includes a predicted angle offset;
  • the anchor frame containing defects is used as the positive anchor frame.
  • the boundary regression calculation may be further performed on the positive anchor frame to obtain the predicted offset corresponding to each positive anchor frame.
  • the boundary regression calculation can be implemented by using the boundary regression calculation method in the RPN.
  • the predicted offset also includes the predicted angle offset, Specifically, the predicted angle offset may be an angle value, and the angle value represents the direction angle difference between the positive anchor frame and the real defect area in the product image. Of course, the angle value is only a predicted value.
  • the RPN can be trained in advance so that the predicted offset calculated by boundary regression is closer to the real offset.
  • Step S50 generating the area position of the predicted defect area in the product image according to the positive anchor frame and the corresponding predicted offset.
  • the area position of the predicted defect area in the product image can be generated according to the positive anchor frame and the corresponding predicted offset.
  • the positive anchor frame can be translated according to the translation amount in the predicted offset, then scaled according to the scaling scale in the predicted offset, and then rotated according to the predicted angle offset to obtain the regional position of the predicted defect area . Since the obtained area position is the result of prediction, it is called the area position of the predicted defect area.
  • the region position can be represented by (x1, y1, x2, y2, ⁇ ), where (x1, y1) is the coordinates of the upper left corner of the region, (x2, y2) is the coordinates of the lower right corner of the region, and ⁇ is relative to the region.
  • the angular component of the reference direction is the intersection of the region.
  • the area position After obtaining the area position of the defect area, the area position can be output, or the area position can be marked in the product image, and the marked product image can be output, so that the production personnel can analyze the defects of the target product according to the output results. .
  • the anchor frame containing defects is used as the positive anchor frame, and boundary regression calculation is performed on the positive anchor frame to obtain the predicted offset corresponding to the positive anchor frame, and the predicted offset
  • the quantity includes the predicted angle offset.
  • the area position of the predicted defect area is obtained, which realizes the precise positioning of the defect areas in different directions in the target product, so that the production personnel can obtain the target product. The exact location of the defective area in the product.
  • step S50 it also includes:
  • Step S60 classifying the predicted defect area to obtain the defect type of the predicted defect area.
  • the predicted defect area may be further classified to obtain the defect type of the defect area.
  • the product may have different types of defects, such as wool impurities, scratches and poor edge bonding, etc. Therefore, the predicted defect area can be further classified to determine which defect type the area belongs to.
  • the classification can be implemented by using a multi-classification network in Mask RCNN or Faster RCNN, which will not be described in detail in this embodiment.
  • the defect type and area location can be output together, or the defect type can also be marked in the product image, and the marked product image can be output, so that the production personnel can easily know the target product The defect location and defect type in .
  • instance segmentation can be performed on the predicted defect area to obtain a mask map of the predicted defect area, and each point in the mask map corresponds to whether each point in the predicted defect area is a defect point. , and output the mask image directly or after merging it with the product image of the target product, so that the production personnel can easily know the specific position of the defect in the target product.
  • step S50 it also includes:
  • Step S70 calculating the real offset of the positive anchor frame relative to the real defect area pre-marked in the product image, wherein the real offset includes the real angle offset;
  • Step S80 Calculate a loss function according to the predicted offset and the actual offset, and update the regression parameters used in the boundary regression calculation according to the loss function.
  • the product image of the target product may be a training sample for training a network model.
  • the defect area in the product image can be marked manually or by machine to obtain the area position of the real defect area in the product image; these product images are processed according to the above steps to obtain the predicted offset corresponding to the positive anchor frame
  • the real offset of the positive anchor frame relative to the real defect frame can be calculated according to the area position of the real defect area, and the real offset also includes the real angle offset; according to the predicted offset and the real offset
  • the regression loss function the regression parameters used in the boundary regression calculation and each parameter in the feature extraction network are trained (updated) according to the method of backpropagation, so that Continuously reduce the gap between the predicted offset and the true offset, thereby improving the detection accuracy of predicted defect areas.
  • the regression loss function and the classification loss function can also be used as a total loss function to train each part of the parameters in the network.
  • the network training process may be implemented according to a conventional machine learning model training method, which will not be described in detail
  • the regression loss function can use a loss function commonly used in boundary regression calculation in RPN, such as smooth L1 loss. Since the anchor frame in this embodiment has an angle component, the predicted offset also includes the predicted angle offset, so the loss The predicted loss for the angle component is also included in the function.
  • the regression loss function L reg uses the smooth L1 loss combined with the prediction loss of the angle component, which is expressed as follows:
  • t is the positive anchor frame
  • t* is the real defect area corresponding to the positive anchor frame
  • (x1, y1) represents the upper-left corner vertex coordinates
  • (x2, y2) represents the lower-right corner vertex coordinates
  • ⁇ represents the angle component.
  • the product images of the products to be detected are detected according to the above steps, and the detection results of whether the product images contain defects are obtained, as well as the area location and defect type of the defect area when the defect is included.
  • an anchor frame with an angle component is generated, and the prediction loss of the angle component is added to the regression loss function, so that anchors with different angle components can be trained during the network training process.
  • Positioning and matting remove the background of the acquired image except the diaphragm product, and fill the removed background with pure black to avoid the interference of the background part to the subsequent defect detection; then use the resize function to uniformly scale the image to 2048*2048 Size, input into the Mask-RCNN network, first through the ResNet101+FPN (residual network + feature pyramid network) in the network to perform feature extraction to obtain feature maps; then generate anchors with angle components for the feature map; Anchors are input to the RPN network; two networks are separated from RPN: RPN-Class and RPN_BBox, where RPN-Class uses softmax classification to determine whether the anchors are defective, and RPN_BBox is used for boundary regression to predict the offset of anchors; calculate classification loss and regression loss , and update the parameters in the network through the back-propagation algorithm to complete the network training; input the diaphragm image to be detected into the trained network to obtain the detection result of whether the diaphragm image has defects, and the location of the defect area when there
  • the method further includes:
  • Step A10 performing symmetry detection on the product image of the target product to determine whether the target product is an axisymmetric product
  • Step A20 if the target product is an axisymmetric product, select a preset rotation angle from 0 to 180 degrees;
  • Step A30 if the target product is not an axisymmetric product, select a preset rotation angle from 0 to 360 degrees.
  • symmetry detection can be performed on the product image of the target product in advance to determine whether the target product is an axisymmetric product.
  • the symmetry detection may be to divide the product image with one or more dividing lines to obtain two images, and after flipping one of the images with the dividing line as the axis, perform image matching with the other image to obtain a similar image. If the similarity is greater than a certain threshold, for example, greater than 99%, it is determined that the target product is an axisymmetric product. Or, in other implementation manners, the symmetry detection may also be performed in other manners.
  • the preset rotation angle can be selected from 0 to 180 degrees.
  • the interval from 0 to 180 degrees may be divided into multiple intervals equally or unequally, and the front end point of each interval is used as the preset rotation angle. For example, it is divided into 4 intervals to obtain 4 intervals [0, 45), [45, 90), [90, 135) and [135, 180), and the front-end points of these 4 intervals are 0, 45, 90 and 135 as the preset rotation angle.
  • the number of intervals can be set in advance according to the needs, that is, a quantization rotation step size is selected; when the defect detection accuracy is required to be higher, the number of intervals can be set to be larger, that is, the quantization rotation step size is smaller, thereby Generate anchor boxes with more compact direction intervals and locate more directions; however, the more the number of intervals is set, the greater the amount of calculation and the slower the calculation speed in the defect detection process. Therefore, it can be set according to the requirements of calculation amount and accuracy rate.
  • the number of intervals to be centered.
  • a preset rotation angle can be selected from 0 to 360 degrees.
  • the interval from 0 to 360 degrees can also be divided into a plurality of intervals, and the front end point of each interval is used as the preset rotation angle.
  • the number of intervals can also be preset.
  • the rotation angle is selected from 0 to 180 degrees for axisymmetric products, and the rotation angle is selected from 0 to 360 degrees for non-axisymmetric products, so that the anchor frame
  • the rotation angle can be set according to the shape of the product, so that the anchor frame can cover the defects of the product in different directions.
  • step S30 it also includes:
  • Step A40 calculate the intersection ratio of each anchor frame and the product image respectively
  • the intersection ratio of each anchor frame and the product image can be calculated respectively, and the intersection ratio corresponding to each anchor frame is obtained.
  • the calculation method of the intersection ratio is to calculate the area of the overlapping area of the anchor frame and the product image, and then divide it by the area of the merged area of the anchor frame and the product image.
  • Step S30 includes:
  • Step S301 classifying target anchor frames whose intersection ratio is greater than a preset value corresponding to each anchor frame, to obtain a detection result of whether the target anchor frame contains defects.
  • An anchor frame with a corresponding intersection ratio of each anchor frame greater than a preset value is used as a target anchor frame, and the target anchor frame is classified to obtain a detection result of whether the target anchor frame contains defects.
  • the preset value can be set as required, for example, 0.7. That is, in this embodiment, not all anchor frames are classified, but only anchor frames with more overlapping areas with the product image are classified.
  • the anchor frames that intersect with the product image are generally located in the edge area of the product image. This type of anchor frame is often difficult to learn the product defect features, so this type of anchor frame can be eliminated to avoid affecting product defect detection. The above pair reduces the amount of calculation and improves the detection efficiency.
  • an embodiment of the present invention also provides a product defect detection device, referring to FIG. 4 , the device includes:
  • the extraction module 10 is configured to perform feature extraction on the product image of the target product to obtain a feature map
  • the generating module 20 is used to generate anchor frames with a first preset size, a preset aspect ratio and a preset rotation angle for each anchor point in the feature map;
  • the classification module 30 is configured to classify each anchor frame to obtain a detection result of whether each anchor frame contains a defect.
  • the above-mentioned device also includes:
  • the regression module is used to perform boundary regression calculation on the positive anchor frames containing defects in each anchor frame according to the detection result, and obtain the predicted offset corresponding to each positive anchor frame, wherein the predicted offset includes the predicted angle offset;
  • the position generation module is used to generate the area position of the predicted defect area in the product image according to the positive anchor frame and the corresponding predicted offset.
  • the above-mentioned device also includes:
  • the defect classification module is used to classify the predicted defect area to obtain the defect type of the predicted defect area.
  • the above-mentioned device also includes:
  • a first calculation module used to calculate the real offset of the positive anchor frame relative to the real defect area pre-marked in the product image, wherein the real offset includes the real angle offset;
  • the update module is used to calculate the loss function according to the predicted offset and the real offset, and update the regression parameters used in the boundary regression calculation according to the loss function.
  • the above-mentioned device also includes:
  • the detection module is used to perform symmetry detection on the product image of the target product to determine whether the target product is an axisymmetric product;
  • the first selection module is used to select a preset rotation angle from 0 to 180 degrees if the target product is an axisymmetric product;
  • the second selection module is used to select a preset rotation angle from 0 to 360 degrees if the target product is not an axisymmetric product.
  • the above-mentioned device also includes:
  • the second calculation module is used to calculate the intersection ratio of each anchor frame and the product image respectively;
  • Classification module 30 is also used to:
  • the extraction module 10 includes:
  • the positioning unit is used to collect a preliminary image of the target product and locate the product area in the preliminary image
  • the filling unit is used to fill the area except the product area in the preliminary image with solid color to obtain the product image
  • the extraction unit is configured to perform feature extraction after scaling the product image to a second preset size to obtain a feature map.
  • the expanded content of the specific implementation manner of the product defect detection device of the present invention is basically the same as that of the above-mentioned embodiments of the product defect detection method, and will not be repeated here.
  • an embodiment of the present invention also provides a computer-readable storage medium, where a product defect detection program is stored on the computer storage medium.
  • a product defect detection program is stored on the computer storage medium.

Abstract

本发明公开了一种产品缺陷检测方法、装置、设备及计算机可读存储介质，对目标产品的产品图像进行特征提取得到特征图；对所述特征图中的各锚点生成第一预设尺寸、预设长宽比例和预设旋转角度的锚框；对各所述锚框进行分类得到各所述锚框是否包含缺陷的检测结果。本发明能够更充分地定位到产品中方向各异的细微缺陷，从而提高了产品的缺陷检测准确率。

Description

产品缺陷检测方法、装置、设备及计算机可读存储介质 技术领域
本发明涉及人工智能技术领域，尤其涉及一种产品缺陷检测方法、装置、设备及计算机可读存储介质。
发明背景
产品在生成过程中，会因为价格工艺、设备、原材料等因素造成品质缺陷。为了保证产品的品质，在发货之前需要对产品表面进行缺陷检测。对于一些尺寸较小的产品，目前的缺陷检测方法有两种：一种是依靠人力使用显微镜观察产品，另一种是依靠传统计算机视觉进行检测。人工肉眼检测缺陷存在诸多问题，例如每日生产产品的数量巨大，在检验环节就需求大量人力，而且人工检验很大程度受到主观标准、劳累程度和情绪的制约，无法保证发货品质的一致性；而使用传统计算机视觉检测细微划痕、印痕等缺陷存在较大困难。
结合近年深度学习在计算机视觉领域的发展，深度卷积神经网络算法在表面缺陷检测得到广泛的应用。但是由于一些产品表面的毛纤杂质、划伤和边缘粘接不良等缺陷往往是方向各异，导致目前的神经网络算法对此类缺陷的检测准确率较低。
发明内容
本发明的主要目的在于提供一种产品缺陷检测方法、装置、设备及计算机可读存储介质，旨在解决目前的神经网络算法对产品表面方向各异的细微缺陷检测准确率较低的技术问题。
为实现上述目的，本发明实施例提供一种产品缺陷检测方法，该方法包括：
对目标产品的产品图像进行特征提取得到特征图；
对所述特征图中的各锚点生成第一预设尺寸、预设长宽比例和预设旋转角度的锚框；
对各所述锚框进行分类得到各锚框是否包含缺陷的检测结果。
为实现上述目的，本发明实施例还提供一种产品缺陷检测装置，包括：
提取模块，用于对目标产品的产品图像进行特征提取得到特征图；
生成模块，用于对特征图中的各锚点生成第一预设尺寸、预设长宽比例和 预设旋转角度的锚框；
分类模块，用于对各锚框进行分类得到各锚框是否包含缺陷的检测结果。
为实现上述目的，本发明实施例还提供一种产品缺陷检测设备，该产品缺陷检测设备包括：存储器、处理器及存储在存储器上并可在处理器上运行的产品缺陷检测程序，产品缺陷检测程序被处理器执行时实现如上的产品缺陷检测方法。
此外，为实现上述目的，本发明还提出一种计算机可读存储介质。该计算机可读存储介质上存储有产品缺陷检测程序，该产品缺陷检测程序被处理器执行时实现如上的产品缺陷检测方法。
本发明中，通过对目标产品的产品图像进行特征提取得到特征图，对特征图中的各个锚点生成预设长宽比例、预设尺寸和预设旋转角度的锚框，对各个锚框进行分类得到各个锚框是否包含缺陷的检测结果，以实现对目标产品是否包含缺陷的检测。由于本发明相比于现有的锚框生成方式还增加了预设旋转角度，生成带有角度分量的锚框，从而能够更充分地定位到产品中方向各异的细微缺陷，从而提高了产品的缺陷检测准确率
附图简要说明
图1为本发明实施例方案涉及的硬件运行环境的结构示意图；
图2为本发明产品缺陷检测方法第一实施例的流程示意图；
图3为本发明产品缺陷检测方法实施例涉及的一种锚框示意图；
图4为本发明产品缺陷检测方法实施例涉及的一种振膜图像缺陷检测流程示意图；
图5为本发明产品缺陷检测装置较佳实施例的功能模块示意图。
本发明目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定 本发明。
如图1所示，图1是本发明实施例方案涉及的硬件运行环境的设备结构示意图。
需要说明的是，本发明实施例产品缺陷检测设备可以是智能手机、个人计算机和服务器等设备，在此不做具体限制。
如图1所示，该产品缺陷检测设备可以包括：处理器1001，例如CPU，网络接口1004，用户接口1003，存储器1005，通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。用户接口1003可以包括显示屏(Display)、输入单元比如键盘(Keyboard)，可选用户接口1003还可以包括标准的有线接口、无线接口。网络接口1004可选的可以包括标准的有线接口、无线接口(如WI-FI接口)。存储器1005可以是高速RAM存储器，也可以是稳定的存储器(non-volatile memory)，例如磁盘存储器。存储器1005可选的还可以是独立于前述处理器1001的存储装置。
本领域技术人员可以理解，图1中示出的设备结构并不构成对产品缺陷检测设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
如图1所示，作为一种计算机存储介质的存储器1005中可以包括操作系统、网络通信模块、用户接口模块以及产品缺陷检测程序。操作系统是管理和控制设备硬件和软件资源的程序，支持产品缺陷检测程序以及其它软件或程序的运行。在图1所示的设备中，用户接口1003主要用于与客户端进行数据通信；网络接口1004主要用于与服务器建立通信连接；而处理器1001可以用于调用存储器1005中存储的产品缺陷检测程序，并执行以下操作：
对目标产品的产品图像进行特征提取得到特征图；
对特征图中的各锚点生成第一预设尺寸、预设长宽比例和预设旋转角度的锚框；
对各锚框进行分类得到各锚框是否包含缺陷的检测结果。
进一步地，对各锚框进行分类得到各锚框是否包含缺陷的检测结果的步骤之后，处理器1001还可以用于调用存储器1005中存储的产品缺陷检测程序，并执行以下操作：
根据检测结果对各锚框中包含缺陷的正锚框进行边界回归计算，得到各正锚框对应的预测偏移量，其中，预测偏移量包括预测角度偏移量；
根据正锚框和对应的预测偏移量生成产品图像中预测缺陷区域的区域位置。
进一步地，根据正锚框和对应的预测偏移量生成产品图像中缺陷区域的区域位置的步骤之后，处理器1001还可以用于调用存储器1005中存储的产品缺陷检测程序，并执行以下操作：
对预测缺陷区域进行分类得到预测缺陷区域的缺陷类型。
进一步地，根据正锚框和对应的预测偏移量生成产品图像中预测缺陷区域的区域位置的步骤之后，处理器1001还可以用于调用存储器1005中存储的产品缺陷检测程序，并执行以下操作：
计算正锚框相对于产品图像中预标注的真实缺陷区域的真实偏移量，其中，真实偏移量包括真实角度偏移量；
根据预测偏移量和真实偏移量计算损失函数，并根据损失函数更新边界回归计算中采用的回归参数。
进一步地，对目标产品的产品图像进行特征提取得到特征图的步骤之前，处理器1001还可以用于调用存储器1005中存储的产品缺陷检测程序，并执行以下操作：
对目标产品的产品图像进行对称性检测，以确定目标产品是否属于轴对称产品；
若目标产品属于轴对称产品，则从0到180度之间选择预设旋转角度；
若目标产品不属于轴对称产品，则从0到360度之间选择预设旋转角度。
进一步地，对各锚框进行分类得到各锚框是否包含缺陷的检测结果的步骤之前，处理器1001还可以用于调用存储器1005中存储的产品缺陷检测程序，并执行以下操作：
分别计算各锚框与产品图像的交并比；
对各锚框进行分类得到各锚框是否包含缺陷的检测结果的步骤包括：
对各锚框中对应的交并比大于预设值的目标锚框进行分类，得到目标锚框是否包含缺陷的检测结果。
进一步地，对目标产品的产品图像进行特征提取得到特征图的步骤包括：
采集目标产品的初步图像，并定位初步图像中的产品区域；
对初步图像中除产品区域外的区域进行纯色填充得到产品图像；
将产品图像缩放至第二预设尺寸后进行特征提取得到特征图。
基于上述的结构，提出产品缺陷检测方法的各个实施例。
参照图2，图2为本发明产品缺陷检测方法第一实施例的流程示意图。
本发明实施例提供了产品缺陷检测方法的实施例，需要说明的是，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。本发明产品缺陷检测方法各个实施例的执行主体可以是智能手机、个人计算机和服务器等设备，为便于描述，以下各实施例中省略执行主体进行阐述。在本实施例中，产品缺陷检测方法包括：
步骤S10，对目标产品的产品图像进行特征提取得到特征图；
在本实施例中，产品在生产过程中，可能会因为价格工艺、设备、原材料等因素造成产品表面的一些缺陷，故需要对产品表面的缺陷进行检测，对存在缺陷的产品进行修复或排除，以保证产品的品质。本实施例中的产品可以是各种表面可能会存在缺陷的产品，例如耳机振膜、陶瓷制品等等。
可将需要检测缺陷的产品作为目标产品，并采用摄像机拍摄包含产品待检测区域的图像，也即产品图像。对该产品图像进行特征提取得到特征图(feature map)。其中，可采用常用的特征提取网络模型来进行特征提取操作，例如可采用Mask RCNN或Faster RCNN中的卷积层来进行特征提取，其中，Mask RCNN和Faster RCNN都是实例分割(Instance segmentation)算法。具体地，可先将产品图像进行尺寸缩放或裁剪等预处理操作，以符合特征提取网络模型的输入数据的数据结构要求，将预处理后的图像输入到特征提取网络模型，经过模型中各个层的逐次处理，得到特征图。特征图一般是有多张，对各张特征图的处理步骤类似，故在以下各实施例中，或将以一张特征图进行举例说明。
进一步地，步骤S10包括：
步骤S101，采集目标产品的初步图像，并定位初步图像中的产品区域；
步骤S102，对初步图像中除产品区域外的区域进行纯色填充得到产品图像；
步骤S103，将产品图像缩放至第二预设尺寸后进行特征提取得到特征图。
在本实施例中，为避免拍摄的产品图像中背景部分对后续的缺陷检测造成干扰，可将摄像机拍摄的目标产品的图像作为初步图像。定位初步图像中的产品区域，对初步图像中除该产品区域外的区域进行纯色填充，所得的图像作为产品图像。其中，可采用VisionPro(一种计算机式视觉软件)中的PMAlign工具实现初步图像中产品区域的定位和抠图。纯色填充可以是填充黑色或白色等。 在得到产品图像后，可将产品图像缩放至预设尺寸(以下称为第二预设尺寸以示区分)，再对缩放后的图像进行特征提取得到特征图。其中，第二预设尺寸可以是预先根据需要进行设置的，具体可根据产品的尺寸进行设置，例如，对于耳机振膜，可以将第二预设尺寸设置为2048*2048。
在一实施方式中，可将摄像机设置于生产线的缺陷检测工位处，当产品被传输或被放置于该工位处时，由摄像机拍摄产品图像，检测设备与摄像机通过有线或无线方式通信连接，从摄像机获取产品图像，对产品图像进行实施或离线地缺陷检测。
步骤S20，对特征图中的各锚点生成第一预设尺寸、预设长宽比例和预设旋转角度的锚框；
在得到特征图后，可将特征图中的各个点作为锚点，对各个锚点分别生成预设尺寸(以下称为第一预设尺寸以示区分)、预设长宽比例和预设旋转角度的锚框(anchor)。其中，预设尺寸、预设长宽比例和预设旋转角度均可以预先根据需要进行设置，预设尺寸、预设长宽比例和预设旋转角度分别都可以设置多个，那么，将尺寸、比例和旋转角度三个参数排列组合得到多个参数组，对于一个锚点而言，分别根据每个参数组生成一个锚框，得到该锚点对应的多个锚框。
本实施例在设置旋转角度时可以采用如下方式：
为生成的每一个锚框设置对应的量化旋转步长；具体的，根据计算量和计算效率设置量化旋转步长，量化旋转步长的数值越大，分出的旋转区间越大，计算量越大，速度越慢、效率低。必须权衡计算量和计算效率以选取合适的量化旋转步长。
各个锚框对应的量化旋转步长可以相同，比如，在[0，π)的旋转区间内将旋转区间平均分成4部分，即[0，kπ/4)(k＝1，2，3，4)时，每一次旋转使用的量化旋转步长相同为π/4；或者，各个锚框对应的量化旋转步长也可以不相同，例如数值递增或递减；或者，先将锚框标记，相同标记的锚框采用的量化旋转步长，不同标记的锚框采用的量化旋转步长不同。
具体地，对于一组参数组，可先按照参数组中的比例和尺寸生成一个参考方向的锚框，再将该锚框按照参数组中的旋转角度进行旋转，得到该参数组对应的锚框。例如，预设尺寸有2、4和8三个尺寸，预设长宽比例有长宽比1∶3、1∶5和1∶7三个比例，预设旋转角度有0度、45度、90度和135度四个旋转角度， 对于锚点A，以该锚点A作为锚框的中心点，生成3×3×4＝36个锚框。如图3所示，在M*N尺寸的特征图中，以长宽比1∶3、尺寸为2的锚框为例，一共生成0度、45度、90度和135度四个旋转角度的锚框。
其中，锚框的生成方式可按照Mask RCNN或Faster RCNN中RPN(Region Proposal Networks，区域建议网络)的锚框生成方法生成，相比于现有的RPN，增加了旋转角度，也即，将原本一定尺寸和一定比例的锚框，还按照一定的旋转角度进行旋转，从而得到带有方向分量的锚框。由于生成的各锚框是带有角度分量的锚框，不同旋转角度的锚框可以定位不同方向的缺陷，从而能够更加准确地识别各个方向的缺陷。对于细长型且方向复杂多变的缺陷，可通过设置长宽差异较大的比例和多个旋转角度，来生成多个方向的细长型锚框，对这些缺陷进行识别，以增加缺陷检测准确率。
步骤S30，对各锚框进行分类得到各锚框是否包含缺陷的检测结果。
在生成锚框后，对各个锚框进行分类得到各锚框是否包含缺陷的检测结果。具体地，可采用常规的二分类算法来对各个锚框进行分类，得到是否包含包括的检测结果，或可将各个锚框分别输入到二分类器中进行分类，该二分类器输出该锚框是否包含缺陷的检测结果。二分类器可采用常用的分类器来实现，例如softmax。在本实施例中，目标产品的产品图像可以是用于训练网络模型的训练样本，对于多个产品图像，可对产品图像中的缺陷区域进行人工或机器标注，得到产品图像中真实缺陷区域的区域位置；将这些产品图像按照上述步骤进行处理得到各个锚框的检测结果；根据真实缺陷区域的区域位置和锚框的位置，确定各个锚框是否包含缺陷的真实结果；根据检测结果和真实结果计算分类损失函数，根据分类损失函数按照反向传播的方法对分类器和特征提取操作中使用的特征提取网络中的各个参数进行训练，以不断减小检测结果与真实结果之间的差距，从而提高检测结果的准确度；训练完成后，对于需要检测的产品图像，再按照上述步骤处理得到各个锚框的检测结果。其中，分类损失函数可采用常用的二分类损失函数。
在得到各个锚框的检测结果后，即可根据检测结果确定目标产品是否存在缺陷。具体地，若其中一个锚框的检测结果是包含缺陷，则可确定目标产品是存在缺陷的，若所有的锚框检测结果都是不存在缺陷，则可确定目标产品不存在确定。或者，当生产线上允许目标产品中存在少量缺陷时，可以设置一个阈值；当包含缺陷的锚框数量小于该阈值时，可确定目标产品不存在缺陷，将目 标产品送往下一工位进行继续加工或打包；当包含缺陷的锚框数量不大于该阈值时，可确定目标产品缺陷，并可输出该锚框的位置，以供生产人员复核该锚框位置是否存在缺陷。
在本实施例中，通过对目标产品的产品图像进行特征提取得到特征图，对特征图中的各个锚点生成预设长宽比例、预设尺寸和预设旋转角度的锚框，对各个锚框进行分类得到各个锚框是否包含缺陷的检测结果，以实现对目标产品是否包含缺陷的检测。由于本实施例中相比于现有的锚框生成方式还增加了预设旋转角度，生成带有角度分量的锚框，从而能够更充分地定位到产品中方向各异的细微缺陷，从而提高了产品的缺陷检测准确率。
进一步地，基于上述第一实施例，提出本发明产品缺陷检测方法的第二实施例，在本实施例中，步骤S30之后，还包括：
步骤S40，根据检测结果对各锚框中包含缺陷的正锚框进行边界回归计算，得到各正锚框对应的预测偏移量，其中，预测偏移量包括预测角度偏移量；
根据各个锚框的检测结果，将包含缺陷的锚框作为正锚框。在本实施例中，若存在正锚框，则可进一步地对正锚框进行边界回归计算，得到各正锚框对应的预测偏移量。其中，边界回归计算可以是采用RPN中的边界回归计算方法来实现，相比于RPN中对平移量和缩放尺度的回归预测，本实施例中，预测偏移量还包括预测角度偏移量，具体地，该预测角度偏移量可以是一个角度值，该角度值表示了正锚框与产品图像中真实缺陷区域之间的方向角度差值，当然该角度值只是一个预测值。预先可对通过对RPN进行训练，以使得边界回归计算得到的预测偏移量更接近于真实的偏移量。
步骤S50，根据正锚框和对应的预测偏移量生成产品图像中预测缺陷区域的区域位置。
在得到各个正锚框的预测角度偏移量后，可根据正锚框和对应的预测偏移量生成该产品图像中预测缺陷区域的区域位置。具体地，可将正锚框按照预测偏移量中的平移量进行平移，再按照预测偏移量中的缩放尺度进行缩放，再按照预测角度偏移量进行旋转，得到预测缺陷区域的区域位置。由于得到的区域位置是预测的结果，因此称为预测缺陷区域的区域位置。区域位置可以采用(x1，y1，x2，y2，α)来表示，其中，(x1，y1)是区域左上角顶点的坐标，(x2，y2)是区域右下角顶点的坐标，α是相对于参考方向的角度分量。
在得到缺陷区域的区域位置后，可将区域位置输出，或者在产品图像中标记出该区域位置，并将经过标记的产品图像输出，以便于生产人员根据输出结果来对目标产品的缺陷进行分析。
在本实施例中，在得到各个锚框的检测结果后，将包含缺陷的锚框作为正锚框，对正锚框进行边界回归计算，得到正锚框对应的预测偏移量，预测偏移量中包括预测角度偏移量，根据正锚框和预测角度偏移量得到预测缺陷区域的区域位置，实现了对目标产品中方向各异的缺陷区域的精确定位，使得生产人员能够获取到目标产品中缺陷区域的准确位置。
进一步地，步骤S50之后，还包括：
步骤S60，对预测缺陷区域进行分类得到预测缺陷区域的缺陷类型。
在得到预测缺陷区域后，还可进一步地对预测缺陷区域进行分类，得到该缺陷区域的缺陷类型。具体地，产品可能存在不同类型的缺陷，例如毛纤杂质、划伤和边缘粘接不良等，因此，可以对预测缺陷区域进行进一步地分类，以确定该区域是属于哪一缺陷类型。分类可以采用Mask RCNN或Faster RCNN中的多分类网络来实现，在本实施例中不再详细赘述。
在得到预测缺陷区域的缺陷类型后，可将缺陷类型和区域位置一并输出，或将缺陷类型也标注在产品图像中，将标记后的产品图像输出，以便于生产人员能够方便地获知目标产品中的缺陷位置和缺陷类型。
进一步地，还可以按照Mask RCNN中的掩码分支，对预测缺陷区域进行实例分割，得到预测缺陷区域的掩码图，掩码图中各个点对应表示预测缺陷区域中的各个点是否是缺陷点，将掩码图直接输出或与目标产品的产品图像融合后输出，以使得生产人员能够方便地获知目标产品中缺陷的具体位置。
进一步地，步骤S50之后，还包括：
步骤S70，计算正锚框相对于产品图像中预标注的真实缺陷区域的真实偏移量，其中，真实偏移量包括真实角度偏移量；
步骤S80，根据预测偏移量和真实偏移量计算损失函数，并根据损失函数更新边界回归计算中采用的回归参数。
在一实施方式中，目标产品的产品图像可以是用于训练网络模型的训练样本。对于多个产品图像，可对产品图像中的缺陷区域进行人工或机器标注，得到产品图像中真实缺陷区域的区域位置；将这些产品图像按照上述步骤进行处理得到正锚框对应的预测偏移量后，可根据真实缺陷区域的区域位置，计算正 锚框相对于真实缺陷框的真实偏移量，真实偏移量也同样包括真实角度偏移量；根据该预测偏移量和该真实偏移量计算损失函数(以下称为回归损失函数以示区分)，根据回归损失函数按照反向传播的方法对边界回归计算中采用的回归参数和特征提取网络中的各个参数进行训练(更新)，以不断减小预测偏移量和真实偏移量之间的差距，从而提高预测缺陷区域的检测准确率。进一步地，也可以将回归损失函数和分类损失函数作为一个总的损失函数来对网络中的各部分参数进行训练。网络训练过程可按照常规的机器学习模型训练方法来实现，在本实施例中不再详细赘述。
其中，回归损失函数可采用RPN中边界回归计算常用的损失函数，例如smooth L1 loss，由于本实施例中锚框是带有角度分量的，预测偏移量也包括预测角度偏移量，因此损失函数中还包括对角度分量的预测损失。例如，回归损失函数L reg采用smooth L1 loss结合角度分量的预测损失，表示如下：
Figure PCTCN2021126935-appb-000001
Figure PCTCN2021126935-appb-000002
其中，t是正锚框，t*是该正锚框对应的真实缺陷区域，(x1，y1)表示左上角顶点坐标，(x2，y2)表示右下角顶点坐标，α表示角度分量。
在训练完成后，再按照上述步骤对需要检测的产品的产品图像进行检测，得到产品图像是否包含缺陷的检测结果，以及得到包含缺陷时缺陷区域的区域位置和缺陷类型。
在本实施例中，通过设置锚框的预设旋转角度，生成带有角度分量的锚框，并在回归损失函数中加入角度分量的预测损失，使得网络训练过程中能够训练不同角度分量的锚框识别缺陷的能力，保持不同方向的缺陷特征的独立性，从而提升了识别产品表面方向各异的细微缺陷的准确率。
在一实施方式中，以对振膜产品进行缺陷检测为例，如图4所示，首先使用工业相机成像系统拍摄带有缺陷的振膜图像，再使用VisionPro中的PMAlign工具实现振膜图像的定位和抠图，将获取的图像去除掉除振膜产品以外的背景，抠掉的背景以纯黑色填充，避免背景部分对后续的缺陷检测造成干扰；然后使用resize函数统一将图像缩放成2048*2048大小，输入进入Mask-RCNN网络，先经过网络中ResNet101+FPN(残差网络+特征金字塔网络)进行特征提取得到 特征图；再对特征图生成带有角度分量的anchors；将带有角度分量的anchors输入RPN网络；从RPN分出两支网络：RPN-Class和RPN_BBox，其中RPN-Class使用softmax分类判断anchors是否存在缺陷，RPN_BBox用于边界回归预测anchors的偏移量；计算分类损失和回归损失，通过反向传播算法更新网络中参数，以完成网络训练；将待检测的振膜图像输入训练完成网络中，得到振膜图像是否存在缺陷的检测结果，以及存在缺陷时缺陷区域的位置。
进一步地，基于上述第一和/或第二实施例，提出本发明产品缺陷检测方法的第三实施例，在本实施例中，步骤S10之前，还包括：
步骤A10，对目标产品的产品图像进行对称性检测，以确定目标产品是否属于轴对称产品；
步骤A20，若目标产品属于轴对称产品，则从0到180度之间选择预设旋转角度；
步骤A30，若目标产品不属于轴对称产品，则从0到360度之间选择预设旋转角度。
在本实施例中，可预先对目标产品的产品图像进行对称性检测，以确定目标产品是否属于轴对称产品。其中，对称性检测可以是将产品图像以一条或多条分割线进行切分得到两张图像，将其中一个图像以该分割线为轴进行翻转后，与另一张图像进行图像匹配得到一个相似度，若该相似度大于一定的阈值，例如大于99％，则确定该目标产品是轴对称产品。或者，在其他实施方式中，也可以是采用其他方式来进行对称性检测。
若目标产品属于轴对称产品，则可以从0到180度之间选择预设旋转角度。具体地，可将0到180度这一区间等分或非等分地划分为多个区间，以每个区间的前端点作为预设旋转角度。例如，等分为4个区间，得到[0，45)、[45，90)、[90，135)和[135，180)4个区间，将这4个区间的前端点0、45、90和135作为预设旋转角度。其中，区间数量可以是预先根据需要进行设置，也即选取一个量化旋转步长；当对缺陷检测准确率要求更高时，区间数量可设置得较多，也即量化旋转步长较小，从而生成方向间隔更紧凑的锚框，定位更多的方向；但区间数量设置得越多缺陷检测过程中的计算量越大，计算速度越慢，因此，可以根据计算量和准确率的需求，设置一个居中的区间数量。
若目标产品不属于轴对称产品，则可以从0到360度之间选择预设旋转角 度。具体地，同样可将0到360度这一区间划分为多个区间，以每个区间的前端点作为预设旋转角度。区间数量也可以是预先设置。
在本实施例中，通过对产品图像进行对称性分析，针对轴对称产品从0到180度之间选择旋转角度，针对非轴对称产品从0到360度之间选择旋转角度，以使得锚框旋转角度可以根据产品形态进行设置，以使得锚框可以覆盖产品不同方向的缺陷。
进一步地，步骤S30之前，还包括：
步骤A40，分别计算各锚框与产品图像的交并比；
在本实施例中，在生成各个锚框后，可以分别计算各个锚框与产品图像的交并比，得到各个锚框对应的交并比。其中，交并比计算方法即计算锚框与产品图像的重叠区域的面积，再除以锚框和产品图像的合并区域的面积。
步骤S30包括：
步骤S301，对各锚框中对应的交并比大于预设值的目标锚框进行分类，得到目标锚框是否包含缺陷的检测结果。
将各个锚框中对应的交并比大于预设值的锚框作为目标锚框，对目标锚框进行分类，得到目标锚框是否包含缺陷的检测结果。其中，预设值可以根据需要进行设置，例如0.7。也即，在本实施例中，并不是对所有的锚框进行分类，而是只对与产品图像重叠区域较多的锚框进行分类。与产品图像交并比较小的锚框一般处于产品图像的边缘区域，这一类锚框往往难以学习到产品缺陷特征，故可将这类锚框进行剔除，以在不影响产品缺陷检测的基础上对减小计算量，提高检测效率。
此外，此外本发明实施例还提出一种产品缺陷检测装置，参照图4，该装置包括：
提取模块10，用于对目标产品的产品图像进行特征提取得到特征图；
生成模块20，用于对特征图中的各锚点生成第一预设尺寸、预设长宽比例和预设旋转角度的锚框；
分类模块30，用于对各锚框进行分类得到各锚框是否包含缺陷的检测结果。
进一步地，上述装置还包括：
回归模块，用于根据检测结果对各锚框中包含缺陷的正锚框进行边界回归计算，得到各正锚框对应的预测偏移量，其中，预测偏移量包括预测角度偏移 量；
位置生成模块，用于根据正锚框和对应的预测偏移量生成产品图像中预测缺陷区域的区域位置。
进一步地，上述装置还包括：
缺陷分类模块，用于对预测缺陷区域进行分类得到预测缺陷区域的缺陷类型。
进一步地，上述装置还包括：
第一计算模块，用于计算正锚框相对于产品图像中预标注的真实缺陷区域的真实偏移量，其中，真实偏移量包括真实角度偏移量；
更新模块，用于根据预测偏移量和真实偏移量计算损失函数，并根据损失函数更新边界回归计算中采用的回归参数。
进一步地，上述装置还包括：
检测模块，用于对目标产品的产品图像进行对称性检测，以确定目标产品是否属于轴对称产品；
第一选择模块，用于若目标产品属于轴对称产品，则从0到180度之间选择预设旋转角度；
第二选择模块，用于若目标产品不属于轴对称产品，则从0到360度之间选择预设旋转角度。
进一步地，上述装置还包括：
第二计算模块，用于分别计算各锚框与产品图像的交并比；
分类模块30还用于：
对各锚框中对应的交并比大于预设值的目标锚框进行分类，得到目标锚框是否包含缺陷的检测结果。
进一步地，提取模块10包括：
定位单元，用于采集目标产品的初步图像，并定位初步图像中的产品区域；
填充单元，用于对初步图像中除产品区域外的区域进行纯色填充得到产品图像；
提取单元，用于将产品图像缩放至第二预设尺寸后进行特征提取得到特征图。
本发明产品缺陷检测装置的具体实施方式的拓展内容与上述产品缺陷检测方法各实施例基本相同，在此不做赘述。
此外，本发明实施例还提出一种计算机可读存储介质，该计算机存储介质上存储有产品缺陷检测程序。产品缺陷检测程序被处理器执行时实现上述任一方法实施例中的产品缺陷检测方法的步骤。
本发明产品缺陷检测设备和计算机可读存储介质的被执行时实现的具体功能，均可参照本发明产品缺陷检测方法各个实施例，此处不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个......”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
上述本发明实施例序号仅仅为了描述，不代表实施例的优劣。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本发明各个实施例所述的方法。
以上仅为本发明的优选实施例，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。

Claims (13)

  1. 一种产品缺陷检测方法，其中，所述方法包括以下步骤：
    对目标产品的产品图像进行特征提取得到特征图；
    对所述特征图中的各锚点生成第一预设尺寸、预设长宽比例和预设旋转角度的锚框；
    对各所述锚框进行分类得到各所述锚框是否包含缺陷的检测结果。
  2. 如权利要求1所述的产品缺陷检测方法，其中，所述对各所述锚框进行分类得到各所述锚框是否包含缺陷的检测结果的步骤之后，还包括：
    根据所述检测结果对各所述锚框中包含缺陷的正锚框进行边界回归计算，得到各所述正锚框对应的预测偏移量，其中，所述预测偏移量包括预测角度偏移量；
    根据所述正锚框和对应的预测偏移量生成所述产品图像中预测缺陷区域的区域位置。
  3. 如权利要求2所述的产品缺陷检测方法，其中，所述根据所述正锚框和对应的预测偏移量生成所述产品图像中缺陷区域的区域位置的步骤之后，还包括：
    对所述预测缺陷区域进行分类得到所述预测缺陷区域的缺陷类型。
  4. 如权利要求2所述的产品缺陷检测方法，其中，所述根据所述正锚框和对应的预测偏移量生成所述产品图像中预测缺陷区域的区域位置的步骤之后，还包括：
    计算所述正锚框相对于所述产品图像中预标注的真实缺陷区域的真实偏移量，其中，所述真实偏移量包括真实角度偏移量；
    根据所述预测偏移量和所述真实偏移量计算损失函数，并根据所述损失函数更新所述边界回归计算中采用的回归参数。
  5. 如权利要求1所述的产品缺陷检测方法，其中，所述对目标产品的产品图像进行特征提取得到特征图的步骤之前，还包括：
    对所述目标产品的产品图像进行对称性检测，以确定所述目标产品是否属 于轴对称产品；
    若所述目标产品属于轴对称产品，则从0到180度之间选择所述预设旋转角度；
    若所述目标产品不属于轴对称产品，则从0到360度之间选择所述预设旋转角度。
  6. 如权利要求1所述的产品缺陷检测方法，其中，所述对各所述锚框进行分类得到各所述锚框是否包含缺陷的检测结果的步骤之前，还包括：
    分别计算各所述锚框与所述产品图像的交并比；
    所述对各所述锚框进行分类得到各所述锚框是否包含缺陷的检测结果的步骤包括：
    对各所述锚框中对应的交并比大于预设值的目标锚框进行分类，得到所述目标锚框是否包含缺陷的检测结果。
  7. 如权利要求1所述的产品缺陷检测方法，其中，所述对目标产品的产品图像进行特征提取得到特征图的步骤包括：
    采集目标产品的初步图像，并定位所述初步图像中的产品区域；
    对所述初步图像中除所述产品区域外的区域进行纯色填充得到产品图像；
    将所述产品图像缩放至第二预设尺寸后进行特征提取得到特征图。
  8. 一种产品缺陷检测装置，其中，所述装置包括：
    提取模块，用于对目标产品的产品图像进行特征提取得到特征图；
    生成模块，用于对所述特征图中的各锚点生成第一预设尺寸、预设长宽比例和预设旋转角度的锚框；
    分类模块，用于对各所述锚框进行分类得到各所述锚框是否包含缺陷的检测结果。
  9. 如权利要求8所述的产品缺陷检测装置，其中，所述装置还包括：
    回归模块，用于根据所述检测结果对各所述锚框中包含缺陷的正锚框进行边界回归计算，得到各所述正锚框对应的预测偏移量，其中，所述预测偏移量包括预测角度偏移量；
    位置生成模块，用于根据所述正锚框和对应的预测偏移量生成所述产品图像中预测缺陷区域的区域位置。
  10. 如权利要求8所述的产品缺陷检测装置，其中，所述装置还包括：
    检测模块，用于对所述目标产品的产品图像进行对称性检测，以确定所述目标产品是否属于轴对称产品；
    第一选择模块，用于若所述目标产品属于轴对称产品，则从0到180度之间选择所述预设旋转角度；
    第二选择模块，用于若所述目标产品不属于轴对称产品，则从0到360度之间选择所述预设旋转角度。
  11. 如权利要求8所述的产品缺陷检测装置，其中，所述装置还包括：
    第二计算模块，用于分别计算各所述锚框与所述产品图像的交并比；
    所述分类模块还用于：
    对各所述锚框中对应的交并比大于预设值的目标锚框进行分类，得到所述目标锚框是否包含缺陷的检测结果。
  12. 一种产品缺陷检测设备，其中，所述产品缺陷检测设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的产品缺陷检测程序，所述产品缺陷检测程序被所述处理器执行时实现下述的产品缺陷检测方法：
    对目标产品的产品图像进行特征提取得到特征图；
    对所述特征图中的各锚点生成第一预设尺寸、预设长宽比例和预设旋转角度的锚框；
    对各所述锚框进行分类得到各所述锚框是否包含缺陷的检测结果。
  13. 一种计算机可读存储介质，其中，所述计算机可读存储介质上存储有产品缺陷检测程序，所述产品缺陷检测程序被处理器执行时实现如下的产品缺陷检测方法：
    对目标产品的产品图像进行特征提取得到特征图；
    对所述特征图中的各锚点生成第一预设尺寸、预设长宽比例和预设旋转角度的锚框；
    对各所述锚框进行分类得到各所述锚框是否包含缺陷的检测结果。
PCT/CN2021/126935 2021-01-05 2021-10-28 产品缺陷检测方法、装置、设备及计算机可读存储介质 WO2022148109A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110009783.XA CN112712513A (zh) 2021-01-05 2021-01-05 产品缺陷检测方法、装置、设备及计算机可读存储介质
CN202110009783.X 2021-01-05

Publications (1)

Publication Number Publication Date
WO2022148109A1 true WO2022148109A1 (zh) 2022-07-14

Family

ID=75548322

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/126935 WO2022148109A1 (zh) 2021-01-05 2021-10-28 产品缺陷检测方法、装置、设备及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN112712513A (zh)
WO (1) WO2022148109A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116563237A (zh) * 2023-05-06 2023-08-08 大连工业大学 一种基于深度学习的鸡胴体缺陷高光谱图像检测方法
CN117351062A (zh) * 2023-12-04 2024-01-05 尚特杰电力科技有限公司 风机扇叶缺陷诊断方法、装置、系统及电子设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112712513A (zh) * 2021-01-05 2021-04-27 歌尔股份有限公司 产品缺陷检测方法、装置、设备及计算机可读存储介质
CN114187269B (zh) * 2021-12-07 2024-04-05 北京工业大学 小零器件表面缺陷边缘快速检测方法
WO2023218441A1 (en) * 2022-05-11 2023-11-16 Inspekto A.M.V. Ltd. Optimizing a reference group for visual inspection

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130336575A1 (en) * 2012-06-13 2013-12-19 Applied Materials Israel Ltd. System, method and computer program product for detection of defects within inspection images
CN108875577A (zh) * 2018-05-11 2018-11-23 深圳市易成自动驾驶技术有限公司 目标检测方法、装置及计算机可读存储介质
CN110163841A (zh) * 2019-04-12 2019-08-23 中科院微电子研究所昆山分所 物体表面缺陷的检测方法、装置、设备及存储介质
CN110414464A (zh) * 2019-08-05 2019-11-05 北京深醒科技有限公司 一种小尺度密集行人检测方法
US20190347828A1 (en) * 2018-05-09 2019-11-14 Beijing Kuangshi Technology Co., Ltd. Target detection method, system, and non-volatile storage medium
CN112085735A (zh) * 2020-09-28 2020-12-15 西安交通大学 一种基于自适应锚框的铝材质图像缺陷检测方法
CN112712513A (zh) * 2021-01-05 2021-04-27 歌尔股份有限公司 产品缺陷检测方法、装置、设备及计算机可读存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110555842A (zh) * 2019-09-10 2019-12-10 太原科技大学 一种基于锚点集优化的硅片图像缺陷检测方法
CN111539428A (zh) * 2020-05-06 2020-08-14 中国科学院自动化研究所 基于多尺度特征整合和注意力机制的旋转目标检测方法
CN111798447B (zh) * 2020-07-18 2023-03-10 太原理工大学 一种基于Faster RCNN的深度学习塑化材料缺陷检测方法
CN111881846B (zh) * 2020-07-30 2024-04-02 北京市商汤科技开发有限公司 图像处理方法和相关装置、设备、存储介质
CN111986174A (zh) * 2020-08-17 2020-11-24 深圳市商汤科技有限公司 缺陷检测方法、装置、电子设备及计算机存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130336575A1 (en) * 2012-06-13 2013-12-19 Applied Materials Israel Ltd. System, method and computer program product for detection of defects within inspection images
US20190347828A1 (en) * 2018-05-09 2019-11-14 Beijing Kuangshi Technology Co., Ltd. Target detection method, system, and non-volatile storage medium
CN108875577A (zh) * 2018-05-11 2018-11-23 深圳市易成自动驾驶技术有限公司 目标检测方法、装置及计算机可读存储介质
CN110163841A (zh) * 2019-04-12 2019-08-23 中科院微电子研究所昆山分所 物体表面缺陷的检测方法、装置、设备及存储介质
CN110414464A (zh) * 2019-08-05 2019-11-05 北京深醒科技有限公司 一种小尺度密集行人检测方法
CN112085735A (zh) * 2020-09-28 2020-12-15 西安交通大学 一种基于自适应锚框的铝材质图像缺陷检测方法
CN112712513A (zh) * 2021-01-05 2021-04-27 歌尔股份有限公司 产品缺陷检测方法、装置、设备及计算机可读存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116563237A (zh) * 2023-05-06 2023-08-08 大连工业大学 一种基于深度学习的鸡胴体缺陷高光谱图像检测方法
CN116563237B (zh) * 2023-05-06 2023-10-20 大连工业大学 一种基于深度学习的鸡胴体缺陷高光谱图像检测方法
CN117351062A (zh) * 2023-12-04 2024-01-05 尚特杰电力科技有限公司 风机扇叶缺陷诊断方法、装置、系统及电子设备
CN117351062B (zh) * 2023-12-04 2024-02-23 尚特杰电力科技有限公司 风机扇叶缺陷诊断方法、装置、系统及电子设备

Also Published As

Publication number Publication date
CN112712513A (zh) 2021-04-27

Similar Documents

Publication Publication Date Title
WO2022148109A1 (zh) 产品缺陷检测方法、装置、设备及计算机可读存储介质
CN106875381B (zh) 一种基于深度学习的手机外壳缺陷检测方法
CN111080693A (zh) 一种基于YOLOv3的机器人自主分类抓取方法
CN110175982B (zh) 一种基于目标检测的缺陷检测方法
WO2019080203A1 (zh) 一种机器人的手势识别方法、系统及机器人
CN111815564B (zh) 一种检测丝锭的方法、装置及丝锭分拣系统
CN110598698B (zh) 基于自适应区域建议网络的自然场景文本检测方法和系统
CN114299066B (zh) 基于显著性特征预提取与图像分割的缺陷检测方法和装置
CN112613097A (zh) 一种基于计算机视觉的bim快速化建模方法
CN112528913A (zh) 一种基于图像的砂石颗粒物粒径检测分析系统
CN109598298B (zh) 图像物体识别方法和系统
WO2022147966A1 (zh) 晶圆检测方法、装置、设备及存储介质
WO2022002262A1 (zh) 基于计算机视觉的字符序列识别方法、装置、设备和介质
US20210390282A1 (en) Training data increment method, electronic apparatus and computer-readable medium
CN111696079A (zh) 一种基于多任务学习的表面缺陷检测方法
CN115880298A (zh) 一种基于无监督预训练的玻璃表面缺陷检测方法及系统
CN116228780B (zh) 基于计算机视觉的硅片缺陷检测方法及系统
WO2024055530A1 (zh) 一种图像的目标检测方法、系统、设备及存储介质
CN115937626A (zh) 基于实例分割的半虚拟数据集自动生成方法
CN115830004A (zh) 表面缺陷检测方法、装置、计算机设备和存储介质
CN113052234A (zh) 一种基于图像特征和深度学习技术的玉石分类方法
CN113362221A (zh) 用于门禁的人脸识别系统与人脸识别方法
CN116229236A (zh) 一种基于改进YOLO v5模型的结核杆菌检测方法
WO2019223763A1 (zh) 图像检测方法、装置、设备及介质，图案化控制系统及方法
CN114913129A (zh) 一种针对复合材料铺放缺陷识别定位系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21917151

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21917151

Country of ref document: EP

Kind code of ref document: A1