WO2020211823A1 - 模型训练方法、装置及缺陷检测方法、装置 - Google Patents

模型训练方法、装置及缺陷检测方法、装置 Download PDF

Info

Publication number
WO2020211823A1
WO2020211823A1 PCT/CN2020/085205 CN2020085205W WO2020211823A1 WO 2020211823 A1 WO2020211823 A1 WO 2020211823A1 CN 2020085205 W CN2020085205 W CN 2020085205W WO 2020211823 A1 WO2020211823 A1 WO 2020211823A1
Authority
WO
WIPO (PCT)
Prior art keywords
area
label
training sample
defect
probability value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/085205
Other languages
English (en)
French (fr)
Inventor
陈佳伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Publication of WO2020211823A1 publication Critical patent/WO2020211823A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0008—Industrial image inspection checking presence/absence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/77—Determining position or orientation of objects or cameras using statistical methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30124—Fabrics; Textile; Paper

Definitions

  • This application relates to the field of computer technology, in particular to a model training method and device, and a defect detection method and device.
  • ASI Automated Surface Inspection
  • the defect detection is carried out by using a neural network based on deep learning, but the detection error rate is relatively high.
  • the present application provides a defect detection method and device to solve the problem of relatively high detection error detection rate in related technologies.
  • a model training method including:
  • the labels include at least a first label and a second label
  • the first label is used to record that the first area marked in the training sample is an area prone to misdetection
  • the second label Used to record that the marked second area in the training sample is a defective area
  • a detection model for detecting defects is trained.
  • a model training device including:
  • the acquisition module is used to acquire multiple frames of labeled training samples, where the labels include at least a first label and a second label, and the first label is used to record that a marked first area in the training sample is an area prone to misdetection, The second label is used to record that the marked second area in the training sample is a defective area;
  • the training module is used to train a detection model for detecting defects by using each labeled training sample and the location information of the first region and the location information of the second region in each training sample.
  • a defect detection method using a detection model trained by the model training method described in the first aspect including:
  • a defect detection device using a detection model trained by the model training method described in the first aspect including:
  • the detection module is configured to input the target image to be detected into the detection model, so that the target image to be detected is detected by the detection model to obtain the target defect probability value of each pixel in the target image to be detected;
  • the determining module is configured to determine whether there is a defect area in the target image to be detected according to the target defect probability value of each pixel in the target image to be detected.
  • a method for obtaining defect detection samples including:
  • the area is determined to be an area prone to misdetection, and the area is marked with the first label.
  • each acquired training sample carries a second label
  • the second label is used to record that the marked second area in the training sample is a defective area.
  • a device for acquiring a defect detection sample including:
  • the detection module is used to detect the defect area in each training sample obtained
  • the labeling module is used to determine that in the detected defect area, if there is an area that does not overlap with the second area marked with the second label, the area is determined to be an area prone to misdetection, and the area is used as the first label Mark to obtain training samples with the first label and the second label;
  • each acquired training sample carries a second label
  • the second label is used to record that the marked second area in the training sample is a defective area.
  • Fig. 1A is a diagram of a chopstick to be tested according to an exemplary embodiment of the application
  • Fig. 1B is a defect diagram of chopsticks obtained by using related technologies according to an exemplary embodiment of the present application
  • Fig. 2A is a flowchart of an embodiment of a model training method according to an exemplary embodiment of this application;
  • FIG. 2B is a schematic diagram showing a type of cloth defect according to the embodiment shown in FIG. 2A in this application;
  • FIG. 2C is a schematic diagram of a model training structure according to the embodiment shown in FIG. 2A according to this application;
  • FIG. 2D is a schematic diagram of another model training structure according to the embodiment shown in FIG. 2A according to this application;
  • Fig. 3A is a flowchart of a defect detection method according to an exemplary embodiment of the application.
  • FIG. 3B is a schematic diagram of a defect detection structure according to the embodiment shown in FIG. 3A;
  • Fig. 3C is a defect diagram of the chopsticks shown in Fig. 1A according to an exemplary embodiment of this application;
  • Fig. 3D is a picture of a cloth to be inspected according to an exemplary embodiment of this application.
  • FIG. 3E is a defect diagram of the cloth to be inspected shown in FIG. 3D obtained according to the embodiment shown in FIG. 3A in this application;
  • Fig. 4 is a flow chart of a method for obtaining defect detection samples according to an exemplary embodiment of the application
  • Fig. 5 is a hardware structure diagram of an electronic device according to an exemplary embodiment of the application.
  • Fig. 6 is a structural diagram of a model training device according to an exemplary embodiment of the application.
  • Fig. 7 is a structural diagram of a defect detection device according to an exemplary embodiment of the present application.
  • Fig. 8 is a structural diagram of a device for acquiring a defect detection sample according to an exemplary embodiment of the application.
  • first, second, third, etc. may be used in this application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as second information, and similarly, the second information may also be referred to as first information.
  • word “if” as used herein can be interpreted as "when” or “when” or "in response to determination”.
  • the training process of the neural network based on deep learning currently used is: obtain a sample image of the target object, and mark the defective area and the non-defective area in each frame of the sample image with different labels, and then use the sample image and the labeled label Train the neural network model.
  • the detection results obtained by the currently trained network model have over-fitting problems, resulting in a relatively high false detection rate.
  • Fig. 1A is the image to be inspected with chopsticks as the target object.
  • the image to be inspected is input to the trained network model, and the image to be inspected is inspected by the network model, and the defect map determined by the inspection result is shown in Fig.
  • the black areas are non-defective areas, and the white areas are defective areas, but the white areas enclosed by dashed circles: Area 1, Area 2, Area 3, Area 4, and Area 5 are all misdetected It is the defect area.
  • this application proposes a model training method by acquiring multiple frames of labeled training samples.
  • the labels include at least a first label and a second label.
  • the first label is used to record the marked first label in the training sample.
  • One area is easy to be misdetected
  • the second label is used to record the marked second area in the training sample as a defective area, and then use each labeled training sample and the position information of the first area in each training sample,
  • the location information of the second area is used to train a detection model for detecting defects.
  • the detection model when training the detection model, by adding the tags of the areas that are prone to false detections, the learning of the features of the false detection areas is enhanced according to the tags and location information of the areas that are prone to false detections, so as to reduce the false detection of the detection model.
  • the purpose of improving the detection accuracy of the detection model when training the detection model, by adding the tags of the areas that are prone to false detections, the learning of the features of the false detection areas is enhanced according to the tags and location information of the areas that are prone to false detections, so as to reduce the false detection of the detection model.
  • Fig. 2A is a flowchart of an embodiment of a model training method according to an exemplary embodiment of this application.
  • the model training method includes the following steps:
  • Step 201 Obtain multiple frames of labeled training samples.
  • the labels include at least a first label and a second label.
  • the first label is used to record that the first area marked in the training sample is an area prone to misdetection, and the second label is used The second area marked in the recording training sample is a defective area.
  • training samples can be selected or captured according to actual detection requirements, and the objects in each training sample belong to the same category.
  • the defect area existing in the training sample can be detected.
  • the area is determined to be a false detection area, and the area is marked with a first label to obtain training samples with the first label and the second label.
  • the marking process of the second label used to mark the defective area can be realized by related technologies, which will not be described in detail here.
  • the labeled training sample may further include a third label, and the third label is used to record that the marked third area in the training sample is an area without defects.
  • This application does not limit the specific representation forms of the first label, the second label, and the third label, as long as three different areas can be distinguished.
  • the white areas enclosed by dashed lines: Area 1, Area 2, Area 3, Area 4, and Area 5 are areas that are easy to be misdetected, marked with the first label, and the rest are white
  • the area is a defective area, which is marked with a second label
  • the black area is a defective area, which is marked with a third label.
  • Step 202 Using each labeled training sample and the location information of the first region and the location information of the second region in each training sample, train a detection model for detecting defects.
  • each labeled training sample when the detection model is a single model, it is possible to combine each labeled training sample with the location information of the first region, the location information of the second region, and the first region in each training sample.
  • the location information of the three regions is input to the designated model training engine, so that the model training engine uses the input labeled training samples, the location information of the first region, the location information of the second region, and the location information of the third region for training Develop an inspection model for detecting defects.
  • the weight of the first label is greater than the weight of the second label and the weight of the third label, so that when performing gradient backhaul, more computing power can be used to learn areas that are prone to misdetection.
  • the weight of the first label is greater than the weight of the second label and the weight of the third label, so that when performing gradient backhaul, more computing power can be used to learn areas that are prone to misdetection.
  • the detection model when the detection model includes the first sub-detection model and the second sub-detection model, the position information of the first region in each training sample and each labeled training sample Input to the first model training engine, the first model training engine uses the input labeled training samples and the first region location information to train the first sub-detection model, and by combining each labeled training sample and each training sample
  • the location information of the second area and the location information of the third area are input to the second model training engine, so that the second model training engine uses the input labeled training samples and the location information of the second area and the location of the third area
  • the information trains the second sub-detection model.
  • the output of the first sub-detection model is the probability value of each pixel in the training sample that is easy to distinguish defects.
  • the output of the second sub-detection model Is the probability value of defects in each pixel in the training sample. The larger the value, the greater the probability of defects.
  • the first sub-detection model can be trained through a deep network learning method, and the adopted network structure can be the structure of a convolutional neural network.
  • the convolutional neural network can include a convolutional layer, a pooling layer, and BN (Batch Normalization). , Batch normalization) layer, fully connected layer and other computing layers.
  • the second sub-detection model can be obtained by the same method as the first sub-detection model, but it can also be obtained by a method different from the first sub-detection model, which is not limited in this application.
  • multiple frames of labeled training samples are obtained, and the labels include at least a first label and a second label, and the first label is used to record that the first area marked in the training sample is a false detection area
  • the second label is used to record that the marked second area in the training sample is a defective area, and then use each labeled training sample and the position information of the first area and the second area in each training sample to train Develop an inspection model for detecting defects.
  • the detection model when training the detection model, by adding the tags of the areas that are prone to false detections, the learning of the features of the false detection areas is enhanced according to the tags and location information of the areas that are prone to false detections, so as to reduce the false detection of the detection model.
  • the purpose of improving the detection accuracy of the detection model when training the detection model, by adding the tags of the areas that are prone to false detections, the learning of the features of the false detection areas is enhanced according to the tags and location information of the areas that are prone to false detections, so as to reduce the false detection of the detection model.
  • Fig. 3A is a flowchart of an embodiment of a defect detection method according to an exemplary embodiment of this application. This embodiment is based on the embodiment shown in Fig. 2A and applies the detection model to realize defect detection. The method includes the following steps:
  • Step 301 Input the target image to be detected into the detection model, and the target image to be detected is detected by the detection model to obtain the target defect probability value of each pixel in the target image to be detected.
  • the detection model when the detection model includes a first sub-detection model and a second sub-detection model, the image of the object to be detected may be input into the first sub-detection model, so that the second A sub-detection model detects the target image to be detected, and obtains the first candidate probability value of each pixel in the target image to be detected.
  • the first candidate probability value is used to indicate the probability of easily distinguishing defects.
  • the target image to be detected is also input into a second sub-detection model, so that the second sub-detection model detects the target image to be detected to obtain a second candidate for each pixel in the target image to be detected
  • the probability value, the second candidate probability value is used to indicate the probability of a defect, and then the first candidate probability value and the second candidate probability value of each pixel are fused to obtain the target defect probability value of each pixel.
  • the detection result output by the first sub-detection model represents the probability that each pixel in the target image to be detected is easy to distinguish defects
  • the larger the value the easier it is to distinguish defects, and the smaller the value, the more difficult it is to distinguish defects.
  • the detection result output by the second sub-detection model represents the probability that each pixel in the target image to be detected has a defect. The larger the value, the greater the probability that there is a defect. Therefore, the detection of the output of the first sub-detection model is required.
  • the result is fused with the detection result output by the second sub-detection model to obtain the target defect probability value of each pixel.
  • the first fusion method For each pixel, the average value of the first candidate probability value and the second candidate probability value of the pixel can be used as the target defect probability value of the pixel.
  • Formula 1 is as follows:
  • Target defect probability value 0.5* (first candidate probability value + second candidate probability value)
  • the second fusion method for each pixel, if the first candidate probability value of the pixel is less than the preset value, then the first candidate probability value of the pixel is used as the target defect probability value of the pixel, if The first candidate probability value of the pixel is not less than the preset value, and the second candidate probability value of the pixel is taken as the target defect probability value of the pixel.
  • Formula 2 is as follows:
  • the first candidate probability value of the pixel is less than the preset value, it means that the pixel is more difficult to distinguish as a defect (that is, the pixel has a high probability of not being a defect).
  • the second candidate probability of the pixel The reliability of the value is not high, and it is likely to be the wrong detection result. Therefore, the first candidate probability value is selected as the target defect probability value of the pixel; if the first candidate probability value of the pixel is not less than the preset value, it means The pixel is easier to distinguish defects (that is, the pixel has a high probability of being a defect). In this case, the second candidate probability value of the pixel is more reliable, so the second candidate probability value is selected as the pixel Point’s target defect probability value.
  • the preset value is 0.5
  • the first candidate probability value is taken as the target defect probability value
  • the second candidate probability value is taken as the target defect probability value
  • the target defect probability value of each pixel in the target image to be detected is obtained by using the combination of the first sub-detection model and the second sub-detection model. Because the first sub-detection model is used to detect whether each pixel is It is easy to distinguish defects, and by fusing the detection results of the first sub-detection model with the detection results of the second sub-detection model, it is possible to avoid misdetecting pixels that are not easily distinguishable as defects.
  • Step 302 Determine whether there is a defect area in the target image to be detected according to the target defect probability value of each pixel in the target image to be detected.
  • the target defect probability value of each pixel in the target image to be detected may be converted into a defect image to facilitate the judgment of the defect area.
  • the conversion process can be: determine the gray value of the target defect probability value mapping of each pixel in the target image to be detected, and use the gray value of the target defect probability value mapping of each pixel in the target image to be detected to generate the target to be detected The defect diagram corresponding to the diagram.
  • the target defect probability value of the pixel has a value range of 0 to 1
  • the defect map uses a gray value
  • the value range is 0 to 255, so it is necessary to convert the target defect probability value of the pixel to
  • the gray value used in the defect map, and the mapping relationship of the gray value used in the conversion from the target defect probability value to the defect map can be set in advance based on practical experience, as long as the greater the target defect probability value, the larger the mapped gray value. .
  • an area formed by pixels with a gray value greater than a preset threshold in the defect map may be determined as a defective area.
  • the image to be inspected shown in FIG. 1A is input into the inspection model, and the image to be inspected is inspected by the inspection model to obtain the target defect probability value of each pixel in the image to be inspected. Then the target defect probability value of each pixel in the image to be inspected is converted into a defect map, as shown in Fig. 3C.
  • the black area is the defect-free area
  • the white area is the defect area, which is similar to the defect map obtained by the related technology.
  • the false detection area in Figure 1B is filtered out by the detection model.
  • FIG. 3D it is the cloth image to be inspected.
  • the cloth image to be inspected is input into the inspection model, and the cloth image to be inspected is inspected by the inspection model to obtain the cloth image to be inspected.
  • the target defect probability value of each pixel in the figure, and then the target defect probability value of each pixel in the cloth image to be inspected is converted into a defect map, as shown in the defect map of the cloth as shown in Figure 3E, the black area is a defect-free area.
  • White areas are defective areas.
  • the defect detection model is trained by training samples with additional labels of areas that are prone to false detections, the labels and location information of the areas that are prone to false detections can strengthen the learning of the characteristics of the false detection areas.
  • the defect detection error rate realized by the detection model is low, and the detection accuracy is high.
  • Fig. 4 is a flowchart of an embodiment of a method for obtaining defect detection samples according to an exemplary embodiment of the present application.
  • the method for obtaining defect detection samples includes the following steps:
  • Step 401 For each acquired training sample, detect the defect area in the training sample.
  • each acquired training sample carries a second label
  • the second label is used to record that the marked second area in the training sample is a defective area.
  • the defect area in the detection training sample can be detected by the detection model used in the related technology, so that the detected defect area and the marked defect area can be distinguished and compared to obtain easy misdetection. Area.
  • Step 402 In the detected defect area, if there is an area that does not overlap with the second area marked with the second label, determine that the area is an area prone to misdetection, and mark the area with the first label to obtain Training samples with first label and second label.
  • each acquired training sample may also carry a third label, and the third label is used to record that the marked third area in the training sample is an area without defects.
  • the error-prone area is automatically obtained by comparing the detected defect area with the area that has been marked with defects, so as to realize the labeling of the error-prone area, which is more accurate than manual marking. And it can also save labor costs.
  • FIG. 5 is a hardware structure diagram of an electronic device according to an exemplary embodiment of this application.
  • the electronic device includes: a communication interface 501, a processor 502, a machine-readable storage medium 503, and a bus 504; wherein, the communication interface 501
  • the processor 502 and the machine-readable storage medium 503 communicate with each other through the bus 504.
  • the processor 502 reads and executes the machine executable instructions in the machine readable storage medium 503 to execute the model training method, defect detection method, and defect detection sample acquisition method described above. For details of these methods, please refer to the above implementation For example, I won’t repeat them here.
  • the machine-readable storage medium 503 mentioned in this application can be any electronic, magnetic, optical or other physical storage device, and can contain or store information, such as executable instructions, data, and so on.
  • the machine-readable storage medium may be: volatile memory, non-volatile memory, or similar storage media.
  • the machine-readable storage medium 503 may be RAM (Radom Access Memory), flash memory, storage drive (such as hard disk drive), any type of storage disk (such as optical disc, DVD, etc.), or similar storage Medium, or a combination of them.
  • Fig. 6 is a structural diagram of an embodiment of a model training device according to an exemplary embodiment of the application, and the model training device includes:
  • the acquiring module 610 is configured to acquire multiple frames of labeled training samples, the tags include at least a first tag and a second tag, and the first tag is used to record that the first area marked in the training sample is a false detection area , The second label is used to record that the marked second area in the training sample is a defective area;
  • the training module 620 is used to train a detection model for detecting defects by using each labeled training sample and the location information of the first region and the location information of the second region in each training sample.
  • the tag may further include a third tag; the third tag is used to record that the marked third area in the training sample is an area without defects;
  • the training module 620 is specifically configured to input each labeled training sample and the location information of the first region, the location information of the second region, and the location information of the third region in each training sample to a designated model training engine to
  • the model training engine uses the input labeled training samples, the location information of the first area, the location information of the second area, and the location information of the third area to train a detection model for detecting defects.
  • the detection model includes a first sub-detection model and a second sub-detection model
  • the training module 620 is further specifically configured to input each labeled training sample and the location information of the first region in each training sample to the first model training engine, so that the first model training engine uses the input labeled
  • the first sub-detection model is trained by the training samples and the location information of the first region; each labeled training sample and the location information of the second region and the location information of the third region in each training sample are input to the second model training engine ,
  • the second sub-detection model is trained by the second model training engine using the input labeled training sample, the location information of the second area, and the location information of the third area.
  • the acquisition module 610 is specifically configured to detect a defect area in the training sample for each training sample; in the detected defect area, if there is a defect area and is marked with a second label If the second area does not overlap, it is determined that the area is easy to be misdetected, and the area is marked with a first label to obtain a training sample with the first label and the second label.
  • Fig. 7 is a structural diagram of an embodiment of a defect detection device according to an exemplary embodiment of the application, and the defect detection device includes:
  • the detection module 710 is configured to input a target image to be detected into a detection model to detect the target image to be detected by the detection model to obtain a target defect probability value of each pixel in the target image to be detected;
  • the determining module 720 is configured to determine whether there is a defect area in the target image to be detected according to the target defect probability value of each pixel in the target image to be detected.
  • the detection model includes a first sub-detection model and a second sub-detection model
  • the detection module 710 is specifically configured to input the to-be-detected target image into a first sub-detection model, so that the first sub-detection model detects the to-be-detected target image to obtain the to-be-detected target image
  • the first candidate probability value of each pixel point, the first candidate probability value is used to indicate the probability of easily distinguishing defects
  • the target image to be detected is input into the second sub-detection model, so that the second sub-detection model Detect the target image to be detected, and obtain a second candidate probability value of each pixel in the target image to be detected, and the second candidate probability value is used to indicate the probability of a defect
  • the first candidate probability value and the second candidate probability value are fused to obtain the target defect probability value of each pixel.
  • the detection module 710 is specifically configured to, in the process of fusing the first candidate probability value and the second candidate probability value of each pixel point, for each pixel point, the pixel point
  • the average value of the first candidate probability value and the second candidate probability value of is used as the target defect probability value of the pixel.
  • the detection module 710 is specifically configured to merge the first candidate probability value and the second candidate probability value of each pixel point, for each pixel point, if the pixel point is If the first candidate probability value of the pixel is less than the preset value, then the first candidate probability value of the pixel is taken as the target defect probability value of the pixel; if the first candidate probability value of the pixel is not less than the preset value, the The second candidate probability value of the pixel is used as the target defect probability value of the pixel.
  • Fig. 8 is a structural diagram of an embodiment of a device for acquiring a defect detection sample according to an exemplary embodiment of the present application.
  • the device for acquiring a defect detection sample includes:
  • the detection module 810 is configured to detect a defect area in each training sample obtained
  • the labeling module 820 is used to determine that in the detected defect area, if there is an area that does not overlap with the second area marked with the second label, the area is determined to be an area prone to misdetection, and the area is used as the first Label marking to obtain training samples with the first label and the second label;
  • each acquired training sample carries a second label
  • the second label is used to record that the marked second area in the training sample is a defective area.
  • each acquired training sample may also carry a third label, and the third label is used to record that the marked third area in the training sample is an area without defects.
  • the relevant part can refer to the part of the description of the method embodiment.
  • the device embodiments described above are merely illustrative.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units.
  • Some or all of the modules can be selected according to actual needs to achieve the objectives of the solution of this application. Those of ordinary skill in the art can understand and implement it without creative work.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

本申请提供一种模型训练方法及装置，方法包括：获取多帧带标签的训练样本，所述标签至少包括第一标签和第二标签，所述第一标签用于记录训练样本中被标记的第一区域为易误检区域，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域；利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型。通过增加易误检区域的标签，在训练模型时可以根据增加的易误检区域的标签和位置加强对误检区域特征的学习，降低检测模型的误检率，提升检测模型的检出准确度。

Description

模型训练方法、装置及缺陷检测方法、装置 技术领域
本申请涉及计算机技术领域，尤其涉及一种模型训练方法、装置及缺陷检测方法、装置。
背景技术
在工业生产中，需要对生产出的产品进行检测(ASI，Automated Surface Inspection)以确定其表面的缺陷。比如，在纺织业生产出布匹后，需要检测布匹中是否存在瑕疵或缺陷，便于及时修补，提高布匹质量。
目前通过采用基于深度学习的神经网络进行缺陷检测，然而检测误检率比较高。
发明内容
有鉴于此，本申请提供一种缺陷检测方法及装置，以解决相关技术中的检测误检率比较高的问题。
根据本申请实施例的第一方面，提供一种模型训练方法，所述方法包括：
获取多帧带标签的训练样本，所述标签至少包括第一标签和第二标签，所述第一标签用于记录训练样本中被标记的第一区域为易误检区域，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域；
利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型。
根据本申请实施例的第二方面，提供一种模型训练装置，所述装置包括：
获取模块，用于获取多帧带标签的训练样本，所述标签至少包括第一标签和第二标签，所述第一标签用于记录训练样本中被标记的第一区域为易误检区域，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域；
训练模块，用于利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型。
根据本申请实施例的第三方面，提供一种应用如上述第一方面所述模型训练方法训 练的检测模型的缺陷检测方法，包括：
将待检测目标图输入检测模型，以由所述检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的目标缺陷概率值；
依据所述待检测目标图中各个像素点的目标缺陷概率值确定所述待检测目标图中是否存在缺陷区域。
根据本申请实施例的第四方面，提供一种应用如上述第一方面所述模型训练方法训练的检测模型的缺陷检测装置，所述装置包括：
检测模块，用于将待检测目标图输入检测模型，以由所述检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的目标缺陷概率值；
确定模块，用于依据所述待检测目标图中各个像素点的目标缺陷概率值确定所述待检测目标图中是否存在缺陷区域。
根据本申请实施例的第五方面，提供一种缺陷检测样本的获取方法，所述方法包括：
针对获取的每一训练样本，检测该训练样本中存在的缺陷区域；
在检测到的缺陷区域中，若存在与被标记有第二标签的第二区域未重合的区域，则确定该区域为易误检区域，并将该区域用第一标签标记，得到带有第一标签和第二标签的训练样本；
其中，获取的每一训练样本携带有第二标签，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域。
根据本申请实施例的第六方面，提供一种缺陷检测样本的获取装置，所述装置包括：
检测模块，用于针对获取的每一训练样本，检测该训练样本中存在的缺陷区域；
打标签模块，用于在检测到的缺陷区域中，若存在与被标记有第二标签的第二区域未重合的区域，则确定该区域为易误检区域，并将该区域用第一标签标记，得到带有第一标签和第二标签的训练样本；
其中，获取的每一训练样本携带有第二标签，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域。
附图说明
图1A为本申请根据一示例性实施例示出的一种待检测的筷子图；
图1B为本申请根据一示例性实施例示出的一种采用相关技术得到的筷子的缺陷图；
图2A为本申请根据一示例性实施例示出的一种模型训练方法的实施例流程图；
图2B为本申请根据图2A所示实施例示出的一种布匹缺陷类型示意图；
图2C为本申请根据图2A所示实施例示出的一种模型训练结构示意图；
图2D为本申请根据图2A所示实施例示出的另一种模型训练结构示意图；
图3A为本申请根据一示例性实施例示出的一种缺陷检测方法的流程图；
图3B为本申请根据图3A所示实施例示出的一种缺陷检测结构示意图；
图3C为本申请根据一示例性实施例示出的一种图1A所示筷子的缺陷图；
图3D为本申请根据一示例性实施例示出的一种待检测的布匹图；
图3E为本申请根据图3A所示实施例得出的图3D所示待检测的布匹的缺陷图；
图4为本申请根据一示例性实施例示出的一种缺陷检测样本的获取方法的流程图；
图5为本申请根据一示例性实施例示出的一种电子设备的硬件结构图；
图6为本申请根据一示例性实施例示出的一种模型训练装置的结构图；
图7为本申请根据一示例性实施例示出的一种缺陷检测装置的结构图；
图8为本申请根据一示例性实施例示出的一种缺陷检测样本的获取装置的结构图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
在本申请使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。在本申请和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/ 或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本申请可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本申请范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
目前采用的基于深度学习的神经网络的训练过程为：获取目标对象的样本图，并对每帧样本图中的缺陷区域和无缺陷区域分别用不同的标签标记，然后利用样本图和标记的标签对神经网络模型进行训练。但采用目前训练好的网络模型得到的检测结果，存在过拟合问题，导致误检率比较高。
在一个例子中，图1A是目标对象为筷子的待检测图，将待检测图输入训练好的网络模型，由该网络模型对该待检测图进行检测，并利用检测结果确定的缺陷图如图1B所示，如图1B所示，黑色区域是无缺陷区域，白色区域是缺陷区域，但用虚线圈起来的白色区域：区域1、区域2、区域3、区域4以及区域5均被误检为缺陷区域。
为解决上述问题，本申请提出一种模型训练方法，通过获取多帧带标签的训练样本，所述标签至少包括第一标签和第二标签，第一标签用于记录训练样本中被标记的第一区域为易误检区域，第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域，然后利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型。
基于上述描述可知，在训练检测模型时，通过增加易误检区域的标签，以根据增加的易误检区域的标签和位置信息加强对误检区域特征的学习，从而达到降低检测模型的误检率，提升检测模型检出准确度的目的。
下面以具体实施例对本申请提出的模型训练方法进行详细阐述。
图2A为本申请根据一示例性实施例示出的一种模型训练方法的实施例流程图，所述模型训练方法包括如下步骤：
步骤201：获取多帧带标签的训练样本，所述标签至少包括第一标签和第二标签，第一标签用于记录训练样本中被标记的第一区域为易误检区域，第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域。
示例性的，训练样本可以根据实际检测需求选取或抓拍，每一训练样本中的对象属 于同一类别。
假设训练的模型用于检测布匹表面的缺陷，那么可以获取多帧布匹训练样本，如图2B所示，其中图(a)中用黑线圈起来的区域表示错纱缺陷、图(b)中用黑线圈起来的区域表示坏针缺陷、图(c)中用黑线圈起来的区域表示开幅线缺陷、图(d)中用黑线圈起来的区域表示破洞缺陷；假设训练的模型用于检测筷子表面的缺陷(如凹坑)，那么可以获取多帧筷子训练样本。
在一实施例中，对于第一标签的标注过程，可以针对每一训练样本，检测该训练样本中存在的缺陷区域，在检测到的缺陷区域中，若存在与被标记有第二标签的第二区域未重合的区域，则确定该区域为易误检区域，并将该区域用第一标签标记，进而得到带有第一标签和第二标签的训练样本。
其中，用于标记存在缺陷的区域的第二标签的标注过程可以通过相关技术实现，在此不再详述。
值得说明的是，带标签的训练样本还可包括第三标签，第三标签用于记录训练样本中被标记的第三区域为不存在缺陷的区域。
本申请对第一标签、第二标签、第三标签的具体表示形式不进行限定，只要可以区分出三个不同的区域即可。
在一示例性场景中，如图1B所示，用虚线圈起来的白色区域：区域1、区域2、区域3、区域4以及区域5为易误检区域，用第一标签标记，其余的白色区域为存在缺陷的区域，用第二标签标记，黑色区域为不存在缺陷的区域，用第三标签标记。
本领域技术人员可以理解的是，对于存在缺陷的区域还可以进一步采用不同的标签来区分缺陷的种类。
步骤202：利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型。
在一实施例中，如图2C所示，当检测模型为单独一个模型时，可以通过将各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息以及第三区域的位置信息输入至指定的模型训练引擎，以由所述模型训练引擎利用输入的带标签的训练样本、第一区域的位置信息、第二区域的位置信息以及第三区域的位置信息训练出用于检测缺陷的检测模型。
示例性的，在训练过程中，第一标签的权重大于第二标签的权重和第三标签的权重，从而在进行梯度回传时，可以使得用更多的计算力去学习易误检区域的特征，降低误检率。
在另一实施例中，如图2D所示，当检测模型包括第一子检测模型和第二子检测模型时，可以通过将各带标签的训练样本和各训练样本中第一区域的位置信息输入至第一模型训练引擎，以由第一模型训练引擎利用输入的带标签的训练样本和第一区域位置信息训练出第一子检测模型，并通过将各带标签的训练样本和各训练样本中第二区域的位置信息、第三区域的位置信息输入至第二模型训练引擎，以由第二模型训练引擎利用输入的带标签的训练样本和第二区域的位置信息、第三区域的位置信息训练出第二子检测模型。
其中，第一子检测模型输出的是训练样本中每个像素点易于区分缺陷的概率值，值越大，表示越容易区分缺陷，值越小，表示比较难区分缺陷；第二子检测模型输出的是训练样本中每个像素点存在缺陷的概率值，值越大，表示存在缺陷的几率越大。
示例性的，第一子检测模型可以通过深度网络学习方法训练得到，其采用的网络结构可以卷积神经网络的结构，该卷积神经网络可以包括卷积层、池化层、BN(Batch Normalization，批量归一化)层、全连接层等计算层。第二子检测模型可以采用与第一子检测模型相同的方法得到，但也可以采用与第一子检测模型不同的方法得到，本申请对此不进行限定。
在本申请实施例中，通过获取多帧带标签的训练样本，所述标签至少包括第一标签和第二标签，第一标签用于记录训练样本中被标记的第一区域为易误检区域，第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域，然后利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型。
基于上述描述可知，在训练检测模型时，通过增加易误检区域的标签，以根据增加的易误检区域的标签和位置信息加强对误检区域特征的学习，从而达到降低检测模型的误检率，提升检测模型检出准确度的目的。
图3A为本申请根据一示例性实施例示出的一种缺陷检测方法的实施例流程图，本实施例以上述图2A所示实施例为基础，应用上述检测模型实现缺陷检测，所述缺陷检测方法包括如下步骤：
步骤301：将待检测目标图输入检测模型，以由所述检测模型对待检测目标图进行检测，得到待检测目标图中每一像素点的目标缺陷概率值。
在一实施例中，如图3B所示，当检测模型包括第一子检测模型和第二子检测模型时，可以通过将所述待检测目标图输入第一子检测模型，以由所述第一子检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的第一候选概率值，所述第一候选概率值用于表示易于区分缺陷的概率，同时也将所述待检测目标图输入第二子检测模型，以由所述第二子检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的第二候选概率值，所述第二候选概率值用于表示存在缺陷的概率，然后对每个像素点的第一候选概率值和第二候选概率值进行融合，得到每个像素点的目标缺陷概率值。
其中，由于第一子检测模型输出的检测结果表示的是待检测目标图中每个像素点易于区分缺陷的概率，值越大，表示越容易区分缺陷，值越小，表示比较难区分缺陷，而第二子检测模型输出的检测结果表示的是待检测目标图中每个像素点存在缺陷的概率，值越大，表示存在缺陷的几率越大，因此需要对第一子检测模型输出的检测结果和第二子检测模型输出的检测结果进行融合，以得到每个像素点的目标缺陷概率值。
针对每个像素点的第一候选概率值和第二候选概率值进行融合的过程，下面详细介绍两种融合方式，当然也可以采用其它融合方式，本申请对此不进行限定。
第一种融合方式：可以针对每个像素点，将该像素点的第一候选概率值和第二候选概率值的均值作为该像素点的目标缺陷概率值。公式1如下：
目标缺陷概率值＝0.5*(第一候选概率值+第二候选概率值)  公式1
第二种融合方式：可以针对每个像素点，若该像素点的第一候选概率值小于预设数值，则将该像素点的第一候选概率值作为该像素点的目标缺陷概率值，若该像素点的第一候选概率值不小于预设数值，则将该像素点的第二候选概率值作为该像素点的目标缺陷概率值。公式2如下：
Figure PCTCN2020085205-appb-000001
其中，若像素点的第一候选概率值小于预设数值，表示该像素点比较难区分是缺陷(即该像素点大概率不是缺陷)，在这种情况下，该像素点的第二候选概率值可信度不高，很有可能是错误的检测结果，因此选择将第一候选概率值作为该像素点的目标缺陷 概率值；若像素点的第一候选概率值不小于预设数值，表示该像素点比较容易区分缺陷(即该像素点大概率是缺陷)，在这种情况下，该像素点的第二候选概率值可信度比较高，因此选择将第二候选概率值作为该像素点的目标缺陷概率值。
假设预设数值为0.5，如果第一候选概率值小于0.5，则将第一候选概率值作为目标缺陷概率值，如果第一候选概率值大于等于0.5，则将第二候选概率值作为目标缺陷概率值。
基于上述描述，通过采用第一子检测模型和第二子检测模型的联合得到待检测目标图中每一像素点的目标缺陷概率值，由于第一子检测模型是用于检测每个像素点是否易于区分缺陷，通过将第一子检测模型的检测结果与第二子检测模型的检测结果融合，可以避免将不易于区分缺陷的像素点误检为缺陷。
步骤302：依据待检测目标图中各个像素点的目标缺陷概率值确定待检测目标图中是否存在缺陷区域。
在一实施例中，可以通过将待检测目标图中各个像素点的目标缺陷概率值转换为缺陷图，以便于实现缺陷区域的判断。转换过程可以是：确定待检测目标图中每一像素点的目标缺陷概率值映射的灰度值，并利用待检测目标图中各个像素点的目标缺陷概率值映射的灰度值生成待检测目标图对应的缺陷图。
其中，由于像素点的目标缺陷概率值的取值范围为0～1，而缺陷图所采用的是灰度值，取值范围为0～255，因此需要将像素点的目标缺陷概率值转换为缺陷图所用的灰度值，对于由目标缺陷概率值转换缺陷图所用的灰度值的映射关系可以预先根据实践经验设置，只要确保目标缺陷概率值越大，映射的灰度值越大即可。
示例性的，可以通过将缺陷图中灰度值大于预设阈值的像素点所形成的区域确定为存在缺陷的区域。
以检测筷子缺陷为例，将上述图1A所示的待检测图输入检测模型，由所述检测模型对该待检测图进行检测，得到该待检测图中每个像素点的目标缺陷概率值，进而将待检测图中各个像素点的目标缺陷概率值转换为缺陷图，如图3C所示的缺陷图，黑色区域为无缺陷区域，白色区域为缺陷区域，与上述采用相关技术得到的缺陷图1B对比，图1B中的误检区域被检测模型过滤掉。
以检测布匹缺陷为例，如图3D所示，为待检测的布匹图，将该待检测的布匹图输入检测模型，由检测模型对该待检测的布匹图进行检测，得到该待检测的布匹图中每个 像素点的目标缺陷概率值，进而将待检测布匹图中各像素点的目标缺陷概率值转换为缺陷图，如图3E所示的布匹的缺陷图，黑色区域为无缺陷区域，白色区域为缺陷区域。
在本实施例中，由于进行缺陷检测的模型是由增加了易误检区域的标签的训练样本训练得到的，易误检区域的标签和位置信息能够加强对误检区域特征的学习，因此通过该检测模型实现的缺陷检测误检率低，检出准确度高。
图4为本申请根据一示例性实施例示出的一种缺陷检测样本的获取方法的实施例流程图，该缺陷检测样本的获取方法包括如下步骤：
步骤401：针对获取的每一训练样本，检测该训练样本中存在的缺陷区域。
其中，获取的每一训练样本携带有第二标签，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域。针对训练样本的描述可以参见上述步骤201中的相关描述，在此不再赘述。
本领域技术人员可以理解的是，第二标签的标注过程可以通过相关技术实现，在此不再详述。
值得说明的是，检测训练样本中存在的缺陷区域可以通过相关技术中所采用的检测模型进行检测，以便于将检测到的缺陷区域与已标记的存在缺陷的区域进行区别对比，获得易误检的区域。
步骤402：在检测到的缺陷区域中，若存在与被标记有第二标签的第二区域未重合的区域，则确定该区域为易误检区域，并将该区域用第一标签标记，得到带有第一标签和第二标签的训练样本。
在一实施例中，获取的每一训练样本还可携带第三标签，所述第三标签用于记录训练样本中被标记的第三区域为不存在缺陷的区域。
本领域技术人员可以理解的是，第三标签的标注过程也可以通过相关技术实现，在此不再详述。
在本实施例中，由于易误检区域是通过将检测到的缺陷区域与已标记了存在缺陷的区域对比自动获得，进而实现对易误检区域的标注，相对于人工手动标注方式更加准确，且还可节省人力成本。
图5为本申请根据一示例性实施例示出的一种电子设备的硬件结构图，该电子设备包括：通信接口501、处理器502、机器可读存储介质503和总线504；其中，通信接口 501、处理器502和机器可读存储介质503通过总线504完成相互间的通信。处理器502通过读取并执行机器可读存储介质503中的机器可执行指令，可执行上文描述的模型训练方法、缺陷检测方法和缺陷检测样本的获取方法，这些方法的具体内容参见上述实施例，此处不再赘述。
本申请中提到的机器可读存储介质503可以是任何电子、磁性、光学或其它物理存储装置，可以包含或存储信息，如可执行指令、数据，等等。例如，机器可读存储介质可以是：易失存储器、非易失性存储器或者类似的存储介质。具体地，机器可读存储介质503可以是RAM(Radom Access Memory，随机存取存储器)、闪存、存储驱动器(如硬盘驱动器)、任何类型的存储盘(如光盘、DVD等)，或者类似的存储介质，或者它们的组合。
图6为本申请根据一示例性实施例示出的一种模型训练装置的实施例结构图，所述模型训练装置包括：
获取模块610，用于获取多帧带标签的训练样本，所述标签至少包括第一标签和第二标签，所述第一标签用于记录训练样本中被标记的第一区域为易误检区域，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域；
训练模块620，用于利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型。
在一可选实现方式中，所述标签还可包括第三标签；所述第三标签用于记录训练样本中被标记的第三区域为不存在缺陷的区域；
所述训练模块620，具体用于将各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息以及第三区域的位置信息输入至指定的模型训练引擎，以由所述模型训练引擎利用输入的带标签的训练样本、第一区域的位置信息、第二区域的位置信息以及第三区域的位置信息训练出用于检测缺陷的检测模型。
在一可选实现方式中，所述检测模型包括第一子检测模型和第二子检测模型；
所述训练模块620，还具体用于将各带标签的训练样本和各训练样本中第一区域的位置信息输入至第一模型训练引擎，以由所述第一模型训练引擎利用输入的带标签的训练样本和第一区域的位置信息训练出第一子检测模型；将各带标签的训练样本和各训练样本中第二区域的位置信息、第三区域的位置信息输入至第二模型训练引擎，以由所述第二模型训练引擎利用输入的带标签的训练样本和第二区域的位置信息、第三区域的 位置信息训练出第二子检测模型。
在一可选实现方式中，所述获取模块610，具体用于针对每一训练样本，检测该训练样本中存在的缺陷区域；在检测到的缺陷区域中，若存在与被标记有第二标签的第二区域未重合的区域，则确定该区域为易误检区域，并将该区域用第一标签标记，得到带有第一标签和第二标签的训练样本。
图7为本申请根据一示例性实施例示出的一种缺陷检测装置的实施例结构图，所述缺陷检测装置包括：
检测模块710，用于将待检测目标图输入检测模型，以由所述检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的目标缺陷概率值；
确定模块720，用于依据所述待检测目标图中各个像素点的目标缺陷概率值确定所述待检测目标图中是否存在缺陷区域。
在一可选实现方式中，所述检测模型包括第一子检测模型和第二子检测模型；
所述检测模块710，具体用于将所述待检测目标图输入第一子检测模型，以由所述第一子检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的第一候选概率值，所述第一候选概率值用于表示易于区分缺陷的概率；将所述待检测目标图输入第二子检测模型，以由所述第二子检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的第二候选概率值，所述第二候选概率值用于表示存在缺陷的概率；对每个像素点的第一候选概率值和第二候选概率值进行融合，得到每个像素点的目标缺陷概率值。
在一可选实现方式中，所述检测模块710，具体用于在将每个像素点的第一候选概率值和第二候选概率值进行融合过程中，针对每个像素点，将该像素点的第一候选概率值和第二候选概率值的均值作为该像素点的目标缺陷概率值。
在一可选实现方式中，所述检测模块710，具体用于在将每个像素点的第一候选概率值和第二候选概率值进行融合过程中，针对每个像素点，若该像素点的第一候选概率值小于预设数值，则将该像素点的第一候选概率值作为该像素点的目标缺陷概率值；若该像素点的第一候选概率值不小于预设数值，则将该像素点的第二候选概率值作为该像素点的目标缺陷概率值。
图8为本申请根据一示例性实施例示出的一种缺陷检测样本的获取装置的实施例结构图，所述缺陷检测样本的获取装置包括：
检测模块810，用于针对获取的每一训练样本，检测该训练样本中存在的缺陷区域；
打标签模块820，用于在检测到的缺陷区域中，若存在与被标记有第二标签的第二区域未重合的区域，则确定该区域为易误检区域，并将该区域用第一标签标记，得到带有第一标签和第二标签的训练样本；
其中，获取的每一训练样本携带有第二标签，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域。
在一可选实现方式中，获取的每一训练样本还可携带第三标签，所述第三标签用于记录训练样本中被标记的第三区域为不存在缺陷的区域。
上述装置中各个单元的功能和作用的实现过程具体详见上述方法中对应步骤的实现过程，在此不再赘述。
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本申请方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。
还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、商品或者设备中还存在另外的相同要素。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (14)

  1. 一种模型训练方法，包括：
    获取多帧带标签的训练样本，所述标签至少包括第一标签和第二标签，所述第一标签用于记录训练样本中被标记的第一区域为易误检区域，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域；
    利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型。
  2. 根据权利要求1所述的方法，其中，所述标签还包括第三标签；所述第三标签用于记录训练样本中被标记的第三区域为不存在缺陷的区域；
    利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型，包括：
    将各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息以及第三区域的位置信息输入至指定的模型训练引擎，以由所述模型训练引擎利用输入的带标签的训练样本、第一区域的位置信息、第二区域的位置信息以及第三区域的位置信息训练出用于检测缺陷的检测模型。
  3. 根据权利要求2所述的方法，其中，所述检测模型包括第一子检测模型和第二子检测模型；
    利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型，包括：
    将各带标签的训练样本和各训练样本中第一区域的位置信息输入至第一模型训练引擎，以由所述第一模型训练引擎利用输入的带标签的训练样本和第一区域的位置信息训练出所述第一子检测模型；
    将各带标签的训练样本和各训练样本中第二区域的位置信息、第三区域的位置信息输入至第二模型训练引擎，以由所述第二模型训练引擎利用输入的带标签的训练样本和第二区域的位置信息、第三区域的位置信息训练出所述第二子检测模型。
  4. 根据权利要求1所述的方法，其中，获取所述多帧带标签的训练样本，包括：
    针对每一训练样本，检测该训练样本中存在的缺陷区域；
    在检测到的缺陷区域中，若存在与被标记有第二标签的第二区域未重合的区域，则确定该区域为易误检区域，并将该区域用第一标签标记，得到带有第一标签和第二标签的训练样本。
  5. 一种缺陷检测样本的获取方法，包括：
    针对获取的每一训练样本，检测该训练样本中存在的缺陷区域；
    在检测到的缺陷区域中，若存在与被标记有第二标签的第二区域未重合的区域，则确定该区域为易误检区域，并将该区域用第一标签标记，得到带有第一标签和第二标签的训练样本；
    其中，获取的每一训练样本携带有第二标签，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域。
  6. 根据权利要求5所述的方法，其中，获取的每一训练样本还携带有第三标签，所述第三标签用于记录训练样本中被标记的第三区域为不存在缺陷的区域。
  7. 一种应用如上述权利要求1所述模型训练方法训练的检测模型的缺陷检测方法，包括：
    将待检测目标图输入所述检测模型，以由所述检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的目标缺陷概率值；
    依据所述待检测目标图中各个像素点的目标缺陷概率值确定所述待检测目标图中是否存在缺陷区域。
  8. 根据权利要求7所述的方法，其中，所述检测模型包括第一子检测模型和第二子检测模型；
    将待检测目标图输入检测模型，包括：
    将所述待检测目标图输入所述第一子检测模型，以由所述第一子检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的第一候选概率值，所述第一候选概率值用于表示易于区分缺陷的概率；
    将所述待检测目标图输入所述第二子检测模型，以由所述第二子检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的第二候选概率值，所述第二候选概率值用于表示存在缺陷的概率；
    对每个像素点的第一候选概率值和第二候选概率值进行融合，得到每个像素点的目标缺陷概率值。
  9. 根据权利要求8所述的方法，其中，将每个像素点的第一候选概率值和第二候选概率值进行融合，包括：
    针对每个像素点，将该像素点的第一候选概率值和第二候选概率值的均值作为该像素点的目标缺陷概率值。
  10. 根据权利要求8所述的方法，其中，将每个像素点的第一候选概率值和第二候选概率值进行融合，包括：
    针对每个像素点，若该像素点的第一候选概率值小于预设数值，则将该像素点的第一候选概率值作为该像素点的目标缺陷概率值；
    若该像素点的第一候选概率值不小于预设数值，则将该像素点的第二候选概率值作为该像素点的目标缺陷概率值。
  11. 一种模型训练装置，包括：
    获取模块，用于获取多帧带标签的训练样本，所述标签至少包括第一标签和第二标签，所述第一标签用于记录训练样本中被标记的第一区域为易误检区域，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域；
    训练模块，用于利用各带标签的训练样本和各训练样本中第一区域的位置信息、第二区域的位置信息，训练出用于检测缺陷的检测模型。
  12. 一种缺陷检测样本的获取装置，包括：
    检测模块，用于针对获取的每一训练样本，检测该训练样本中存在的缺陷区域；
    打标签模块，用于在检测到的缺陷区域中，若存在与被标记有第二标签的第二区域未重合的区域，则确定该区域为易误检区域，并将该区域用第一标签标记，得到带有第一标签和第二标签的训练样本；
    其中，获取的每一训练样本携带有第二标签，所述第二标签用于记录训练样本中被标记的第二区域为存在缺陷的区域。
  13. 一种应用如上述权利要求1所述模型训练方法训练的检测模型的缺陷检测装置，包括：
    检测模块，用于将待检测目标图输入检测模型，以由所述检测模型对所述待检测目标图进行检测，得到所述待检测目标图中每一像素点的目标缺陷概率值；
    确定模块，用于依据所述待检测目标图中各个像素点的目标缺陷概率值确定所述待检测目标图中是否存在缺陷区域。
  14. 一种电子设备，包括：
    处理器；和
    存储有机器可执行指令的机器可读存储介质；
    其中，所述处理器通过读取所述机器可执行指令执行权利要求1-10任一所述的方法。
PCT/CN2020/085205 2019-04-18 2020-04-16 模型训练方法、装置及缺陷检测方法、装置 Ceased WO2020211823A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910312755.8 2019-04-18
CN201910312755.8A CN111861966B (zh) 2019-04-18 2019-04-18 模型训练方法、装置及缺陷检测方法、装置

Publications (1)

Publication Number Publication Date
WO2020211823A1 true WO2020211823A1 (zh) 2020-10-22

Family

ID=72838053

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/085205 Ceased WO2020211823A1 (zh) 2019-04-18 2020-04-16 模型训练方法、装置及缺陷检测方法、装置

Country Status (2)

Country Link
CN (1) CN111861966B (zh)
WO (1) WO2020211823A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112651941A (zh) * 2020-12-25 2021-04-13 北京巅峰科技有限公司 车辆缺陷识别方法、装置、电子装置和存储介质
CN112784997A (zh) * 2021-01-22 2021-05-11 北京百度网讯科技有限公司 标注复核方法、装置、设备、存储介质以及程序产品
CN113706462A (zh) * 2021-07-21 2021-11-26 南京旭锐软件科技有限公司 产品表面缺陷检测方法、装置、设备及存储介质
CN113744252A (zh) * 2021-09-07 2021-12-03 全芯智造技术有限公司 用于标记和检测缺陷的方法、设备、存储介质和程序产品
CN115984959A (zh) * 2022-12-20 2023-04-18 山东大学 基于神经网络与质心跟踪的牛类异常行为检测方法及系统
CN116012634A (zh) * 2021-10-21 2023-04-25 北京首钢冷轧薄板有限公司 一种带钢板形分类方法、装置、计算机介质和设备
CN116363065A (zh) * 2023-02-22 2023-06-30 元始智能科技(南通)有限公司 基于事件相机的工件缺陷检测方法及装置
CN116580407A (zh) * 2023-05-12 2023-08-11 北京中关村科金技术有限公司 文本检测模型的训练方法、文本检测方法及装置
CN117282687A (zh) * 2023-10-18 2023-12-26 广州市普理司科技有限公司 印刷品视觉检测自动剔补标控制系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117934470B (zh) * 2024-03-22 2024-09-10 宁德时代新能源科技股份有限公司 模型训练方法、缺陷检测方法、装置、设备和存储介质
CN119251182B (zh) * 2024-09-25 2025-07-15 江西广信药业有限公司 一种药物包装缺陷的智能检测方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875381A (zh) * 2017-01-17 2017-06-20 同济大学 一种基于深度学习的手机外壳缺陷检测方法
CN109146873A (zh) * 2018-09-04 2019-01-04 凌云光技术集团有限责任公司 一种基于学习的显示屏缺陷智能检测方法及装置
US20190042953A1 (en) * 2017-08-07 2019-02-07 International Business Machines Corporation Filter for harmful training samples in online learning systems
CN109389160A (zh) * 2018-09-27 2019-02-26 南京理工大学 基于深度学习的电力绝缘端子缺陷检测方法

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103473925B (zh) * 2013-08-28 2016-08-10 惠州市德赛工业发展有限公司 一种道路车辆检测系统的验证方法
CN104156734B (zh) * 2014-08-19 2017-06-13 中国地质大学（武汉） 一种基于随机蕨分类器的全自主在线学习方法
CN107871134A (zh) * 2016-09-23 2018-04-03 北京眼神科技有限公司 一种人脸检测方法及装置
CN106548155A (zh) * 2016-10-28 2017-03-29 安徽四创电子股份有限公司 一种基于深度信念网络的车牌检测方法
JP6705777B2 (ja) * 2017-07-10 2020-06-03 ファナック株式会社 機械学習装置、検査装置及び機械学習方法
CN107966447B (zh) * 2017-11-14 2019-12-17 浙江大学 一种基于卷积神经网络的工件表面缺陷检测方法
CN107886133A (zh) * 2017-11-29 2018-04-06 南京市测绘勘察研究院股份有限公司 一种基于深度学习的地下管道缺陷自动识别方法
CN108562589B (zh) * 2018-03-30 2020-12-01 慧泉智能科技（苏州）有限公司 一种对磁路材料表面缺陷进行检测的方法
CN108921111A (zh) * 2018-07-06 2018-11-30 南京旷云科技有限公司 对象检测后处理方法及对应装置
CN109410190B (zh) * 2018-10-15 2022-04-29 广东电网有限责任公司 基于高分辨率遥感卫星影像的杆塔倒断检测模型训练方法
CN109558902A (zh) * 2018-11-20 2019-04-02 成都通甲优博科技有限责任公司 一种快速目标检测方法
CN109522968A (zh) * 2018-11-29 2019-03-26 济南浪潮高新科技投资发展有限公司 一种基于串行双任务网络的病灶区检测方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875381A (zh) * 2017-01-17 2017-06-20 同济大学 一种基于深度学习的手机外壳缺陷检测方法
US20190042953A1 (en) * 2017-08-07 2019-02-07 International Business Machines Corporation Filter for harmful training samples in online learning systems
CN109146873A (zh) * 2018-09-04 2019-01-04 凌云光技术集团有限责任公司 一种基于学习的显示屏缺陷智能检测方法及装置
CN109389160A (zh) * 2018-09-27 2019-02-26 南京理工大学 基于深度学习的电力绝缘端子缺陷检测方法

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112651941A (zh) * 2020-12-25 2021-04-13 北京巅峰科技有限公司 车辆缺陷识别方法、装置、电子装置和存储介质
CN112784997A (zh) * 2021-01-22 2021-05-11 北京百度网讯科技有限公司 标注复核方法、装置、设备、存储介质以及程序产品
CN112784997B (zh) * 2021-01-22 2023-11-10 北京百度网讯科技有限公司 标注复核方法、装置、设备、存储介质以及程序产品
CN113706462A (zh) * 2021-07-21 2021-11-26 南京旭锐软件科技有限公司 产品表面缺陷检测方法、装置、设备及存储介质
CN113744252A (zh) * 2021-09-07 2021-12-03 全芯智造技术有限公司 用于标记和检测缺陷的方法、设备、存储介质和程序产品
CN116012634A (zh) * 2021-10-21 2023-04-25 北京首钢冷轧薄板有限公司 一种带钢板形分类方法、装置、计算机介质和设备
CN115984959A (zh) * 2022-12-20 2023-04-18 山东大学 基于神经网络与质心跟踪的牛类异常行为检测方法及系统
CN116363065A (zh) * 2023-02-22 2023-06-30 元始智能科技(南通)有限公司 基于事件相机的工件缺陷检测方法及装置
CN116580407A (zh) * 2023-05-12 2023-08-11 北京中关村科金技术有限公司 文本检测模型的训练方法、文本检测方法及装置
CN117282687A (zh) * 2023-10-18 2023-12-26 广州市普理司科技有限公司 印刷品视觉检测自动剔补标控制系统
CN117282687B (zh) * 2023-10-18 2024-05-28 广州市普理司科技有限公司 印刷品视觉检测自动剔补标控制系统

Also Published As

Publication number Publication date
CN111861966A (zh) 2020-10-30
CN111861966B (zh) 2023-10-27

Similar Documents

Publication Publication Date Title
WO2020211823A1 (zh) 模型训练方法、装置及缺陷检测方法、装置
CN113344857B (zh) 缺陷检测网络的训练方法、缺陷检测方法和存储介质
US11080839B2 (en) System and method for training a damage identification model
Yao et al. A feature memory rearrangement network for visual inspection of textured surface defects toward edge intelligent manufacturing
TWI716012B (zh) 樣本標註方法、裝置、儲存媒體和計算設備、損傷類別的識別方法及裝置
WO2024002187A1 (zh) 缺陷检测方法、缺陷检测设备及存储介质
CN114627089B (zh) 缺陷识别方法、装置、计算机设备及计算机可读存储介质
US11315229B2 (en) Method for training defect detector
JP2021190716A5 (zh)
CN115719460A (zh) 一种基于深度学习的气体泄露检测方法、系统及终端设备
CN111353521A (zh) 用于噪声标签的对抗训练系统和方法
CN111696077A (zh) 一种基于WaferDet网络的晶圆缺陷检测方法
CN109919934A (zh) 一种基于多源域深度迁移学习的液晶面板缺陷检测方法
CN110232406A (zh) 一种基于统计学习的液晶面板cf图片识别方法
CN113095444B (zh) 图像标注方法、装置及存储介质
CN106127746A (zh) 电路板元件漏件检测方法和系统
CN108761237A (zh) 无人机电力巡检图像关键电力部件自动诊断与标注系统
CN116433651B (zh) 一种小样本面板缺陷检测方法、系统、设备及存储介质
CN114663687A (zh) 模型训练方法、目标识别方法、装置、设备及存储介质
US12094098B2 (en) Defect management apparatus, method and non-transitory computer readable medium
CN111311545A (zh) 集装箱检测方法、装置及计算机可读存储介质
TW202029070A (zh) 神經網路的訓練方法、基於神經網路的分類方法及其裝置
CN110619619A (zh) 一种缺陷检测方法、装置及电子设备
CN115294039A (zh) 一种钢卷端面缺陷检测方法
CN114782349A (zh) 领域自适应原油泄露检测模型训练方法、检测方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20790777

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20790777

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 22.05.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20790777

Country of ref document: EP

Kind code of ref document: A1