WO2022088628A1 - 缺陷检测方法、装置、计算机设备及存储介质 - Google Patents

缺陷检测方法、装置、计算机设备及存储介质 Download PDF

Info

Publication number
WO2022088628A1
WO2022088628A1 PCT/CN2021/089654 CN2021089654W WO2022088628A1 WO 2022088628 A1 WO2022088628 A1 WO 2022088628A1 CN 2021089654 W CN2021089654 W CN 2021089654W WO 2022088628 A1 WO2022088628 A1 WO 2022088628A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
map
image
point
feature map
Prior art date
Application number
PCT/CN2021/089654
Other languages
English (en)
French (fr)
Inventor
牛临潇
李�诚
Original Assignee
北京市商汤科技开发有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京市商汤科技开发有限公司 filed Critical 北京市商汤科技开发有限公司
Priority to KR1020217037651A priority Critical patent/KR20220058842A/ko
Priority to JP2021566097A priority patent/JP2023503751A/ja
Publication of WO2022088628A1 publication Critical patent/WO2022088628A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/001—Industrial image inspection using an image reference approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular, to a defect detection method, an apparatus, a computer device, and a storage medium.
  • defect detection on the surface of objects is also an important link in standardized production.
  • the current defect detection methods based on neural networks generally use a large number of sample images marked with defect locations to train the neural network, and then use the trained neural network to detect defects on the images of objects to be detected. This detection method has the problem of low detection accuracy.
  • Embodiments of the present disclosure provide at least one defect detection method, apparatus, computer device, and storage medium.
  • an embodiment of the present disclosure provides a defect detection method, including: acquiring an image to be detected and a template image; performing feature extraction on the image to be detected to obtain a first feature map of the image to be detected, and Perform feature extraction on the template image to obtain a second feature map corresponding to the template image; perform feature confusion processing on the first feature map and the second feature map to obtain a feature fusion image; based on the The image is fused with features to obtain the defect detection result of the image to be detected.
  • the performing feature extraction on the to-be-detected image to obtain the first feature map of the to-be-detected image includes: performing multi-level feature extraction on the to-be-detected image, and obtaining and each level of feature extraction.
  • the obtained feature fusion image includes more features in the image to be detected and the template image, and then the defect detection of the image to be detected is determined based on the feature fusion image. As a result, there is higher precision.
  • obtaining the defect detection result of the to-be-detected image based on the feature fusion image includes: obtaining each first feature map based on the feature fusion image corresponding to each first feature map.
  • the defect detection result of the feature map; the defect detection result of the to-be-detected image is obtained based on the defect detection result of the first feature map corresponding to the multi-level feature extraction.
  • the defect detection result of the image to be detected determined by using the corresponding defect detection results of the multi-level feature extraction has higher detection accuracy.
  • multi-level feature extraction is performed on the to-be-detected image, and an intermediate feature map corresponding to each level of feature extraction is obtained; in the case that each level of feature extraction is the last level of feature extraction, the The intermediate feature map corresponding to the last-level feature extraction is used as the first feature map corresponding to the last-level feature extraction; in the case that the feature extraction of each level is the feature extraction of other levels except the last-level feature extraction, the The intermediate feature map corresponding to each level of feature extraction is feature-fused with the first feature map corresponding to the next-level feature extraction of this level of feature extraction to obtain a first feature map corresponding to each level of feature extraction.
  • the first feature maps obtained by extracting different levels of features include different features in the image to be detected, so that the corresponding first feature maps based on the multi-level feature extraction are extracted.
  • the defect detection result, the determined defect detection result of the image to be detected has higher detection accuracy.
  • the intermediate feature map corresponding to each level of feature extraction and the first feature map corresponding to the next-level feature extraction of this level of feature extraction are subjected to feature fusion to obtain a feature corresponding to each level of feature extraction.
  • Extracting the corresponding first feature map includes: up-sampling the first feature map corresponding to the next-level feature extraction of this level of feature extraction to obtain an up-sampling vector; After the intermediate feature maps are superimposed, the first feature map corresponding to this level of feature extraction is obtained.
  • the performing feature confusion processing on each of the first feature maps and the second feature maps corresponding to the each of the first feature maps includes: based on the each of the first feature maps. feature map, and a second feature map corresponding to each of the first feature maps, perform feature enhancement processing on the second feature map corresponding to the first feature map to obtain a second feature corresponding to the first feature map a feature-enhanced image of the map; and obtaining an attention mask corresponding to each of the first feature maps based on the each of the first feature maps and the second feature maps corresponding to the each of the first feature maps image; wherein, the pixel value of any pixel point in the attention mask image represents the abnormality value of the defect that the first feature point whose position matches the any pixel point in the first feature map; based on the The feature-enhanced image and the attention mask image are used to obtain a feature fusion image corresponding to each of the first feature maps.
  • feature enhancement can be performed on the second feature map of the template image to reduce the difference between the image to be detected and the template image caused by acquisition noise, matching errors, and production errors existing in the image to be detected. to improve the defect detection accuracy of the image to be inspected.
  • the pixel value of each pixel in the attention mask image represents the first feature at the corresponding position in the first feature map
  • the abnormality value of whether the point has defects, and then according to the attention mask image, the defect detection result of the first feature map is determined, which has higher detection accuracy.
  • the feature enhancement process is performed on the second feature map corresponding to each first feature map based on each first feature map and the second feature map corresponding to each first feature map. , including: for each first feature point in each first feature map, from a plurality of second feature points in the second feature map corresponding to each first feature map, determine the corresponding first feature point. A plurality of associated feature points; wherein, for each associated feature point corresponding to the first feature point, the distance between the target second feature point matching the position of the first feature point satisfies a preset condition; based on the first feature point and the For the similarity between each associated feature point, feature enhancement processing is performed on the target second feature point that matches the position of the first feature point.
  • the corresponding third pixel point is then obtained.
  • the abnormal degree value of the second pixel point corresponding to the third pixel point makes the abnormal degree value of the second pixel point affected by multiple pixels in the template image, so as to reduce the production error, matching error, acquisition noise, etc. in the image to be detected.
  • the influence of the defect detection result of the second pixel point improves the defect detection accuracy of the image to be processed.
  • feature enhancement processing is performed on the target second feature point that matches the position of the first feature point, including: Based on the similarity between the first feature point and each associated feature point, and the feature value of each associated feature point, feature enhancement processing is performed on the target second feature point that matches the position of the first feature point.
  • the feature value of the target second feature point whose position matches the first feature point is re-determined, so that the re-determined feature value
  • Various errors existing with the first feature point can be reduced, so as to have higher detection accuracy in the case of defect detection based on the feature-enhanced image.
  • the target first feature point matching the position of the first feature point is determined.
  • feature enhancement processing on two feature points including: based on the similarity between the first feature point and each associated feature point, weighting and summing the feature values corresponding to the multiple associated feature points corresponding to the first feature point respectively , obtain the first sum value; sum the corresponding degrees of similarity of multiple associated feature points to obtain the second sum value; take the ratio of the first sum value and the second sum value as the target The feature value of the second feature point after feature enhancement processing is performed.
  • the attention mask image corresponding to the first feature map is obtained, including : For each first feature point in the first feature map, from a plurality of second feature points in the second feature map corresponding to the first feature map, determine a plurality of second feature points corresponding to the first feature point Associated feature points; wherein, the distance between each associated feature point corresponding to the first feature point and the target second feature point matching the position of the first feature point satisfies a preset condition; based on the first feature point and each The similarity between the associated feature points is determined to determine the abnormality value of the first feature point; the attention mask image is obtained based on the abnormality value corresponding to each first feature point in the first feature map.
  • determining the abnormality value of the first feature point based on the similarity between the first feature point and each associated feature point includes: determining that a plurality of associated feature points are respectively associated with the associated feature point. The maximum similarity of the similarity between the first feature points; based on the maximum similarity, the abnormality value of the first feature point is determined.
  • the similarity between the first feature point and any associated feature point corresponding to the first feature point is determined in the following manner: The position in the feature map and the preset distance threshold, to obtain a first feature sub-map; and based on the position of any associated feature point corresponding to the first feature point in the second feature map, and the distance threshold to obtain a second feature submap; based on the first feature submap and the second feature submap, determine the relationship between the first feature point and any associated feature point corresponding to the first feature point similarity.
  • the obtaining a feature fusion image corresponding to the first feature map based on the feature enhancement image and the attention mask image includes: performing the feature enhancement image and the The first feature map is merged to obtain a merged feature map corresponding to the first feature map; based on the attention mask image and the merged feature map, the feature fusion image is obtained.
  • an embodiment of the present disclosure further provides a defect detection device, comprising: an acquisition part configured to acquire an image to be detected and a template image; a feature extraction part configured to perform feature extraction on the image to be detected, obtaining the first feature map of the image to be detected, and performing feature extraction on the template image to obtain a second feature map corresponding to the template image; the feature confusion part is configured to combine the first feature map, and the second feature map performs feature confusion processing to obtain a feature fusion image; the detection part is configured to obtain a defect detection result of the to-be-detected image based on the feature fusion image.
  • the feature extraction part is configured to perform feature extraction on the to-be-detected image when a first feature map of the to-be-detected image is obtained.
  • Multi-level feature extraction obtaining a first feature map corresponding to each level of feature extraction
  • the feature extraction part in the case of performing feature extraction on the template image to obtain a second feature map corresponding to the template image, is configured to perform multi-level feature extraction on the template image, and obtain a second feature map corresponding to each of the first feature maps
  • the feature confusion part will combine the first feature map and the
  • the second feature map is subjected to feature confusion processing to obtain a feature fusion image, it is configured to perform, for each first feature map, each first feature map and the corresponding
  • the second feature map performs feature confusion processing to obtain a feature fusion image corresponding to each of the first feature maps.
  • the detection part in the case of obtaining the defect detection result of the image to be detected based on the feature fusion image, is configured to: based on the feature fusion corresponding to each first feature map. image to obtain the defect detection result of each first feature map; based on the defect detection result of the first feature map corresponding to the multi-level feature extraction, obtain the defect detection result of the to-be-detected image.
  • the feature extraction part is configured to: perform multi-level feature extraction on the to-be-detected image and obtain a first feature map corresponding to each level of feature extraction, and be configured to: Multi-level feature extraction is performed on the image to be detected, and an intermediate feature map corresponding to each level of feature extraction is obtained; in the case that each level of feature extraction is the last level of feature extraction, the last level of feature extraction is corresponding to the intermediate feature map, As the first feature map corresponding to the last level feature extraction; in the case that the feature extraction of each level is the feature extraction of other levels except the last level feature extraction, the intermediate feature map corresponding to the feature extraction of each level Feature fusion is performed on the first feature map corresponding to the next-level feature extraction of this level of feature extraction to obtain the first feature map corresponding to each level of feature extraction.
  • the feature extraction part performs feature fusion between the intermediate feature map corresponding to each level of feature extraction and the first feature map corresponding to the next-level feature extraction of this level of feature extraction, to obtain the same feature.
  • the first feature map corresponding to each level of feature extraction it is configured to: perform up-sampling on the first feature map corresponding to the next-level feature extraction of this level of feature extraction to obtain an up-sampling vector; After the upsampling vector is superimposed with the intermediate feature map corresponding to the feature extraction of this level, the first feature map corresponding to the feature extraction of this level is obtained.
  • the feature confusion section when performing feature confusion processing on each of the first feature maps and the second feature maps corresponding to each of the first feature maps, is Configured as:
  • feature enhancement processing is performed on the second feature maps corresponding to the first feature maps to obtain each first feature map. a feature-enhanced image of the second feature map corresponding to the feature map;
  • an attention mask image corresponding to each first feature map is obtained; wherein, the attention The pixel value of any pixel point in the force mask image, representing the abnormality value of the defect that the first feature point whose position matches the any pixel point in the first feature map;
  • a feature fusion image corresponding to each first feature map is obtained.
  • the feature confusion part is based on each first feature map and the second feature map corresponding to each first feature map, for the second feature corresponding to each first feature map.
  • the feature enhancement processing is performed on the map, it is configured to: for each first feature point in each first feature map, from a plurality of second feature points in the second feature map corresponding to each first feature map , determine a plurality of associated feature points corresponding to the first feature point; wherein, the distance between each associated feature point corresponding to the first feature point and the target second feature point matching the position of the first feature point satisfies the predetermined Setting conditions; based on the similarity between the first feature point and each associated feature point, perform feature enhancement processing on the target second feature point that matches the position of the first feature point.
  • the feature confusion part based on the similarity between the first feature point and each associated feature point, performs a feature on the target second feature point that matches the position of the first feature point.
  • enhancement processing it is configured to: based on the similarity between the first feature point and each associated feature point, and the feature value of each associated feature point, for the target first feature point matching the position of the first feature point. Two feature points are processed for feature enhancement.
  • the feature confusion part based on the similarity between the first feature point and each associated feature point, and the feature value of each associated feature point, confuses the first feature point with the first feature point.
  • the feature enhancement processing is performed on the target second feature point of the location matching, it is configured to: based on the similarity between the first feature point and each associated feature point, multiple associated features corresponding to the first feature point
  • the feature values corresponding to the points are weighted and summed to obtain the first sum value; the similarity corresponding to multiple associated feature points is summed to obtain the second sum value; the first sum value and the second sum value are obtained.
  • the ratio of the sum value is used as the feature value after feature enhancement processing is performed on the target second feature point.
  • the feature confusion part obtains the attention corresponding to the first feature map based on the first feature map and the second feature map corresponding to the first feature map.
  • a mask image it is configured to: for each first feature point in the first feature map, from a plurality of second feature points in the second feature map corresponding to the first feature map, determine A plurality of associated feature points corresponding to the first feature point; wherein, the distance between each associated feature point corresponding to the first feature point and the target second feature point matching the position of the first feature point satisfies a preset condition ; Based on the similarity between the first feature point and each associated feature point, determine the abnormality value of the first feature point; Based on the abnormality value corresponding to each first feature point in the first feature map, obtain the attention mask image.
  • the feature confusion part in the case of determining the abnormality value of the first feature point based on the similarity between the first feature point and each associated feature point, is configured as: : determine the maximum similarity of the similarity between the plurality of associated feature points and the first feature point; based on the maximum similarity, determine the abnormality value of the first feature point.
  • the feature confusion part is configured to determine the similarity between the first feature point and any associated feature point corresponding to the first feature point in the following manner: based on the The position of the first feature point in the first feature map and the preset distance threshold, to obtain a first feature sub-map; and based on any associated feature point corresponding to the first feature point in the second feature The position in the figure and the distance threshold are obtained to obtain a second feature sub-map; based on the first feature sub-map and the second feature sub-map, it is determined that the first feature point corresponds to the first feature point The similarity between any of the associated feature points.
  • the feature confusion part in the case of obtaining the feature fusion image corresponding to the first feature map based on the feature enhancement image and the attention mask image, is configured as: : combine the feature enhancement image and the first feature map to obtain a combined feature map corresponding to the first feature map; based on the attention mask image and the combined feature map, obtain the combined feature map. feature fusion image.
  • an embodiment of the present disclosure further provides a computer device, comprising: a processor and a memory connected to each other, the memory stores machine-readable instructions executable by the processor, and when the computer device runs, the Machine-readable instructions are executed by the processor to implement the above-mentioned first aspect, or the defect detection method in any possible implementation manner of the first aspect.
  • embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to execute the first aspect, or any one of the first aspect.
  • a computer program including computer readable code, which when executed in an electronic device, implements the above method when executed by a processor in the electronic device.
  • FIG. 1 shows a flowchart of a defect detection method provided by an embodiment of the present disclosure
  • FIGS. 2a-2d show schematic diagrams of a network structure of feature fusion provided by an embodiment of the present disclosure
  • FIG. 3a shows a schematic diagram of implementing defect detection using a neural network provided by an embodiment of the present disclosure
  • 3b shows a flowchart of a specific method for performing feature confusion processing on a first feature map and a second feature map corresponding to the first feature map provided by an embodiment of the present disclosure
  • FIG. 4a shows a schematic diagram of a process of a feature-enhanced image provided by an embodiment of the present disclosure
  • FIG. 4b shows a schematic structural diagram of a feature confusion network provided by an embodiment of the present disclosure
  • FIG. 5 shows a schematic diagram of a defect detection apparatus provided by an embodiment of the present disclosure
  • FIG. 6 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
  • Template-free method which usually uses simple image processing to obtain the position and category of defects in the image to be detected of the object to be detected; in addition, a large number of samples can be used to train the neural network model, so that the The to-be-detected image of the to-be-detected object is input into the trained neural network model, and a defect detection result of the to-be-detected image of the to-be-detected object is obtained.
  • This stencilless approach may recall a large number of wrong detection targets due to the lack of relevant information in the stencil image and the inability to distinguish between designed parts and defective parts.
  • the current defect detection methods for the object to be detected all have the problem of low detection accuracy.
  • the present disclosure provides a defect detection method and device.
  • a defect detection method and device By performing multi-level feature extraction on an image to be detected, a first feature map image corresponding to each level of feature extraction is obtained, and multi-level feature extraction is performed on a template image, Obtain the second feature map corresponding to each first feature map, and then perform feature confusion processing on the first feature map and the second feature map corresponding to the first feature map for each first feature map.
  • the execution subject of the defect method provided by the embodiment of the present disclosure is generally a computer device with a certain computing capability.
  • the terminal device may be a device dedicated to quality detection, and may also be implemented by the processor calling computer-readable instructions stored in the memory.
  • the defect detection method provided by the embodiment of the present disclosure can not only be used for defect detection of an object to be inspected, but also defect detection of other items, such as workpieces, machine parts, and the like.
  • the defect detection method provided by the embodiment of the present disclosure will be described below by taking the defect detection of the object to be detected as an example.
  • the method includes steps S101-S104, wherein:
  • S102 Perform feature extraction on the image to be detected to obtain a first feature map of the image to be detected, and perform feature extraction on the template image to obtain a second feature map corresponding to the template image;
  • S103 Perform feature confusion processing on the first feature map and the second feature map to obtain a feature fusion image
  • S104 Obtain a defect detection result of the to-be-detected image based on the feature fusion image.
  • the template image refers to a standard design drawing in industrial production, or an image of a qualified object used in the case of defect detection of the object to be inspected.
  • the qualified object is for objects that do not have defects.
  • the image to be detected refers to the image obtained from the object to be detected.
  • the object to be detected includes, for example, at least one of various types of mechanical parts, materials, printed circuit boards, electronic components, and the like.
  • the model or logo of the part to be inspected can be obtained first; then, according to the model or logo of the part, a template image corresponding to the part to be inspected is obtained from a pre-built template image library;
  • a template image without defects can be determined from a plurality of parts to be inspected, and then the image of the template part is acquired to obtain the template. image.
  • the to-be-detected image can be acquired by, for example, an image acquisition module set on the defect detection device, and can also receive the to-be-detected image transmitted by other devices.
  • Multi-level feature extraction is performed on the image to be detected, and a first feature map corresponding to each level of feature extraction is obtained.
  • feature fusion may be performed based on the multi-level features, and then a first feature map corresponding to each level of feature extraction is obtained;
  • the method of the first feature map can use a characteristic image pyramid network, as shown in Figure 2a; a single feature map network can also be used, as shown in Figure 2b; a pyramid feature layer network can also be used, as shown in Figure 2c; A Feature Pyramid Network (Feature Pyramid Networks, FPN) is used, as shown in FIG. 2d ; this is not limited in the embodiment of the present disclosure.
  • each level of feature extraction can obtain an intermediate feature map of the image to be processed.
  • the intermediate feature map obtained by the previous-level feature extraction is the input of the latter-level feature extraction, that is, the latter-level feature extraction is based on the intermediate features extracted from the previous-level feature extraction.
  • the latter-stage feature extraction is performed on the graph to obtain an intermediate feature map of the latter-stage feature extraction.
  • the intermediate feature map corresponding to the last-level feature extraction is used as the first feature map corresponding to the last-level feature extraction; for multi-level feature extraction, the last-level feature is removed.
  • the first feature map performs feature fusion between the intermediate feature map corresponding to each level of feature extraction and the first feature map corresponding to the next-level feature extraction of this level of feature extraction to obtain the feature extraction corresponding to each level.
  • the next-level feature extraction corresponding to this level of feature extraction corresponds to The size of the first feature map is smaller than the intermediate feature map corresponding to this level of feature extraction, and the first feature map corresponding to the next-level feature extraction of this level of feature extraction is up-sampled to obtain an up-sampled image;
  • the size is consistent with the size of the intermediate feature map corresponding to this level of feature extraction, and then the upsampled image and the intermediate feature map corresponding to this level of feature extraction are superimposed to obtain the first feature map corresponding to this level of feature extraction.
  • the next-level feature extraction corresponding to this level of feature extraction corresponds to The size of the first feature map is equal to the intermediate feature map corresponding to this level of feature extraction.
  • the first feature map corresponding to the next-level feature extraction of this level of feature extraction and the intermediate feature map corresponding to this level of feature extraction can be directly extracted. Superposition is performed to obtain the first feature map corresponding to the feature extraction of this level.
  • a pre-trained feature extraction network may be used to perform multi-level feature extraction on the image to be detected.
  • the feature extraction network may adopt a structure based on a convolutional neural network (CNN), such as: AlexNet, a deep convolutional neural network developed by the Visual Geometry Group (VGG) of Oxford University , Residual Neural Network (ResNet), SqueezeNet, DenseNet, GoogLeNet, ShuffleNet, MobileNet, ResNeXt, etc.
  • CNN convolutional neural network
  • AlexNet a deep convolutional neural network developed by the Visual Geometry Group (VGG) of Oxford University
  • VCG Visual Geometry Group
  • ResNet Residual Neural Network
  • SqueezeNet DenseNet
  • GoogLeNet GoogLeNet
  • ShuffleNet MobileNet
  • MobileNet ResNeXt
  • an embodiment of the present disclosure further provides a structural example of a feature extraction network, including: a four-level network layer, the four-level network layer sequentially includes: a first-level network layer, a first-level network layer, a third-level network layer A second-level network layer, a third-level network layer, and a fourth-level network layer.
  • each level of network layer can output an intermediate feature map corresponding to this level of network layer, wherein the first-level network layer performs the first-level feature extraction on the image to be detected.
  • the intermediate feature map A1 the second-level network layer performs the second-level feature extraction on the intermediate feature map A1, and obtains the intermediate feature map A2
  • the third-level network layer performs the third-level feature extraction on the intermediate feature map A2 to obtain the intermediate feature Figure A3
  • the fourth-level network layer performs the fourth-level feature extraction on the intermediate feature map A3 to obtain the intermediate feature map A4.
  • the first feature map A4' corresponding to the intermediate feature map A4 is extracted as the fourth-level feature
  • the third-level network layer After up-sampling the first feature map A4' corresponding to the fourth-level feature extraction, superimpose the intermediate feature map A3 corresponding to the third-level feature extraction to obtain the first feature map corresponding to the third-level feature extraction.
  • a feature map A3' For the third-level network layer, after up-sampling the first feature map A4' corresponding to the fourth-level feature extraction, superimpose the intermediate feature map A3 corresponding to the third-level feature extraction to obtain the first feature map corresponding to the third-level feature extraction.
  • a feature map A3' For the third-level network layer, after up-sampling the first feature map A4' corresponding to the fourth-level feature extraction, superimpose the intermediate feature map A3 corresponding to the third-level feature extraction to obtain the first feature map corresponding to the third-level feature extraction.
  • the second-level network layer After upsampling the first feature map A3' corresponding to the third-level feature extraction, superimpose the intermediate feature map A2 corresponding to the second-level feature extraction to obtain the first feature map corresponding to the second-level feature extraction.
  • a feature map A2' For the second-level network layer, after upsampling the first feature map A3' corresponding to the third-level feature extraction, superimpose the intermediate feature map A2 corresponding to the second-level feature extraction to obtain the first feature map corresponding to the second-level feature extraction.
  • a feature map A2' For the second-level network layer, after upsampling the first feature map A3' corresponding to the third-level feature extraction, superimpose the intermediate feature map A2 corresponding to the second-level feature extraction to obtain the first feature map corresponding to the second-level feature extraction.
  • the first-level network layer After up-sampling the first feature map A2' corresponding to the second-level feature extraction, superimpose the intermediate feature map A1 corresponding to the first-level feature extraction to obtain the first feature map corresponding to the first-level feature extraction.
  • a feature map A1' For the first-level network layer, after up-sampling the first feature map A2' corresponding to the second-level feature extraction, superimpose the intermediate feature map A1 corresponding to the first-level feature extraction to obtain the first feature map corresponding to the first-level feature extraction.
  • a feature map A1' For the first-level network layer, after up-sampling the first feature map A2' corresponding to the second-level feature extraction, superimpose the intermediate feature map A1 corresponding to the first-level feature extraction to obtain the first feature map corresponding to the first-level feature extraction.
  • multi-level feature extraction can also be performed on the template image to obtain a second feature map corresponding to each first feature map; the process of obtaining the second feature map is the same as obtaining the first feature.
  • the process of the figure is similar and will not be repeated here.
  • a pre-trained feature extraction network may be used to perform multi-level feature extraction on the template image, so as to obtain second feature maps corresponding to the multi-level feature extraction respectively.
  • the feature extraction network and the feature extraction network for obtaining the first feature map above may be the same network, or may be two feature extraction branches of the twin network.
  • the two feature extraction networks are two feature extraction branches of the Siamese network, the parameters of the two feature extraction branches are the same.
  • the feature extraction network that obtains the second feature map and the feature extraction network that obtains the first feature highlight are two feature extraction branches of the Siamese network.
  • the feature extraction network used to obtain the second feature map is the same as the feature extraction network used to obtain the first feature map, and also includes a four-level network layer.
  • a second-level network layer, a third-level network layer, and a fourth-level network layer is the same as the feature extraction network used to obtain the first feature map, and also includes a four-level network layer.
  • a second-level network layer, a third-level network layer, and a fourth-level network layer is the same as the feature extraction network used to obtain the first feature map, and also includes a four-level network layer.
  • a second-level network layer, a third-level network layer, and a fourth-level network layer is the same as the feature extraction network used to obtain the first feature map, and also includes a four-level network layer.
  • each level of network layer can output an intermediate feature map corresponding to this level of network layer, wherein the first-level network layer performs the first-level feature extraction on the template image. , obtain the intermediate feature map B1, the second-level network layer performs the second-level feature extraction on the intermediate feature map B1, and obtains the intermediate feature map B2; the third-level network layer performs the third-level feature extraction on the intermediate feature map B2 to obtain the intermediate feature Figure B3; the fourth-level network layer performs the fourth-level feature extraction on the intermediate feature map B3 to obtain the intermediate feature map B4.
  • the intermediate feature map B4 is used as the fourth-level feature to extract the corresponding second feature map B4';
  • the third-level network layer After up-sampling the second feature map B4' corresponding to the fourth-level feature extraction, superimpose the intermediate feature map B3 corresponding to the third-level feature extraction to obtain the third-level feature extraction corresponding to the first feature map B3.
  • Two feature maps B3' Two feature maps B3'.
  • the second-level network layer After up-sampling the second feature map B3' corresponding to the third-level feature extraction, superimpose the intermediate feature map B2 corresponding to the second-level feature extraction to obtain the first-level feature map corresponding to the second-level feature extraction.
  • Two feature maps B2' Two feature maps B2'.
  • the first-level network layer After upsampling the second feature map B2' corresponding to the second-level feature extraction, superimpose the intermediate feature map B1 corresponding to the first-level feature extraction to obtain the first-level feature extraction corresponding to the first level.
  • Two feature maps B1' Two feature maps B1'.
  • the process of multi-level feature extraction can be performed only once, and after the second feature maps corresponding to the multi-level feature extraction are obtained, the second feature maps corresponding to each level of feature extraction can be saved in the preset storage of the execution body. Location.
  • defect detection for a part if there is currently a second feature map of the template image corresponding to the part, it can be directly read from the preset storage location without the need to perform multi-level feature extraction on the template image again. .
  • At least one level of feature extraction may also be performed on the image to be detected, and the output of the last level of feature extraction may be used as the first feature map of the image to be detected; The output of the first-level feature extraction is used as the second feature map of the template image.
  • an embodiment of the present disclosure further provides a method for performing feature confusion processing on a first feature map and a second feature map corresponding to the first feature map, including:
  • a target second target feature point f 2 that matches the first feature point may be determined from the second feature map F 2 based on a normalized cross-correlation matching algorithm (Normalized Cross Correlation, NCC), Then, feature enhancement is performed on the target second feature points f 2 that match the positions of each first feature point in the first feature map, respectively, to obtain a feature-enhanced image F 22 of the second feature map; as shown in FIG. 4 a .
  • NCC Normalized Cross Correlation
  • feature enhancement processing may be performed on the second feature map corresponding to the first feature map in the following manner:
  • each first feature point in the first feature map from a plurality of second feature points in the second feature map corresponding to the first feature map, determine a plurality of associations corresponding to the first feature point feature point; wherein, the distance between each associated feature point corresponding to the first feature point and the target second feature point matching the position of the first feature point satisfies a preset condition; based on the first feature point and each associated feature point
  • the similarity between the feature points is to perform feature enhancement processing on the target second feature point that matches the position of the first feature point.
  • the first feature map is represented as:
  • the second feature map is represented as:
  • the target second feature point matching the position thereof is: b ij .
  • the multiple associated feature points corresponding to any first feature point are, for example, the second feature points whose distance from the target second feature point is smaller than a preset distance threshold in the second feature map.
  • the distance is, for example, any one of L1 distance, L2 distance, Euclidean distance, and Manhattan distance.
  • each first feature point firstly, from the second feature map, determine the target second feature point that matches the position of the first feature point, and then use the second feature map, All the second feature points whose distances from the target second feature point satisfy the preset condition are used as a plurality of associated feature points corresponding to the first feature point. It is also possible to use all the second feature points in the second feature map whose distances from the target second feature points meet the preset conditions as candidate feature points, and then select random sampling or evenly spaced sampling from a plurality of second feature points. A plurality of associated feature points are determined from the candidate feature points.
  • the similarity between each associated feature point and the first feature point can be determined, for example, in the following manner:
  • a first feature sub-map is obtained; and based on any associated feature point corresponding to the first feature point in the From the position in the second feature map and the preset distance threshold, a second feature submap is obtained; based on the first feature submap and the second feature submap, determine the relationship between the first feature point and the second feature submap.
  • the preset distance threshold may be set as required, which is not limited in this embodiment of the present disclosure.
  • the first feature point may be taken as the center of the circle, and the preset The distance threshold is a first circular area with a radius, and the first feature sub-map is obtained based on the first feature points on the first feature map located in the first circular area.
  • the first feature points in the first feature submap may include all the first feature points located in the first circular area, or may only include part of the first feature points located in the first circular area.
  • the second feature sub-map corresponding to any associated feature point for example, on the second feature map, it may be determined that with each associated feature point as the center of the circle and the The preset distance threshold is a second circular area with a radius, and the second feature sub-map is obtained based on the second feature points located in the second circular area on the second feature map.
  • the second feature points in the second feature submap may include all the second feature points located in the second circular area, or may only include a part of the second feature points located in the second circular area.
  • the second feature point in the second feature submap also only includes the first feature point. Part of the second feature points in the circular area; and the positions of the first feature points in the first feature sub-map and the second feature points in the second sub-map match one by one.
  • the first feature sub-map corresponding to the first feature point for example, on the first feature map, it may be determined that the first feature point is the center and the determined
  • the target side length is a first square area with a side length, and the first feature sub-map is obtained based on the first feature points located in the first square area on the first feature map.
  • the first feature point included in the first feature submap for example, includes all the first feature points located in the first square area, or only includes the first feature points located in the first Part of the first feature points in a square area.
  • the target side length may also be determined based on the preset distance threshold; on the second feature map, Determining a second square area with each associated feature point as the center and the determined target side length as the side length, based on the second feature point located in the second square area on the second feature map, The second feature submap is obtained.
  • the second feature point included in the second feature submap for example, includes all the second feature points located in the second square area, or only includes the second feature points located in the second square area. Part of the second feature points in the square area.
  • the similarity between the first feature point and any associated feature point corresponding to the first feature point is determined based on the first feature submap.
  • patch A represents the first feature sub-map
  • Patch Bn represents the second feature sub-map of the n-th associated feature point
  • patch A *Patch Bn represents the first feature sub-map and the n-th associated feature point
  • the second feature Subgraph performs matrix multiplication; sum( ) means summing the element values of all elements in the matrix; N ⁇ n ⁇ 1.
  • Feature enhancement processing is performed on the target second feature point that matches the position of the first feature point.
  • a weighted sum of the feature values corresponding to a plurality of the associated feature points respectively may be performed to obtain a first sum value; and , sum the respective similarities of multiple associated feature points to obtain a second sum value; take the ratio of the first sum value and the second sum value as the target for matching the position of the first feature point
  • the feature value of the second feature point after feature enhancement processing is performed.
  • the feature value ft(B) 2 after the feature enhancement process is performed on the target second feature point whose position matches the first feature point satisfies the following formula (2):
  • ft(B)′ n represents the eigenvalue corresponding to the nth associated feature point.
  • an attention mask image corresponding to any first feature map may be obtained in the following manner: for each first feature point in the first feature map, from the first feature map Among the multiple second feature points of the second feature map corresponding to the feature map, multiple associated feature points corresponding to the first feature point are determined; wherein, each associated feature point corresponding to the first feature point is the same as the first feature point.
  • the distance between the target second feature points whose feature point positions are matched satisfies a preset condition; based on the similarity between the first feature point and each associated feature point, the abnormality value of the first feature point is determined.
  • the manner of the associated feature point corresponding to the first feature point and the manner of determining the similarity between the first feature point and the associated feature point are similar to those in S301 above, and will not be repeated here.
  • the maximum similarity of the similarity between the multiple associated feature points and the first feature point can be determined; based on the maximum similarity degree, and determine the abnormal degree value of the first feature point.
  • the abnormality value S of any first feature point for example, satisfies the following formula (3):
  • H represents the maximum similarity.
  • ⁇ is a preset coefficient, such as 1, 0.5, etc. Specifically, it can be set according to actual needs.
  • the average similarity may be determined according to the similarities between the plurality of associated pixels and the first pixel, and the abnormality value of the first pixel may be determined based on the average similarity.
  • the attention mask image is obtained based on the abnormality value corresponding to each first feature point in the first feature map;
  • an image composed of abnormal degree values corresponding to all the first feature points can be used as an attention mask image.
  • the feature-enhanced image and the first feature map can be merged to obtain the merged feature map corresponding to the first feature map, and then based on the attention mask image and the Merge the feature maps to obtain the feature fusion image.
  • the feature-enhanced image and the first feature map may be superimposed to obtain a combined feature map.
  • the attention mask image and the merged feature map can be matrix-multiplied to obtain the feature fusion image.
  • the process of performing feature confusion processing on the first feature map and the second feature map corresponding to the threshold may be implemented, for example, by using a pre-trained feature confusion network.
  • an embodiment of the present disclosure further provides a structure of a feature confusion network, including: a feature enhancement part and an abnormal attention part.
  • the abnormal attention part is configured to obtain the attention mask image of the first feature map based on the method provided in S302 above.
  • the feature enhancement part is configured to obtain a feature enhanced image corresponding to the second feature map corresponding to the first feature map based on the method provided in the above S302.
  • the feature enhancement image and the first feature map are superimposed to obtain a merged feature map; and based on the attention mask image and the merged feature map, a feature fusion image is obtained.
  • the first feature map and the second feature map can also be subjected to feature confusion processing in a manner similar to the above to obtain a feature fusion image, The specific feature obfuscation processing method will not be repeated here.
  • a pre-trained detection network may be used to perform defect detection processing on the feature fusion image to obtain a defect detection result corresponding to the image to be detected.
  • the pre-trained detection network provided by the embodiment of the present disclosure may be: FCOS, Fovea, RetinaNET, faster R-CNN, etc.; the pre-trained detection network may be set as required, which is not limited in the embodiment of the present disclosure.
  • the pre-trained detection network may employ a Fully Convolutional One-Stage Object Detection (FCOS) network.
  • FCOS Fully Convolutional One-Stage Object Detection
  • the FCOS network is capable of defect category, defect centrality, and the position of the defect frame in the first feature map.
  • the defect centrality is used for the probability that a certain feature point in the first feature map is the center of the defect frame.
  • the position of the defect frame in the first feature map indicates the position where the defect exists in the first feature map.
  • the defect detection results of the first feature map corresponding to the four-level feature extraction are obtained, and the defect detection results include: defect category, defect centrality, and defect frame. s position.
  • non-maximum suppression method Non-Maximum Suppression, NMS
  • NMS Non-Maximum Suppression
  • intersection of the detection results of the first feature maps corresponding to the multi-level feature extraction can also be taken to determine the defect detection result of the image to be detected.
  • multi-level feature extraction is performed on the image to be detected, a first feature map image corresponding to each level of feature extraction is acquired, and a template image is performed multi-level feature extraction to acquire the second feature corresponding to each first feature map Then, for each first feature map, perform feature confusion processing on the first feature map and the second feature map corresponding to the first feature map to obtain the feature fusion corresponding to the first feature map.
  • the defect detection results of the first feature map are obtained, and then the defect detection results of the first feature maps corresponding to the multi-level feature extraction are synthesized, and the defect detection results of the image to be detected are obtained with higher accuracy.
  • the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
  • the embodiments of the present disclosure also provide a defect detection device corresponding to the defect detection method. Reference may be made to the implementation of the method, and repeated descriptions will not be repeated.
  • the apparatus includes: an acquisition part 51 , a feature extraction part 52 , a feature confusion part 53 , and a detection part 54 ; wherein,
  • the acquisition part 51 is configured to acquire the image to be detected and the template image
  • the feature extraction part 52 is configured to perform feature extraction on the to-be-detected image to obtain a first feature map of the to-be-detected image, and to perform feature extraction on the template image to obtain a second feature map corresponding to the template image. feature map;
  • the feature confusion part 53 is configured to perform feature confusion processing on the first feature map and the second feature map to obtain a feature fusion image
  • the detection part 54 is configured to fuse the images based on the features to obtain a defect detection result of the to-be-detected image.
  • the feature extraction part 52 is configured to perform feature extraction on the to-be-detected image to obtain a first feature map of the to-be-detected image. Carry out multi-level feature extraction, and obtain the first feature map corresponding to each level of feature extraction; the feature extraction part 52 performs feature extraction on the template image to obtain a second feature map corresponding to the template image.
  • the second feature map is subjected to feature confusion processing to obtain a feature fusion image, and is configured to perform, for each first feature map, each first feature map and each first feature map with the Feature confusion processing is performed on the second feature map corresponding to the map to obtain a feature fusion image corresponding to each of the first feature maps.
  • the detection part 54 in the case of obtaining the defect detection result of the to-be-detected image based on the feature fusion image, is configured to: based on the corresponding feature of each first feature map. Fusing the images to obtain a defect detection result of each of the first feature maps; and obtaining a defect detection result of the to-be-detected image based on the defect detection results of the first feature maps corresponding to the multi-level feature extraction.
  • the feature extraction part 52 is configured to: perform multi-level feature extraction on the to-be-detected image and obtain a first feature map corresponding to each level of feature extraction. Perform multi-level feature extraction on the image to be detected, and obtain an intermediate feature map corresponding to each level of feature extraction; in the case that each level of feature extraction is the last-level feature extraction, extract the last-level feature corresponding to the intermediate feature map , as the first feature map corresponding to the feature extraction of the last level; in the case that the feature extraction of each level is the feature extraction of other levels except the feature extraction of the last level, the intermediate features corresponding to the feature extraction of each level are extracted Feature fusion is performed between the image and the first feature map corresponding to the next-level feature extraction of this level of feature extraction to obtain the first feature map corresponding to each level of feature extraction.
  • the feature extraction part 52 performs feature fusion between the intermediate feature map corresponding to each level of feature extraction and the first feature map corresponding to the next-level feature extraction of this level of feature extraction to obtain:
  • the first feature map corresponding to the feature extraction of each level it is configured to: perform up-sampling on the first feature map corresponding to the next-level feature extraction of this level of feature extraction to obtain an up-sampling vector; After the above-mentioned up-sampling vector is superimposed with the intermediate feature map corresponding to the feature extraction of this stage, the first feature map corresponding to the feature extraction of this stage is obtained.
  • the feature confusion section 53 performs feature confusion processing on each of the first feature maps and the second feature maps corresponding to each of the first feature maps.
  • the feature confusion part 53 is based on the first feature map and the second feature map corresponding to the first feature map, and the second feature map corresponding to the first feature map.
  • the feature map is subjected to feature enhancement processing, it is configured to: for each first feature point in the first feature map, from a plurality of second feature points in the second feature map corresponding to the first feature map , determine a plurality of associated feature points corresponding to the first feature point; wherein, for each associated feature point corresponding to the first feature point, the distance between the target second feature points that match the position of the first feature point satisfies Preset conditions; based on the similarity between the first feature point and each associated feature point, perform feature enhancement processing on the target second feature point that matches the position of the first feature point.
  • the feature confusion part 53 based on the similarity between the first feature point and each associated feature point, performs the second feature point matching with the position of the first feature point.
  • feature enhancement processing it is configured to: based on the similarity between the first feature point and each associated feature point, and the feature value of each associated feature point, for the target matching the position of the first feature point
  • the second feature point is subjected to feature enhancement processing.
  • the feature confusion part 53 based on the similarity between the first feature point and each associated feature point, and the feature value of each associated feature point, compares the first feature with the first feature.
  • feature enhancement processing is performed on the target second feature point whose point position is matched, it is configured to: based on the similarity between the first feature point and each associated feature point, multiple associated The feature values corresponding to the feature points are weighted and summed to obtain the first sum value; the similarity corresponding to the plurality of associated feature points is summed to obtain the second sum value; the first sum value and the first sum value are obtained.
  • the ratio of the two sum values is used as the feature value after feature enhancement processing is performed on the second feature point of the target.
  • the feature confusion part 53 obtains the attention corresponding to the first feature map based on the first feature map and the second feature map corresponding to the first feature map.
  • a force mask image it is configured to: for each first feature point in the first feature map, from a plurality of second feature points in the second feature map corresponding to the first feature map, Determine a plurality of associated feature points corresponding to the first feature point; wherein, for each associated feature point corresponding to the first feature point, the distance between the target second feature points that match the position of the first feature point satisfies a preset condition; based on the similarity between the first feature point and each associated feature point, determine the abnormality value of the first feature point; based on the abnormality value corresponding to each first feature point in the first feature map, Get the attention mask image.
  • the feature confusion part 53 is configured to determine the abnormality value of the first feature point based on the similarity between the first feature point and each associated feature point. The steps are: determining the maximum similarity of the similarity between the plurality of associated feature points and the first feature point; and determining the abnormality value of the first feature point based on the maximum similarity.
  • the feature confusion part 53 is configured to determine the similarity between the first feature point and any associated feature point corresponding to the first feature point in the following manner: the position of the first feature point in the first feature map and a preset distance threshold to obtain a first feature sub-map; and based on any associated feature point corresponding to the first feature point in the second feature point The position in the feature map and the distance threshold are obtained to obtain a second feature sub-map; based on the first feature sub-map and the second feature sub-map, the first feature point and the first feature point are determined. The corresponding similarity between any of the associated feature points.
  • the feature confusion part 53 is configured to obtain a feature fusion image corresponding to the first feature map based on the feature enhancement image and the attention mask image. are: combine the feature enhancement image and the first feature map to obtain a combined feature map corresponding to the first feature map; based on the attention mask image and the combined feature map, obtain The feature fusion image.
  • a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, a unit, a module or a non-modularity.
  • An embodiment of the present disclosure further provides a computer device 10.
  • a schematic structural diagram of the computer device 10 provided by the embodiment of the present disclosure includes:
  • feature confusion processing is performed on the first feature map and the second feature map corresponding to the first feature map to obtain a feature fusion image corresponding to the first feature map;
  • the defect detection result of the first feature map is obtained
  • the defect detection results of the to-be-detected image are obtained.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the defect detection method described in the above method embodiments are executed.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • the computer program product of the defect detection method provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the steps of the defect detection methods described in the above method embodiments. , for details, refer to the foregoing method embodiments, which will not be repeated here.
  • Embodiments of the present disclosure also provide a computer program, which implements any one of the methods in the foregoing embodiments when the computer program is executed by a processor.
  • the computer program product can be specifically implemented by hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
  • the computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .
  • the present disclosure by fusing the features in the image to be processed and the template image, errors such as production error, matching error, and acquisition noise existing between the first feature map and the second feature map are reduced, and then the feature The images are fused to obtain the defect detection result of the first feature map, and then the defect detection results of the first feature map corresponding to the multi-level feature extraction are synthesized to obtain the defect detection result of the image to be detected with higher accuracy.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

本公开提供了一种缺陷检测方法、装置、计算机设备及存储介质，其中，该方法包括：获取待检测图像、以及模板图像；对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图，并对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图；将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像；基于所述特征融合图像，得到所述待检测图像的缺陷检测结果。本公开实施例具有更高的缺陷检测精度。

Description

缺陷检测方法、装置、计算机设备及存储介质
相关申请的交叉引用
本公开基于申请号为202011191743.3、申请日为2020年10月30日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。
技术领域
本公开涉及图像处理技术领域，具体而言，涉及一种缺陷检测方法、装置、计算机设备及存储介质。
背景技术
随着科技的发展，现代制造业对于各种工业零件、产品的需求也日益增长。一些机械装置、电子元件本身质量也很大程度取决于一些零件是否符合要求。所以物体表面的缺陷检测也是标准化生产中的重要环节。当前基于神经网络的缺陷检测方法，一般利用大量标注了缺陷位置的样本图像对神经网络进行训练，然后利用训练好的神经网络，对物体的待检测图像进行缺陷检测。这种检测方法存在检测精度低的问题。
发明内容
本公开实施例至少提供一种缺陷检测方法、装置、计算机设备及存储介质。
第一方面，本公开实施例提供了一种缺陷检测方法，包括：获取待检测图像、以及模板图像；对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图，并对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图；将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像；基于所述特征融合图像，得到所述待检测图像的缺陷检测结果。
这样，通过将待处理图像的第一特征图和模板图像的第二特征图进行特征混淆处理，来减小第一特征图和第二特征图之间存在的生产误差、匹配误差、以及采集噪声等误差，然后能够利用特征融合图像，得到第一特征图的更精确的缺陷检测结果。
一种可能的实施方式中，所述对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图包括：对所述待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图；所述对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图，包括：对所述模板图像进行多级特征提取，获取与每张所述第一特征图对应的第二特征图；所述将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像，包括：针对每张第一特征图，对所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图进行特征混淆处理，得到所述每张第一特征图对应的特征融合图像。
这样，通过对待检测图像和模板图像分别进行多级特征提取，使得得到的特征融合图像中，包括了待检测图像和模板图像中的更多特征，进而基于特征融合图像确定待检测图像的缺陷检测结果，具有更高的精度。
一种可能的实施方式中，基于所述特征融合图像，得到所述待检测图像的缺陷检测结果，包括：基于所述每张第一特征图对应的特征融合图像，得到所述每张第一特征图的缺陷检测结果；基于与多级特征提取分别对应的第一特征图的缺陷检测结果，得到所述待检测图像的缺陷检测结果。
这样，通过得到多级特征提取分别对应的缺陷检测结果，然后利用多级特征提取分别对应的缺陷检测结果确定的待检测图像的缺陷检测结果具有更高的检测精度。
一种可能的实施方式中，对所述待检测图像进行多级特征提取，获取与每级特征提取对应的中间特征图；在所述每级特征提取为最后一级特征提取的情况下，将最后一级特征提取对应的中间特征图，作为该最后一级特征提取对应的第一特征图；在所述每级特征提取为除最后一级特征提取的其他级特征提取的情况下，将与所述每级特征提取对应的中间特征图与该级特征提取的下一级特征提取对应的第一特征图进行特征融合，得到与所述每级特征提取对应的第一特征图。
这样，通过对待检测图像进行多级特征提取，使得不同级特征提取所得到的第一特征图中包含了待检测图像中不同的特征，进而使得基于多级特征提取分别对应的第一特征图的缺陷检测结果，确定的待检测图像的缺陷检测结果具有更高的检测精度。
一种可能的实施方式中，所述将与每级特征提取对应的中间特征图，与该级特征提取的下一级特征提取对应的第一特征图进行特征融合，得到与所述每级特征提取对应的第一特征图，包括：将与该级特征提取的下一级特征提取对应的第一特征图进行上采样，得到上采样向量；将所述上采样向量与该级特征提取对应的中间特征图进行叠加后，得到该级特征提取对应的第一特征图。
这样，通过上采样，统一第一特征图和对应中间特征图的维度，更方便二者的融合。
一种可能的实施方式中，所述对所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图进行特征混淆处理，包括：基于所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图，对所述第一特征图对应的第二特征图进行特征增强处理，得到该张第一特征图对应的第二特征图的特征增强图像；以及基于所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图，得到与所述每张第一特征图对应的注意力掩码图像；其中，所述注意力掩码图像中任一像素点的像素值，表征该第一特征图中位置与该任一像素点匹配的第一特征点存在缺陷的异常度值；基于所述特征增强图像、以及所述注意力掩码图像，得到所述每张第一特征图对应的特征融合图像。
这样，特征增强，能够通过对模板图像的第二特征图进行特征增强，以减小由于待检测图像存在的采集噪声、匹配误差、以及生产误差，所带来的待检测图像和模板图像之间的差异，提升对待检测图像的缺陷检测精度。
另外，通过生成待检测图像的第一特征图对应的注意力掩码图像，该注意力掩码图像中的每个像素点的像素值，表征了在第一特征图中对应位置的第一特征点是否存在缺陷的异常度值，然后根据注意力掩码图像，确定第一特征图的缺陷检测结果，具有更高的检测精度。
一种可能的实施方式中，所述基于每张第一特征图、以及与每张第一特征图对应的第二特征图，对每张第一特征图对应的第二特征图进行特征增强处理，包括：针对每张第一特征图中的每个第一特征点，从每张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；基于该第一特征点与每个关联特征点之间的相似度，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
这样，通过为每个第一特征点确定关联特征点，并基于多个关联特征点分别与对应第一特征点之间的相似度来确定该对应第三像素点异常度值，进而得到与该第三像素点对应的第二像素点的异常度值，使得第二像素点的异常度值受到模板图像中多个像素点的影响，以降低生产误差、匹配误差、采集噪声等对待检测图像中第二像素点的缺陷检测结果的影响，提升对待处理图像的缺陷检测精度。
一种可能的实施方式中，所述基于该第一特征点与每个关联特征点之间的相似度，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理，包括：基于该第一特征点与每个关联特征点之间的相似度、以及每个关联特征点的特征值，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
这样，通过关联特征点与第一特征点的相似度、以及每个关联特征点的特征值，重新确定第一特征点位置匹配的目标第二特征点的特征值，使得重新确定后的特征值能够降低与第一特征点之间存在的各种误差，以在基于特征增强图像进行缺陷检测的情况下具有更高的检测精度。
一种可能的实施方式中，所述基于该第一特征点与每个关联特征点之间的相似度、以及每个关联特征点的特征值，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理，包括：基于该第一特征点与每个关联特征点之间的相似度，对该第一特征点对应的多个关联特征点分别对应的特征值进行加权求和，得到第一和值；对多个关联特征点分别对应的相似度进行求和，得到第二和值；将所述第一和值和所述第二和值的比值，作为对所述目标第二特征点进行特征增强处理后的特征值。
一种可能的实施方式中，所述基于该张第一特征图、以及与该张第一特征图对应的第二特征图，得到与该张第一特征图对应的注意力掩码图像，包括：针对该张第一特征图中的每个第一特征点，从该张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；基于该第一特征点与每个关联特征点之间的相似度，确定该第一特征点的异常度值；基于所述第一特征图中各个第一特征点对应的异常度值，得到所述注意力掩码图像。
一种可能的实施方式中，所述基于该第一特征点与每个关联特征点之间的相似度，确定该第一特征点的异常度值，包括：确定多个关联特征点分别与该第一特征点之间的相似度的最大相似度；基于所述最大相似度，确定该第一特征点的异常度值。
一种可能的实施方式中，采用下述方式确定第一特征点、和与该第一特征点对应的任一关联特征点之间的相似度：基于所述第一特征点在所述第一特征图中的位置、以及预设的距离阈值，得到第一特征子图；以及基于与该第一特征点对应的任一关联特征点在所述第二特征图中的位置、以及所述距离阈值，得到第二特征子图；基于所述第一特征子图、以及所述第二特征子图，确定所述第一特征点与该第一特征点对应的该任一关联特征点之间的相似度。
一种可能的实施方式中，所述基于所述特征增强图像、以及所述注意力掩码图像，得到所述第一特征图对应的特征融合图像，包括：对所述特征增强图像、以及该张第一特征图进行合并处理，得到该张第一特征图对应的合并特征图；基于所述注意力掩码图像、以及所述合并特征图，得到所述特征融合图像。
第二方面，本公开实施例还提供一种缺陷检测装置，包括：获取部分，被配置为获取待检测图像、以及模板图像；特征提取部分，被配置为对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图，并对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图；特征混淆部分，被配置为将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像；检测部分，被配置为基于所述特征融合图像，得到所述待检测图像的缺陷检测结果。
一种可能的实施方式中，所述特征提取部分，在对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图的情况下，被配置为对所述待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图；所述特征提取部分，在对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图的情况下，被配置为对所述模板图像进行多 级特征提取，获取与每张所述第一特征图对应的第二特征图；所述特征混淆部分，在将将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像的情况下，被配置为针对每张第一特征图，对所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图进行特征混淆处理，得到所述每张第一特征图对应的特征融合图像。
一种可能的实施方式中，检测部分，在基于所述特征融合图像，得到所述待检测图像的缺陷检测结果的情况下，被配置为：基于所述每张第一特征图对应的特征融合图像，得到所述每张第一特征图的缺陷检测结果；基于与多级特征提取分别对应的第一特征图的缺陷检测结果，得到所述待检测图像的缺陷检测结果。
一种可能的实施方式中，所述特征提取部分，在对所述待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图的情况下，被配置为：对所述待检测图像进行多级特征提取，获取与每级特征提取对应的中间特征图；在所述每级特征提取为最后一级特征提取的情况下，将最后一级特征提取对应的中间特征图，作为该最后一级特征提取对应的第一特征图；在所述每级特征提取为除最后一级特征提取的其他级特征提取的情况下，将与所述每级特征提取对应的中间特征图与该级特征提取的下一级特征提取对应的第一特征图进行特征融合，得到与所述每级特征提取对应的第一特征图。
一种可能的实施方式中，所述特征提取部分，在将与每级特征提取对应的中间特征图，与该级特征提取的下一级特征提取对应的第一特征图进行特征融合，得到与所述每级特征提取对应的第一特征图的情况下，被配置为：将与该级特征提取的下一级特征提取对应的第一特征图进行上采样，得到上采样向量；将所述上采样向量与该级特征提取对应的中间特征图进行叠加后，得到该级特征提取对应的第一特征图。
一种可能的实施方式中，所述特征混淆部分，在对所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图进行特征混淆处理的情况下，被配置为：
基于所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图，对所述第一特征图对应的第二特征图进行特征增强处理，得到每张第一特征图对应的第二特征图的特征增强图像；以及，
基于所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图，得到与所述每张第一特征图对应的注意力掩码图像；其中，所述注意力掩码图像中任一像素点的像素值，表征该第一特征图中位置与该任一像素点匹配的第一特征点存在缺陷的异常度值；
基于所述特征增强图像、以及所述注意力掩码图像，得到所述每张第一特征图对应的特征融合图像。
一种可能的实施方式中，所述特征混淆部分，在基于每张第一特征图、以及与每张第一特征图对应的第二特征图，对每张第一特征图对应的第二特征图进行特征增强处理的情况下，被配置为：针对每张第一特征图中的每个第一特征点，从每张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；基于该第一特征点与每个关联特征点之间的相似度，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
一种可能的实施方式中，所述特征混淆部分，在基于该第一特征点与每个关联特征点之间的相似度，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理的情况下，被配置为：基于该第一特征点与每个关联特征点之间的相似度、以及每个关联特征点的特征值，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
一种可能的实施方式中，所述特征混淆部分，在基于该第一特征点与每个关联特征点之间的相似度、以及每个关联特征点的特征值，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理的情况下，被配置为：基于该第一特征点与每个关联特征点之间的 相似度，对该第一特征点对应的多个关联特征点分别对应的特征值进行加权求和，得到第一和值；对多个关联特征点分别对应的相似度进行求和，得到第二和值；将所述第一和值和所述第二和值的比值，作为对所述目标第二特征点进行特征增强处理后的特征值。
一种可能的实施方式中，所述特征混淆部分，在基于该张第一特征图、以及与该张第一特征图对应的第二特征图，得到与该张第一特征图对应的注意力掩码图像的情况下，被配置为：针对该张第一特征图中的每个第一特征点，从该张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；基于该第一特征点与每个关联特征点之间的相似度，确定该第一特征点的异常度值；基于所述第一特征图中各个第一特征点对应的异常度值，得到所述注意力掩码图像。
一种可能的实施方式中，所述特征混淆部分，在基于该第一特征点与每个关联特征点之间的相似度，确定该第一特征点的异常度值的情况下，被配置为：确定多个关联特征点分别与该第一特征点之间的相似度的最大相似度；基于所述最大相似度，确定该第一特征点的异常度值。
一种可能的实施方式中，所述特征混淆部分，被配置为采用下述方式确定第一特征点、和与该第一特征点对应的任一关联特征点之间的相似度：基于所述第一特征点在所述第一特征图中的位置、以及预设的距离阈值，得到第一特征子图；以及基于与该第一特征点对应的任一关联特征点在所述第二特征图中的位置、以及所述距离阈值，得到第二特征子图；基于所述第一特征子图、以及所述第二特征子图，确定所述第一特征点与该第一特征点对应的该任一关联特征点之间的相似度。
一种可能的实施方式中，所述特征混淆部分，在基于所述特征增强图像、以及所述注意力掩码图像，得到所述第一特征图对应的特征融合图像的情况下，被配置为：对所述特征增强图像、以及该张第一特征图进行合并处理，得到该张第一特征图对应的合并特征图；基于所述注意力掩码图像、以及所述合并特征图，得到所述特征融合图像。
第三方面，本公开实施例还提供一种计算机设备，包括：相互连接的处理器和存储器，所述存储器存储有所述处理器可执行的机器可读指令，当计算机设备运行时，所述机器可读指令被所述处理器执行以实现上述第一方面，或第一方面中任一种可能的实施方式中的缺陷检测方法。
第四方面，本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述第一方面，或第一方面中任一种可能的实施方式中的缺陷检测方法。
根据本公开的一方面，提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行时实现上述方法。
为使本公开的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，此处的附图被并入说明书中并构成本说明书中的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本公开实施例所提供的一种缺陷检测方法的流程图；
图2a-2d示出了本公开实施例所提供的特征融合的网络结构示意图；
图3a示出了本公开实施例所提供的利用神经网络实现缺陷检测的示意图；
图3b示出了本公开实施例所提供的对第一特征图、以及与该张第一特征图对应的第二特征图进行特征混淆处理的具体方法的流程图；
图4a示出了本公开实施例所提供的一种特征增强图像的过程示意图；
图4b示出了本公开实施例所提供的一种特征混淆网络的结构示意图；
图5示出了本公开实施例所提供的一种缺陷检测装置的示意图；
图6示出了本公开实施例所提供的一种计算机设备的示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本公开实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本公开的实施例的详细描述并非旨在限制要求保护的本公开的范围，而是仅仅表示本公开的选定实施例。基于本公开的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
经研究发现，对物体表面缺陷进行检测的方法通常有以下两种：
(1)无模板方法，该方法通常使用简单的图像处理来获取待检测对象的待检测图像中存在缺陷的位置和缺陷的类别；此外，还可以利用大量的样本来训练神经网络模型，从而将待检测对象的待检测图像输入至训练好的神经网络模型中，得到待检测对象的待检测图像的缺陷检测结果。这种无模板的方式由于缺少了模板图像中的相关信息，由于无法区分设计的零件和有缺陷的零件，可能会召回大量错误的检测目标。
(2)有模板方法，该方法利用模板图像和待检测对象的待检测图像对缺陷进行定位和分类；但由于在待检测对象生产过程中，常常造成待检测对象存在一定的生产误差；另外，在将待检测对象的待检测图像和模板图像进行比对的情况下，也会存在图像之间的匹配误差；此外，待检测图像在采集过程也可能存在采集噪声；这些误差导致了当前对零件缺陷检测结果存在大量误检区域，造成缺陷检测精度的下降。
因此，当前对待检测对象进行缺陷检测的方法均存在检测精度低的问题。
基于上述研究，本公开提供了一种缺陷检测方法及装置，通过对待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图图像，并对模板图像进行多级特征提取，获取每张第一特征图对应的第二特征图，然后针对每张第一特征图，对该张第一特征图、以及与该张第一特征图对应的第二特征图，进行特征混淆处理，得到该张第一特征图对应的特征融合图像，从而通过将待处理图像和模板图像中特征的融合，来减小第一特征图和第二特征图之间存在的生产误差、匹配误差、以及采集噪声等误差，然后利用特征融合图像，得到第一特征图的缺陷检测结果，进而综合多级特征提取分别对应的第一特征图的缺陷检测结果，以更高的精度获得待检测图像的缺陷检测结果。
针对以上方案所存在的缺陷，均是发明人在经过实践并仔细研究后得出的结果，因此，上述问题的发现过程以及下文中本公开针对上述问题所提出的解决方案，都应该是发明人在本公开过程中对本公开做出的贡献。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
为便于对本实施例进行理解，首先对本公开实施例所公开的一种缺陷检测方法进行详细介绍，本公开实施例所提供的缺陷方法的执行主体一般为具有一定计算能力的计算机设备，计算机设备例如包括：终端设备或服务器或其它处理设备，终端设备可以为专用于进行质量检测的设备，也可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
另外，本公开实施例提供的缺陷检测方法除了能够用于待检测对象进行缺陷检测外，还可以对其他物品进缺陷检测，例如工件、机器部件等。
下面以对待检测对象进行缺陷检测为例对本公开实施例提供的缺陷检测方法加以说明。
参见图1所示，为本公开实施例提供的缺陷检测方法的流程图，所述方法包括步骤S101～S104，其中：
S101：获取待检测图像、以及模板图像；
S102：对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图，并对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图；
S103：将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像；
S104：基于所述特征融合图像，得到所述待检测图像的缺陷检测结果。
下面对上述S101～S104加以详细说明。
I：在上述S101中，模板图像，是指在工业生产中作为标准的设计图纸，或对待检测对象进行缺陷检测的情况下所用的对合格的对象拍摄的图像，此处，合格的对象，即为不存在缺陷的对象。待检测图像，是指对待检测对象获取的图像。
示例性的，待检测对象例如包括：各类机械零件、材料、印制电路板、电子元器件等中至少一种。
以将零件作为待检测对象为例：
在对待检测零件进行缺陷检测的情况下，例如首先可以获取待检测零件的型号或者标识；然后根据零件的型号或者标识，从预先构建的模板图像库中，获取与待检测零件对应的模板图像；又例如，在模板图像库中不存在待检测零件的模板图像的情况下，例如可以首先从多个待检测零件中确定一未存在缺陷的模板零件，然后获取该模板零件的图像，以得到模板图像。
待检测图像例如可以通过缺陷检测设备上设置的图像采集模组来获取，也可以接收其他设备传输的待检测图像。
II：在上述S102中，对待处理图像进行特征提取，例如可以采用下述方式：
对所述待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图。
在本公开实施例中，对待检测图像进行多级特征提取后，可以基于多级特征进行特征融合，进而获取与每级特征提取对应的第一特征图；这里，获取与每级特征提取对应的第一特征图的方式可以采用特征化图像金字塔网络，如图2a所示；也可以采用单一特征图网络，如图2b所示；也可以采用金字塔特征层网络，如图2c所示；还可以采用特征金字塔网络(Feature Pyramid Networks，FPN)，如图2d所示；对此，本公开实施例不作限制。
示例性的，每一级特征提取均能够得到待处理图像的一张中间特征图。其中，对于任意相邻的两级特征提取，前一级特征提取得到的中间特征图，为后一级特征提取的输入，也即，后一级特征提取基于前一级特征提取得到的中间特征图进行该后一级特征提取，得到该后一级特征提取的中间特征图。针对多级特征提取中的最后一级特征提取，将最后一级特征提取对应的中间特征图，作为该最后一级特征提取对应的第一特征图；针对多级特征提取中除最后一级特征提取的其他级特征提取，将与每级特征提取对应的中间特征图，与该级特征提取的下一级特征提取对应的第一特征图进行特征融合，得到与所述每级特征提取对应的第一特征图。
在将与每级特征提取对应的中间特征图，与该级特征提取的下一级特征提取对应的第一特征图进行特征融合的情况下，响应于该级特征提取的下一级特征提取对应的第一特征图的尺寸，小于该级特征提取对应的中间特征图，将该级特征提取的下一级特征提取对应的第一特征图进行上采样，得到上采样图像；该上采样图像的尺寸，与该级特征提取对应 的中间特征图的尺寸一致，后将该上采样图像和该级特征提取对应的中间特征图叠加后，得到该级特征提取对应的第一特征图。
在将与每级特征提取对应的中间特征图，与该级特征提取的下一级特征提取对应的第一特征图进行特征融合的情况下，响应于该级特征提取的下一级特征提取对应的第一特征图的尺寸，等于该级特征提取对应的中间特征图，例如可以直接将该级特征提取的下一级特征提取对应的第一特征图、和该级特征提取对应的中间特征图进行叠加，得到该级特征提取对应的第一特征图。
在本公开一种实施例中，例如可以利用预先训练的特征提取网络对待检测图像进行多级特征提取。
在一些实施例中，特征提取网络可以采用基于卷积神经网络(Convolutional Neural Network，CNN)的结构，例如：AlexNet、由牛津大学计算机视觉组(Visual Geometry Group，VGG)开发的深度卷积神经网络、残差神经网络(Residual Neural Network，ResNet)、挤压网络(SqueezeNet)、密集网络(DenseNet)、GoogLeNet、ShuffleNet、移动端神经网络(MobileNet)、ResNeXt等。
示例性的，参见图3a所示，本公开实施例还提供一种特征提取网络的结构示例，包括：四级网络层，该四级网络层从前向后依次包括：第一级网络层、第二级网络层、第三级网络层、以及第四级网络层。
通过上述四级网络层对待处理图像A进行四级特征提取，每级网络层均能够输出与该级网络层对应的中间特征图，其中，第一级网络层对待检测图像进行第一级特征提取，得到中间特征图A1，第二级网络层对中间特征图A1进行第二级特征提取，得到中间特征图A2；第三级网络层对中间特征图A2进行第三级特征提取，得到中间特征图A3；第四级网络层对中间特征图A3进行第四级特征提取，得到中间特征图A4。
针对第四级网络层，将中间特征图A4作为第四级特征提取对应的第一特征图A4’；
针对第三级网络层，将第四级特征提取对应的第一特征图A4’进行上采样后，与第三级特征提取对应的中间特征图A3进行叠加，得到第三级特征提取对应的第一特征图A3’。
针对第二级网络层，将第三级特征提取对应的第一特征图A3’进行上采样后，与第二级特征提取对应的中间特征图A2进行叠加，得到第二级特征提取对应的第一特征图A2’。
针对第一级网络层，将第二级特征提取对应的第一特征图A2’进行上采样后，与第一级特征提取对应的中间特征图A1进行叠加，得到第一级特征提取对应的第一特征图A1’。
在对模板图像进行特征提取的情况下，例如也可以对模板图像进行多级特征提取，获取与每张第一特征图对应的第二特征图；获取第二特征图的过程与获取第一特征图的过程类似，在此不再赘述。
此处，例如可以采用预先训练的特征提取网络对模板图像进行多级特征提取，以得到多级特征提取分别对应的第二特征图。
此处，该特征提取网络，与上述得到第一特征图的特征提取网络可以是同一网络，也可以是孪生网络的两个特征提取分支。在两个特征提取网络为孪生网络的两个特征提取分支的情况下，两个特征提取分支的参数相同。
示例性的，参见图3a所示的示例中，得到第二特征图的特征提取网络与得到第一特征凸显的特征提取网络为孪生网络的两个特征提取分支。
用于得到第二特征图的特征提取网络，与用于得到第一特征图的特征提取网络相同，也包括四级网络层，四级网络层从前向后依次包括：第一级网络层、第二级网络层、第三级网络层、以及第四级网络层。
通过上述四级网络层对模板图像B进行四级特征提取，每级网络层均能够输出与该级网络层对应的中间特征图，其中，第一级网络层对模板图像进行第一级特征提取，得到中间特征图B1，第二级网络层对中间特征图B1进行第二级特征提取，得到中间特征图B2； 第三级网络层对中间特征图B2进行第三级特征提取，得到中间特征图B3；第四级网络层对中间特征图B3进行第四级特征提取，得到中间特征图B4。
针对第四级网络层，将中间特征图B4作为第四级特征提取对应的第二特征图B4’；
针对第三级网络层，将第四级特征提取对应的第二特征图B4’进行上采样后，与第三级特征提取对应的中间特征图B3进行叠加，得到第三级特征提取对应的第二特征图B3’。
针对第二级网络层，将第三级特征提取对应的第二特征图B3’进行上采样后，与第二级特征提取对应的中间特征图B2进行叠加，得到第二级特征提取对应的第二特征图B2’。
针对第一级网络层，将第二级特征提取对应的第二特征图B2’进行上采样后，与第一级特征提取对应的中间特征图B1进行叠加，得到第一级特征提取对应的第二特征图B1’。
在本公开另一实施例中，在对多个相同的零件进行缺陷检测的情况下，由于多个零件对应的模板图像一般情况下都相同，因此，可以针对多个相同的零件所对应的同一模板图像，可以只进行一次多级特征提取的过程，并在得到多级特征提取分别对应的第二特征图后，将各级特征提取分别对应的第二特征图保存在执行主体的预设存储位置。在对某个零件进行缺陷检测的情况下，若当前存在该零件对应的模板图像的第二特征图，可以直接从预设存储位置中读取，而不需要再次对模板图像进行多级特征提取。
在另一实施例中，也可以对待检测图像进行至少一级特征提取，将最后一级特征提取的输出，作为待检测图像的第一特征图；对模板图像进行至少一级特征提取，将最后一级特征提取的输出，作为模板图像的第二特征图。
III：在上述S103中，在将第一特征图和第二特征图进行特征混淆处理的的情况下，例如可以针对每张第一特征图，对所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图进行特征混淆处理，得到所述每张第一特征图对应的特征融合图像。
示例性的，参见图3b所示，本公开实施例还提供一种对第一特征图、以及与该张第一特征图对应的第二特征图进行特征混淆处理的方法，包括：
S301：基于该张第一特征图、以及与该张第一特征图对应的第二特征图，对该张第一特征图对应的第二特征图进行特征增强处理，得到该张第一特征图对应的第二特征图的特征增强图像。
在一些实施例中，可以基于归一化互相关匹配算法(Normalized Cross Correlation，NCC)，从第二特征图F 2中，确定出与第一特征点匹配的目标第二目标特征点f 2，再针对第一特征图中的每个第一特征点位置匹配的目标第二特征点f 2分别进行特征增强，得到第二特征图的特征增强图像F 22；如图4a所示。
在一些实施例中，例如可以采用下述方式对该张第一特征图对应的第二特征图进行特征增强处理：
针对该张第一特征图中的每个第一特征点，从该张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；基于该第一特征点与每个关联特征点之间的相似度，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
示例性的，针对每个第一特征点，若第一特征图和第二特征图的尺寸均为n×m，其中：
第一特征图表示为：
Figure PCTCN2021089654-appb-000001
第二特征图表示为：
Figure PCTCN2021089654-appb-000002
对于第一特征图中的任一第一特征点a ij，与之位置匹配的目标第二特征点为：b ij。
任一第一特征点对应的多个关联特征点，例如是在第二特征图中，与目标第二特征点之间的距离小于预设的某一距离阈值的第二特征点。
示例性的，该距离例如为L1距离、L2距离、欧式距离、曼哈顿距离中任一种。
在为每个第一特征点确定多个关联特征点的情况下，首先从第二特征图中，为该第一特征点确定位置匹配的目标第二特征点，然后将第二特征图中，与目标第二特征点之间的距离满足预设条件的所有第二特征点均作为该第一特征点对应的多个关联特征点。也可以将第二特征图中，与目标第二特征点之间的距离满足预设条件的所有第二特征点作为备选特征点，然后按照随机采样、或者均匀间隔采样的方式，从多个备选特征点中确定多个关联特征点。
在确定了第一特征点的多个关联特征点后，例如可以采用下述方式确定各个关联特征点和第一特征点之间的相似度：
基于所述第一特征点在所述第一特征图中的位置、以及预设的距离阈值，得到第一特征子图；以及基于与该第一特征点对应的任一关联特征点在所述第二特征图中的位置、以及所述预设的距离阈值，得到第二特征子图；基于所述第一特征子图、以及所述第二特征子图，确定所述第一特征点与该第一特征点对应的该任一关联特征点之间的相似度。这里，预设的距离阈值可以根据需要设置，对此，本公开实施例不作限制。
在一种可能的实施方式中，在确定第一特征点对应的第一特征子图的情况下，例如可以在第一特征图上，以所述第一特征点为圆心、以该预设的距离阈值为半径的第一圆形区域，基于所述第一特征图上位于该第一圆形区域内的第一特征点，得到所述第一特征子图。
此处，第一特征子图中的第一特征点，可以包括位于第一圆形区域内的所有第一特征点，也可以仅仅包括位于第一圆形区域内的部分第一特征点。
在一些实施例中，在确定任一关联特征点对应的第二特征子图的情况下，例如可以在所述第二特征图上，确定以该任一每个关联特征点为圆心、以该预设的距离阈值为半径的第二圆形区域，并基于所述第二特征图上位于该第二圆形区域内的第二特征点，得到所述第二特征子图。
此处，第二特征子图中的第二特征点，可以包括位于第二圆形区域内的所有第二特征点，也可以仅仅包括位于第二圆形区域内的部分第二特征点。
示例性的，在第一特征子图中的第一特征点仅仅包括位于第一圆形区域内的部分第一特征点的情况下，第二特征子图中的第二特征点也仅仅包括第二圆形区域内的部分第二特征点；且第一特征子图中的第一特征点和第二子图中的第二特征点位置一一匹配。
在另一种可能的实施方式中，在确定第一特征点对应的第一特征子图的情况下，例如可以在所述第一特征图上，确定以第一特征点为中心、以确定的所述目标边长为边长的第一正方形区域，基于所述第一特征图上位于该第一正方形区域内的第一特征点，得到所述第一特征子图。
在确定第一特征点对应的第一特征子图的情况下，第一特征子图包括的第一特征点，例如包括位于第一正方形区域内的所有第一特征点，也可以仅仅包括位于第一正方形区域内的部分第一特征点。
在一些实施例中，在确定任一关联特征点对应的第二特征子图的情况下，例如还可以基于所述预设的距离阈值，确定目标边长；在所述第二特征图上，确定以所述每个关联特 征点为中心、以确定的所述目标边长为边长的第二正方形区域，基于所述第二特征图上位于该第二正方形区域内的第二特征点，得到所述第二特征子图。
在确定关联特征点对应的第二特征子图的情况下，第二特征子图包括的第二特征点，例如包括位于第二正方形区域内的所有第二特征点，也可以仅仅包括位于第二正方形区域内的部分第二特征点。
在得到第一特征子图和第二特征子图后，基于第一特征子图确定第一特征点和该第一特征点对应的任一关联特征点之间的相似度。
示例性的，若与任一第一特征点对应的关联特征点有N个，则该第一特征点和第n个关联特征点之间的相似度NCC n满足下述公式(1)：
Figure PCTCN2021089654-appb-000003
其中，patch A表示第一特征子图；Patch Bn表示第n个关联特征点的第二特征子图；patch A*Patch Bn表示将第一特征子图和第n个关联特征点的第二特征子图进行矩阵乘法；sum(·)表示将矩阵中所有元素的元素值求和；N≥n≥1。
在得到第一特征点和每个关联特征点之间的相似度后，例如可以基于该第一特征点与每个关联特征点之间的相似度、以及每个关联特征点的特征值，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
示例性的，例如可以基于所述每个关联特征点与第一特征点之间的相似度，对多个所述关联特征点分别对应的特征值进行加权求和，得到第一和值；以及，对多个关联特征点分别对应的相似度进行求和，得到第二和值；将所述第一和值和所述第二和值的比值，作为对与第一特征点位置匹配的目标第二特征点进行特征增强处理后的特征值。
示例性的，对于任一第一特征点，对该第一特征点位置匹配的目标第二特征点进行特征增强处理后的特征值ft(B) 2满足下述公式(2)：
Figure PCTCN2021089654-appb-000004
其中，ft(B)′ n表征第n个关联特征点对应的特征值。
在针对第一特征图中的每个第一特征点位置匹配的目标第二特征点分别进行特征增强后，得到第二特征图的特征增强图像。
本公开实施例提供的特征混淆处理的过程还包括：
S302：基于该张第一特征图、以及与该张第一特征图对应的第二特征图，得到与该张第一特征图对应的注意力掩码图像；其中，所述注意力掩码图像中任一像素点的像素值，表征该第一特征图中位置与该任一像素点匹配的第一特征点存在缺陷的异常度值。
这里，需要注意的是，上述S301和S302并无先后逻辑关系。
在一些实施例中，例如可以采用下述方式得到与任一张第一特征图对应的注意力掩码图像：针对该张第一特征图中的每个第一特征点，从该张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；基于该第一特征点与每个关联特征点之间的相似度，确定该第一特征点的异常度值。
此处，第一特征点对应的关联特征点的方式、以及确定第一特征点与关联特征点之间的相似度的方式，与上述S301中类似，在此不再赘述。
在确定了第一特征点与每个关联特征点之间的相似度后，例如可以确定多个关联特征 点分别与该第一特征点之间的相似度的最大相似度；基于所述最大相似度，确定该第一特征点的异常度值。
任一第一特征点的异常度值S例如满足下述公式(3)：
S＝1-λ×H         (3)
其中，H表示最大相似度。λ为预设系数，例如为1、0.5等。具体可以根据实际的需要进行设定。
又例如，可以根据多个关联像素点分别与该第一像素点之间的相似度，确定相似度均值，并基于该相似度均值，确定该第一像素点的异常度值。
在确定了第一特征图中每个第一特征点对应的异常度值后，基于所述第一特征图中各个第一特征点对应的异常度值，得到所述注意力掩码图像；此时，例如可以将所有第一特征点分别对应的异常度值构成的图像，作为注意力掩码图像。
S303：基于所述特征增强图像、以所述注意力掩码图像，得到该张第一特征图对应的特征融合图像。
此处，例如可以对所述特征增强图像、以及该张第一特征图进行合并处理，得到该张第一特征图对应的合并特征图后，再基于所述注意力掩码图像、以及所述合并特征图，得到所述特征融合图像。
在一些实施例中，例如可以将特征增强图像和第一特征图进行叠加，得到合并特征图。
在基于注意力掩码图像、以及合并特征图得到特征融合图像的情况下，例如可以将注意力掩码图像和合并特征图进行矩阵相乘，得到特征融合图像。
在本公开实施例中，对第一特征图和阈值对应的第二特征图进行特征混淆处理的过程，例如可以利用预先训练的特征混淆网络来实现。
示例性的，参见图4b所示，本公开实施例还提供一种特征混淆网络的结构，包括：特征增强部分、以及异常注意力部分。
其中，异常注意力部分，被配置为基于上述S302提供的方法，得到第一特征图的注意力掩码图像。特征增强部分，被配置为基于上述S302提供的方法，得到与第一特征图对应的第二特征图对应的特征增强图像。
然后将特征增强图像、和第一特征图进行叠加，得到合并特征图；并基于注意力掩码图像和合并特征图，得到特征融合图像。
在另一实施例中，若第一特征图和第二特征图均仅有一个，也可以按照与上述类似的方式将第一特征图和第二特征图进行特征混淆处理，得到特征融合图像，具体的特征混淆处理方法在此不再赘述。
IV：在上述S104中，在基于特征融合图像，得到所述待检测图像的缺陷检测结果的情况下，响应于得到多级特征处理分别对应的第一特征图和第二特征图，例如可以基于所述每张第一特征图对应的特征融合图像，得到所述每张第一特征图的缺陷检测结果；基于与多级特征提取分别对应的第一特征图的缺陷检测结果，得到所述待检测图像的缺陷检测结果。
示例性的可以采用预先训练的检测网络，对特征融合图像进行缺陷检测处理，得到与待检测图像对应的缺陷检测结果。本公开实施例提供的预先训练的检测网络可以为：FCOS、Fovea、视网膜网络(RetinaNET)、faster R-CNN等；对于预先训练的检测网络可以根据需要设置，本公开实施例不作限制。
在一些实施例中，预先训练的检测网络可以采用全卷积逐像素目标检测(Fully Convolutional One-Stage Object Detection，FCOS)网络。其中FCOS网络能够缺陷类别、缺陷中心度、缺陷框在第一特征图中的位置。
其中，缺陷中心度用于第一特征图中的某个特征点为缺陷框中心的概率。
缺陷框的在第一特征图中的位置，指示了第一特征图中存在缺陷的位置。
示例性的，如图3a所示的示例中，通过FCOS检测头，得到四级特征提取分别对应的第一特征图的缺陷检测结果，该缺陷检测结果包括：缺陷类别、缺陷中心度、缺陷框的位置。
在得到多级特征提取中每一级特征提取对应的第一特征图的检测结果后，例如可以采用非极大抑制法(Non-Maximum Suppression，NMS)将多级特征提取分别对应的第一特征图的检测结果进行合并处理，得到待检测图像的缺陷检测结果。
又例如，还可以取多级特征提取分别对应的第一特征图的检测结果的交集，来确定待检测图像的缺陷检测结果。
本公开实施例通过对待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图图像，并对模板图像进行多级特征提取，获取每张第一特征图对应的第二特征图，然后针对每张第一特征图，对该张第一特征图、以及与该张第一特征图对应的第二特征图，进行特征混淆处理，得到该张第一特征图对应的特征融合图像，从而通过将待处理图像和模板图像中特征的融合，来减小第一特征图和第二特征图之间存在的生产误差、匹配误差、以及采集噪声等误差，然后利用特征融合图像，得到第一特征图的缺陷检测结果，进而综合多级特征提取分别对应的第一特征图的缺陷检测结果，以更高的精度获得待检测图像的缺陷检测结果。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
基于同一发明构思，本公开实施例中还提供了与缺陷检测方法对应的缺陷检测装置，由于本公开实施例中的装置解决问题的原理与本公开实施例上述缺陷检测方法相似，因此装置的实施可以参见方法的实施，重复之处不再赘述。
参照图5所示，为本公开实施例提供的一种缺陷检测装置的示意图，所述装置包括：获取部分51、特征提取部分52、特征混淆部分53、以及检测部分54；其中，
获取部分51，被配置为获取待检测图像、以及模板图像；
特征提取部分52，被配置为对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图，并对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图；
特征混淆部分53，被配置为将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像；
检测部分54，被配置为基于所述特征融合图像，得到所述待检测图像的缺陷检测结果。
一种可能的实施方式中，所述特征提取部分52，在对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图的情况下，被配置为对所述待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图；所述特征提取部分52，在对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图的情况下，被配置为对所述模板图像进行多级特征提取，获取与每张所述第一特征图对应的第二特征图；所述特征混淆部分53，在将将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像的情况下，被配置为针对每张第一特征图，对所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图进行特征混淆处理，得到所述每张第一特征图对应的特征融合图像。
一种可能的实施方式中，检测部分54，在基于所述特征融合图像，得到所述待检测图像的缺陷检测结果的情况下，被配置为：基于所述每张第一特征图对应的特征融合图像，得到所述每张第一特征图的缺陷检测结果；基于与多级特征提取分别对应的第一特征图的缺陷检测结果，得到所述待检测图像的缺陷检测结果。
一种可能的实施方式中，所述特征提取部分52，在对所述待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图的情况下，被配置为：对所述待检测图像进行多级特征提取，获取与每级特征提取对应的中间特征图；在所述每级特征提取为最后一级 特征提取的情况下，将最后一级特征提取对应的中间特征图，作为该最后一级特征提取对应的第一特征图；在所述每级特征提取为除最后一级特征提取的其他级特征提取的情况下，将与所述每级特征提取对应的中间特征图与该级特征提取的下一级特征提取对应的第一特征图进行特征融合，得到与所述每级特征提取对应的第一特征图。
一种可能的实施方式中，所述特征提取部分52，在将与每级特征提取对应的中间特征图，与该级特征提取的下一级特征提取对应的第一特征图进行特征融合，得到与所述每级特征提取对应的第一特征图的情况下，被配置为：将与该级特征提取的下一级特征提取对应的第一特征图进行上采样，得到上采样向量；将所述上采样向量与该级特征提取对应的中间特征图进行叠加后，得到该级特征提取对应的第一特征图。
一种可能的实施方式中，所述特征混淆部分53，在对所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图进行特征混淆处理的情况下，被配置为：基于所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图，对所述第一特征图对应的第二特征图进行特征增强处理，得到该张第一特征图对应的第二特征图的特征增强图像；以及基于所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图，得到与所述每张第一特征图对应的注意力掩码图像；其中，所述注意力掩码图像中任一像素点的像素值，表征该第一特征图中位置与该任一像素点匹配的第一特征点存在缺陷的异常度值；基于所述特征增强图像、以及所述注意力掩码图像，得到所述每张第一特征图对应的特征融合图像。
一种可能的实施方式中，所述特征混淆部分53，在基于该张第一特征图、以及与该张第一特征图对应的第二特征图，对该张第一特征图对应的第二特征图进行特征增强处理的情况下，被配置为：针对该张第一特征图中的每个第一特征点，从该张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；基于该第一特征点与每个关联特征点之间的相似度，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
一种可能的实施方式中，所述特征混淆部分53，在基于该第一特征点与每个关联特征点之间的相似度，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理的情况下，被配置为：基于该第一特征点与每个关联特征点之间的相似度、以及每个关联特征点的特征值，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
一种可能的实施方式中，所述特征混淆部分53，在基于该第一特征点与每个关联特征点之间的相似度、以及每个关联特征点的特征值，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理的情况下，被配置为：基于该第一特征点与每个关联特征点之间的相似度，对该第一特征点对应的多个关联特征点分别对应的特征值进行加权求和，得到第一和值；对多个关联特征点分别对应的相似度进行求和，得到第二和值；将所述第一和值和所述第二和值的比值，作为对所述目标第二特征点进行特征增强处理后的特征值。
一种可能的实施方式中，所述特征混淆部分53，在基于该张第一特征图、以及与该张第一特征图对应的第二特征图，得到与该张第一特征图对应的注意力掩码图像的情况下，被配置为：针对该张第一特征图中的每个第一特征点，从该张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；基于该第一特征点与每个关联特征点之间的相似度，确定该第一特征点的异常度值；基于所述第一特征图中各个第一特征点对应的异常度值，得到所述注意力掩码图像。
一种可能的实施方式中，所述特征混淆部分53，在基于该第一特征点与每个关联特征点之间的相似度，确定该第一特征点的异常度值的情况下，被配置为：确定多个关联特征点分别与该第一特征点之间的相似度的最大相似度；基于所述最大相似度，确定该第一特 征点的异常度值。
一种可能的实施方式中，所述特征混淆部分53，被配置为采用下述方式确定第一特征点、和与该第一特征点对应的任一关联特征点之间的相似度：基于所述第一特征点在所述第一特征图中的位置、以及预设的距离阈值，得到第一特征子图；以及基于与该第一特征点对应的任一关联特征点在所述第二特征图中的位置、以及所述距离阈值，得到第二特征子图；基于所述第一特征子图、以及所述第二特征子图，确定所述第一特征点与该第一特征点对应的该任一关联特征点之间的相似度。
一种可能的实施方式中，所述特征混淆部分53，在基于所述特征增强图像、以所述注意力掩码图像，得到该张第一特征图对应的特征融合图像的情况下，被配置为：对所述特征增强图像、以及该张第一特征图进行合并处理，得到该张第一特征图对应的合并特征图；基于所述注意力掩码图像、以及所述合并特征图，得到所述特征融合图像。
在本公开实施例以及其他的实施例中，“部分”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是单元，还可以是模块也可以是非模块化的。
关于装置中的各模块的处理流程、以及各模块之间的交互流程的描述可以参照上述方法实施例中的相关说明，这里不再详述。
本公开实施例还提供了一种计算机设备10，如图6所示，为本公开实施例提供的计算机设备10结构示意图，包括：
处理器11和存储器12；所述存储器12存储有所述处理器11可执行的机器可读指令，当计算机设备运行时，所述机器可读指令被所述处理器执行以实现下述步骤：
获取待检测图像、以及模板图像；
对所述待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图，并对所述模板图像进行多级特征提取，获取与每张所述第一特征图对应的第二特征图；
针对每张第一特征图，对该张第一特征图、以及与该张第一特征图对应的第二特征图进行特征混淆处理，得到该张第一特征图对应的特征融合图像；
基于所述特征融合图像，得到该张第一特征图的缺陷检测结果；
基于与多级特征提取分别对应的第一特征图的缺陷检测结果，得到所述待检测图像的缺陷检测结果。
上述指令的具体执行过程可以参考本公开实施例中所述的缺陷检测方法的步骤，此处不再赘述。
本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述方法实施例中所述的缺陷检测方法的步骤。其中，该存储介质可以是易失性或非易失的计算机可读取存储介质。
本公开实施例所提供的缺陷检测方法的计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可用于执行上述方法实施例中所述的缺陷检测方法的步骤，具体可参见上述方法实施例，在此不再赘述。
本公开实施例还提供一种计算机程序，该计算机程序被处理器执行时实现前述实施例的任意一种方法。该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。在本公开所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可 以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应所述以权利要求的保护范围为准。
工业实用性
本公开实施例中，通过将待处理图像和模板图像中特征的融合，来减小第一特征图和第二特征图之间存在的生产误差、匹配误差、以及采集噪声等误差，然后利用特征融合图像，得到第一特征图的缺陷检测结果，进而综合多级特征提取分别对应的第一特征图的缺陷检测结果，以更高的精度获得待检测图像的缺陷检测结果。

Claims (17)

  1. 一种缺陷检测方法，包括：
    获取待检测图像、以及模板图像；
    对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图，并对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图；
    将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像；
    基于所述特征融合图像，得到所述待检测图像的缺陷检测结果。
  2. 根据权利要求1所述的缺陷检测方法，其中，所述对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图，包括：
    对所述待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图；
    所述对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图，包括：
    对所述模板图像进行多级特征提取，获取与每张所述第一特征图对应的第二特征图；
    所述将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像，包括：
    针对每张第一特征图，对所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图进行特征混淆处理，得到所述每张第一特征图对应的特征融合图像。
  3. 根据权利要求2所述的缺陷检测方法，其中，所述基于所述特征融合图像，得到所述待检测图像的缺陷检测结果，包括：
    基于所述每张第一特征图对应的特征融合图像，得到所述每张第一特征图的缺陷检测结果；
    基于与多级特征提取分别对应的第一特征图的缺陷检测结果，得到所述待检测图像的缺陷检测结果。
  4. 根据权利要求2或3所述的缺陷检测方法，其中，所述对所述待检测图像进行多级特征提取，获取与每级特征提取对应的第一特征图，包括：
    对所述待检测图像进行多级特征提取，获取与每级特征提取对应的中间特征图；
    在所述每级特征提取为最后一级特征提取的情况下，将最后一级特征提取对应的中间特征图，作为该最后一级特征提取对应的第一特征图；
    在所述每级特征提取为除最后一级特征提取的其他级特征提取的情况下，将与所述每级特征提取对应的中间特征图与该级特征提取的下一级特征提取对应的第一特征图进行特征融合，得到与所述每级特征提取对应的第一特征图。
  5. 根据权利要求4所述的缺陷检测方法，其中，所述将与每级特征提取对应的中间特征图，与该级特征提取的下一级特征提取对应的第一特征图进行特征融合，得到与所述每级特征提取对应的第一特征图，包括：
    将与该级特征提取的下一级特征提取对应的第一特征图进行上采样，得到上采样向量；
    将所述上采样向量与该级特征提取对应的中间特征图进行叠加后，得到该级特征提取对应的第一特征图。
  6. 根据权利要求2-5任一项所述的缺陷检测方法，其中，所述对所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图进行特征混淆处理，包括：
    基于所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图，对所述第一特征图对应的第二特征图进行特征增强处理，得到该张第一特征图对应的第二特征图的特征增强图像；以及
    基于所述每张第一特征图、以及与所述每张第一特征图对应的第二特征图，得到与所 述每张第一特征图对应的注意力掩码图像；其中，所述注意力掩码图像中任一像素点的像素值，表征该第一特征图中位置与该任一像素点匹配的第一特征点存在缺陷的异常度值；
    基于所述特征增强图像、以及所述注意力掩码图像，得到所述每张第一特征图对应的特征融合图像。
  7. 根据权利要求6所述的缺陷检测方法，其中，所述基于每张第一特征图、以及与所述每张第一特征图对应的第二特征图，对所述第一特征图对应的第二特征图进行特征增强处理，包括：
    针对该张第一特征图中的每个第一特征点，从该张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；
    基于该第一特征点与每个关联特征点之间的相似度，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
  8. 根据权利要求7所述的缺陷检测方法，其中，所述基于该第一特征点与每个关联特征点之间的相似度，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理，包括：
    基于该第一特征点与每个关联特征点之间的相似度、以及每个关联特征点的特征值，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理。
  9. 根据权利要求8所述的缺陷检测方法，其中，所述基于该第一特征点与每个关联特征点之间的相似度、以及每个关联特征点的特征值，对与该第一特征点位置匹配的目标第二特征点进行特征增强处理，包括：
    基于该第一特征点与每个关联特征点之间的相似度，对该第一特征点对应的多个关联特征点分别对应的特征值进行加权求和，得到第一和值；
    对多个关联特征点分别对应的相似度进行求和，得到第二和值；
    将所述第一和值和所述第二和值的比值，作为对所述目标第二特征点进行特征增强处理后的特征值。
  10. 根据权利要求6-9任一项所述的缺陷检测方法，其中，所述基于该张第一特征图、以及与该张第一特征图对应的第二特征图，得到与该张第一特征图对应的注意力掩码图像，包括：
    针对该张第一特征图中的每个第一特征点，从该张第一特征图对应的第二特征图的多个第二特征点中，确定与该第一特征点对应的多个关联特征点；其中，该第一特征点对应的各个关联特征点，与该第一特征点位置匹配的目标第二特征点之间的距离满足预设条件；
    基于该第一特征点与每个关联特征点之间的相似度，确定该第一特征点的异常度值；
    基于所述第一特征图中各个第一特征点对应的异常度值，得到所述注意力掩码图像。
  11. 根据权利要求10所述的缺陷检测方法，其中，所述基于该第一特征点与每个关联特征点之间的相似度，确定该第一特征点的异常度值，包括：
    确定多个关联特征点分别与该第一特征点之间的相似度的最大相似度；
    基于所述最大相似度，确定该第一特征点的异常度值。
  12. 根据权利要求7-11任一项所述的缺陷检测方法，采用下述方式确定第一特征点、和与该第一特征点对应的任一关联特征点之间的相似度：
    基于所述第一特征点在所述第一特征图中的位置、以及预设的距离阈值，得到第一特征子图；以及
    基于与该第一特征点对应的任一关联特征点在所述第二特征图中的位置、以及所述距离阈值，得到第二特征子图；
    基于所述第一特征子图、以及所述第二特征子图，确定所述第一特征点与该第一特征 点对应的该任一关联特征点之间的相似度。
  13. 根据权利要求6-12任一项所述的缺陷检测方法，其中，所述基于所述特征增强图像、以及所述注意力掩码图像，得到所述每张第一特征图对应的特征融合图像，包括：
    对所述特征增强图像、以及该张第一特征图进行合并处理，得到该张第一特征图对应的合并特征图；
    基于所述注意力掩码图像、以及所述合并特征图，得到所述特征融合图像。
  14. 一种缺陷检测装置，包括：
    获取部分，被配置为获取待检测图像、以及模板图像；
    特征提取部分，被配置为对所述待检测图像进行特征提取，得到所述待检测图像的第一特征图，并对所述模板图像进行特征提取，得到与所述模板图像对应的第二特征图；
    特征混淆部分，被配置为将所述第一特征图、以及所述第二特征图进行特征混淆处理，得到特征融合图像；
    检测部分，被配置为基于所述特征融合图像，得到所述待检测图像的缺陷检测结果。
  15. 一种计算机设备，包括：相互连接的处理器和存储器，所述存储器存储有所述处理器可执行的机器可读指令，当计算机设备运行时，所述机器可读指令被所述处理器执行以实现如权利要求1至13任一所述的缺陷检测方法。
  16. 一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行如权利要求1至13任一所述的缺陷检测方法。
  17. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算机设备中运行时，所述计算机设备中的处理器执行如权利要求1至13任一所述的缺陷检测方法。
PCT/CN2021/089654 2020-10-30 2021-04-25 缺陷检测方法、装置、计算机设备及存储介质 WO2022088628A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020217037651A KR20220058842A (ko) 2020-10-30 2021-04-25 결함 검출 방법, 장치, 컴퓨터 기기 및 저장 매체
JP2021566097A JP2023503751A (ja) 2020-10-30 2021-04-25 欠陥検出方法、装置、コンピュータ機器および記憶媒体

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011191743.3A CN112288723B (zh) 2020-10-30 2020-10-30 缺陷检测方法、装置、计算机设备及存储介质
CN202011191743.3 2020-10-30

Publications (1)

Publication Number Publication Date
WO2022088628A1 true WO2022088628A1 (zh) 2022-05-05

Family

ID=74352960

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/089654 WO2022088628A1 (zh) 2020-10-30 2021-04-25 缺陷检测方法、装置、计算机设备及存储介质

Country Status (4)

Country Link
JP (1) JP2023503751A (zh)
KR (1) KR20220058842A (zh)
CN (1) CN112288723B (zh)
WO (1) WO2022088628A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115496976A (zh) * 2022-08-29 2022-12-20 锋睿领创(珠海)科技有限公司 多源异构数据融合的视觉处理方法、装置、设备及介质
CN115564775A (zh) * 2022-12-05 2023-01-03 北京矩视智能科技有限公司 基于轻量级注意力机制的缺陷检测方法、装置及存储介质
CN115965856A (zh) * 2023-02-23 2023-04-14 深圳思谋信息科技有限公司 图像检测模型构建方法、装置、计算机设备及存储介质
CN116883417A (zh) * 2023-09-08 2023-10-13 武汉东方骏驰精密制造有限公司 基于机器视觉的工件质检方法及装置
CN117495884A (zh) * 2024-01-02 2024-02-02 湖北工业大学 一种钢铁表面缺陷分割方法、装置、电子设备及存储介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112288723B (zh) * 2020-10-30 2023-05-23 北京市商汤科技开发有限公司 缺陷检测方法、装置、计算机设备及存储介质
CN112801047B (zh) * 2021-03-19 2021-08-17 腾讯科技（深圳）有限公司 缺陷检测方法、装置、电子设备及可读存储介质
CN112967264A (zh) * 2021-03-19 2021-06-15 深圳市商汤科技有限公司 缺陷检测方法及装置、电子设备和存储介质
CN113406092B (zh) * 2021-08-18 2022-01-11 阿里巴巴（中国）有限公司 数字化生产检测系统、方法、装置、设备及存储介质
CN113744268B (zh) * 2021-11-04 2022-04-22 深圳市城市交通规划设计研究中心股份有限公司 一种裂纹检测方法、电子设备及可读存储介质
CN116071356B (zh) * 2023-03-06 2023-06-16 中国市政工程西南设计研究总院有限公司 用于设计企业的设计图缺陷智慧检查平台及检查方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170249732A1 (en) * 2016-02-25 2017-08-31 Kla-Tencor Corporation Method and System for Detecting Defects on a Substrate
CN110807362A (zh) * 2019-09-23 2020-02-18 腾讯科技（深圳）有限公司 一种图像检测方法、装置和计算机可读存储介质
CN111461260A (zh) * 2020-04-29 2020-07-28 上海东普信息科技有限公司 基于特征融合的目标检测方法、装置、设备及存储介质
CN111814905A (zh) * 2020-07-23 2020-10-23 上海眼控科技股份有限公司 目标检测方法、装置、计算机设备和存储介质
CN112288723A (zh) * 2020-10-30 2021-01-29 北京市商汤科技开发有限公司 缺陷检测方法、装置、计算机设备及存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101872475B (zh) * 2009-04-22 2012-03-28 中国科学院自动化研究所 一种扫描文档图像自动配准方法
CN103456009B (zh) * 2013-08-28 2016-04-20 深圳市中瀛鑫科技股份有限公司 目标检测方法与装置、监控系统
CN107464230B (zh) * 2017-08-23 2020-05-08 京东方科技集团股份有限公司 图像处理方法及装置
CN107590453B (zh) * 2017-09-04 2019-01-11 腾讯科技（深圳）有限公司 增强现实场景的处理方法、装置及设备、计算机存储介质
CN109118514B (zh) * 2018-06-11 2022-07-15 西安电子科技大学 一种目标跟踪方法
CN110688951B (zh) * 2019-09-26 2022-05-31 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN111179253B (zh) * 2019-12-30 2023-11-24 歌尔股份有限公司 一种产品缺陷检测方法、装置与系统
CN111160477B (zh) * 2019-12-31 2024-03-19 哈尔滨智兀科技有限公司 一种基于特征点检测的图像模板匹配方法
CN111429403B (zh) * 2020-02-26 2022-11-08 北京航空航天大学杭州创新研究院 一种基于机器视觉的汽车齿轮成品缺陷检测方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170249732A1 (en) * 2016-02-25 2017-08-31 Kla-Tencor Corporation Method and System for Detecting Defects on a Substrate
CN110807362A (zh) * 2019-09-23 2020-02-18 腾讯科技（深圳）有限公司 一种图像检测方法、装置和计算机可读存储介质
CN111461260A (zh) * 2020-04-29 2020-07-28 上海东普信息科技有限公司 基于特征融合的目标检测方法、装置、设备及存储介质
CN111814905A (zh) * 2020-07-23 2020-10-23 上海眼控科技股份有限公司 目标检测方法、装置、计算机设备和存储介质
CN112288723A (zh) * 2020-10-30 2021-01-29 北京市商汤科技开发有限公司 缺陷检测方法、装置、计算机设备及存储介质

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115496976A (zh) * 2022-08-29 2022-12-20 锋睿领创(珠海)科技有限公司 多源异构数据融合的视觉处理方法、装置、设备及介质
CN115496976B (zh) * 2022-08-29 2023-08-11 锋睿领创(珠海)科技有限公司 多源异构数据融合的视觉处理方法、装置、设备及介质
CN115564775A (zh) * 2022-12-05 2023-01-03 北京矩视智能科技有限公司 基于轻量级注意力机制的缺陷检测方法、装置及存储介质
CN115965856A (zh) * 2023-02-23 2023-04-14 深圳思谋信息科技有限公司 图像检测模型构建方法、装置、计算机设备及存储介质
CN115965856B (zh) * 2023-02-23 2023-05-30 深圳思谋信息科技有限公司 图像检测模型构建方法、装置、计算机设备及存储介质
CN116883417A (zh) * 2023-09-08 2023-10-13 武汉东方骏驰精密制造有限公司 基于机器视觉的工件质检方法及装置
CN116883417B (zh) * 2023-09-08 2023-12-05 武汉东方骏驰精密制造有限公司 基于机器视觉的工件质检方法及装置
CN117495884A (zh) * 2024-01-02 2024-02-02 湖北工业大学 一种钢铁表面缺陷分割方法、装置、电子设备及存储介质
CN117495884B (zh) * 2024-01-02 2024-03-22 湖北工业大学 一种钢铁表面缺陷分割方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
JP2023503751A (ja) 2023-02-01
CN112288723B (zh) 2023-05-23
KR20220058842A (ko) 2022-05-10
CN112288723A (zh) 2021-01-29

Similar Documents

Publication Publication Date Title
WO2022088628A1 (zh) 缺陷检测方法、装置、计算机设备及存储介质
Yi et al. An end‐to‐end steel strip surface defects recognition system based on convolutional neural networks
EP3660743B1 (en) Systems and methods for automating information extraction from piping and instrumentation diagrams
JP6710135B2 (ja) 細胞画像の自動分析方法及びシステム
WO2019233166A1 (zh) 一种表面缺陷检测方法、装置及电子设备
US10891476B2 (en) Method, system, and neural network for identifying direction of a document
WO2022088663A1 (zh) 缺陷检测方法及装置、电子设备和存储介质
WO2021217857A1 (zh) 切片缺陷检测方法、装置、电子设备及可读存储介质
CN110245714B (zh) 图像识别方法、装置及电子设备
CN112580668B (zh) 一种背景欺诈检测方法、装置及电子设备
CN111680755B (zh) 医学图像识别模型构建及医学图像识别方法、装置、介质及终端
CN109101866B (zh) 基于分割剪影的行人再识别方法及系统
CN110633711B (zh) 训练特征点检测器的计算机装置、方法及特征点检测方法
CN116596875B (zh) 晶圆缺陷检测方法、装置、电子设备及存储介质
CN111626295A (zh) 车牌检测模型的训练方法和装置
CN112598657A (zh) 缺陷检测方法、装置、模型构造方法和计算机设备
WO2022227342A1 (zh) 眼底图像黄斑区域的识别检测方法和装置及设备
CN114897764A (zh) 基于标准化通道注意力的肺结节假阳性排除方法及装置
US20190370940A1 (en) Method and apparatus with feature data generation
CN114004289A (zh) 一种车辆损伤识别模型建立方法、损伤识别方法及装置
KR101821770B1 (ko) 특징 추출을 위한 기법들
CN116542980A (zh) 缺陷检测方法、装置、计算机设备、存储介质和程序产品
CN114692778B (zh) 用于智能巡检的多模态样本集生成方法、训练方法及装置
Wang et al. Oil tank detection via target-driven learning saliency model
CN115661828B (zh) 一种基于动态分层嵌套残差网络的文字方向识别方法

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021566097

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21884371

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21884371

Country of ref document: EP

Kind code of ref document: A1