CN111738322B - Method, device, equipment and medium for detecting surface defects of article - Google Patents

Method, device, equipment and medium for detecting surface defects of article Download PDF

Info

Publication number
CN111738322B
CN111738322B CN202010538153.7A CN202010538153A CN111738322B CN 111738322 B CN111738322 B CN 111738322B CN 202010538153 A CN202010538153 A CN 202010538153A CN 111738322 B CN111738322 B CN 111738322B
Authority
CN
China
Prior art keywords
defect
defect type
detection model
image
intermediate image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010538153.7A
Other languages
Chinese (zh)
Other versions
CN111738322A (en
Inventor
张滨
冯原
韩树民
辛颖
苑鹏程
朱剑锋
林书妃
王晓迪
刘静伟
徐英博
文石磊
章宏武
丁二锐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202010538153.7A priority Critical patent/CN111738322B/en
Publication of CN111738322A publication Critical patent/CN111738322A/en
Application granted granted Critical
Publication of CN111738322B publication Critical patent/CN111738322B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the application discloses a method, a device, equipment and a medium for detecting surface defects of an article, relates to the fields of artificial intelligent computer vision, deep learning and cloud computing, and particularly relates to an image recognition technology. The specific implementation scheme is as follows: inputting the surface image of the object to be detected into a defect detection model to identify and determine the defect type; the defect detection model is a deep learning model and comprises at least two layers of feature extraction networks; extracting intermediate image features output by at least one layer of feature extraction network in the identification process of the defect detection model; matching the intermediate image features with intermediate image features in a preset intermediate image feature library; and correcting the defect type determined by the defect detection model according to the matching result. According to the embodiment of the application, when the newly added defect types are identified, the defect detection model does not need to be updated, and the newly added defect types can be directly matched through the preset intermediate image feature library, so that the detection efficiency is further improved.

Description

Method, device, equipment and medium for detecting surface defects of article
Technical Field
The embodiment of the application relates to the field of artificial intelligent computer vision, deep learning and cloud computing, in particular to an image recognition technology.
Background
In the field of manufacturing, there is a need for surface defect detection of manufactured products. Taking a textile cloth as an example, defect detection needs to be carried out on surface flaws of the textile cloth, and defect types need to be distinguished.
The existing defect detection methods have strong pertinence to the detection of specific types of defects, and once new types of defects appear, the original detection accuracy is reduced, or the new types of defects can be detected only by retraining a deep-learning detection model. Therefore, the training cost of the detection model is high, and the detection model cannot timely adapt to the detection requirement of the new type of defects.
Disclosure of Invention
The embodiment of the application provides a method, a device, equipment and a medium for detecting surface defects of an article.
According to a first aspect, there is provided a method of detecting surface defects of an article, comprising:
inputting the surface image of the object to be detected into a defect detection model to identify and determine the defect type; the defect detection model is a deep learning model and comprises at least two layers of feature extraction networks;
extracting intermediate image features output by at least one layer of feature extraction network in the identification process of the defect detection model;
Matching the intermediate image features with intermediate image features in a preset intermediate image feature library;
correcting the defect type determined by the defect detection model according to the matching result; the intermediate image features in the preset intermediate image feature library are features extracted from at least one layer of feature extraction network after the input of the defect detection model based on the object surface image marked with the defect type.
According to a second aspect, there is provided an apparatus for detecting surface defects of an article, comprising:
the defect type identification module is used for inputting the surface image of the object to be detected into the defect detection model so as to identify and determine the defect type; the defect detection model is a deep learning model and comprises at least two layers of feature extraction networks;
the image feature extraction module is used for extracting intermediate image features output by at least one layer of feature extraction network in the identification process of the defect detection model;
the image feature matching module is used for matching the intermediate image features with intermediate image features in a preset intermediate image feature library;
the defect type correction module is used for correcting the defect type determined by the defect detection model according to the matching result; the intermediate image features in the preset intermediate image feature library are features extracted from at least one layer of feature extraction network after the input of the defect detection model based on the object surface image marked with the defect type.
According to a third aspect, there is provided an electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method for detecting surface defects of an article provided by any of the embodiments of the present application.
According to a fourth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of detecting surface defects of an article provided by any of the embodiments of the application.
The technology improves the detection efficiency of the surface image of the object.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are included to provide a better understanding of the present application and are not to be construed as limiting the application. Wherein:
FIG. 1 is a schematic flow chart of a method for detecting surface defects of an article according to an embodiment of the present application;
FIG. 2 is a flow chart of another method for detecting surface defects of an article according to an embodiment of the present application;
FIG. 3 is a schematic view of an anomaly detection sliding window according to defect type localization provided in accordance with an embodiment of the present application;
FIG. 4 is a schematic diagram of defect location information for detecting an image of a surface of an object according to a defect detection model according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of an apparatus for detecting surface defects of an article according to an embodiment of the present application;
fig. 6 is a block diagram of an electronic device for implementing a method of detecting surface defects of an article according to an embodiment of the application.
Detailed Description
Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic flow chart of a method for detecting surface defects of an article according to an embodiment of the present application, which is applicable to a situation of detecting defects of an image of a surface of an article. The method can be implemented by a device for detecting surface defects of an article, which can be implemented in hardware and/or software and can be configured in electronic equipment. The method specifically comprises the following steps:
S110, inputting the surface image of the object to be detected into a defect detection model to identify and determine the defect type; the defect detection model is a deep learning model and comprises at least two layers of feature extraction networks.
In the embodiment of the application, the defect detection model is a non-artificial network model constructed based on deep learning and is mainly used for carrying out defect identification on the surface image of the object to be detected so as to identify the defect type of the surface image of the object to be detected; the defect types of the surface images of the object to be detected can be different for different object surfaces. For textiles, for example, defect types may include holes, broken needles, run lines, etc.
Specifically, the defect detection model in the embodiment of the application is set to comprise at least two layers of feature extraction networks, and can extract feature information of different layers of the surface image of the object to be detected by finely dividing the surface image of the object to be detected under different layers and extracting the feature information of different layers of the surface image of the object to be detected aiming at the surface image of the object to be detected with different complexity, so that the extracted feature information can more comprehensively represent the features of the surface image of the object to be detected.
S120, extracting intermediate image features output by at least one layer of feature extraction network in the identification process of the defect detection model.
In the embodiment of the application, the intermediate image features are the high-dimensional information of the surface image of the object to be detected, which is extracted by the feature extraction network layer of the defect detection model, and the high-dimensional information can more completely represent the surface image of the object to be detected, so that the defect type determined by the defect detection model can be accurately and effectively judged in the follow-up process according to the intermediate image features. For example, the feature extracted by the convolution layer as a feature extraction network is a medium-level image feature, which is the feature output by processing the surface image of the object through the convolution layer. For defect recognition models, one or more layers of feature extraction networks, such as a U-net model made up of multiple convolution layers, may be generally included, where each convolution layer may serve as a feature extraction network for extracting mid-level image features, and it may be determined which network layers to extract mid-level image features from, if desired.
And S130, matching the intermediate image features with intermediate image features in a preset intermediate image feature library.
In the embodiment of the application, different defect types and corresponding different intermediate image features are stored in a preset intermediate image feature library, each intermediate image feature is associated with one defect type, when the intermediate image features are matched by utilizing the preset intermediate image feature library, the same intermediate image features can be searched through traversing the intermediate image features in the preset intermediate image feature library, and the defect types associated with the intermediate image features, the matching degree of which reaches the set condition, are used as the defect types matched according to the intermediate image features. The preset intermediate image feature library may input a defect image of a known defect type into the defect detection model, and extract the intermediate image features extracted from the network from the respective features. The medium-level image features output by different feature extraction networks can be stored, compared and matched respectively.
Specifically, if the defect type of the surface image of the object to be detected belongs to a new type, that is, if the defect type of the surface image of the object to be detected does not belong to a new type, that is, the image sample of the defect detection model is trained, at this time, the accuracy of the defect type determined only by the defect detection model is low. In a preset intermediate image feature library, images of known new defect types can be input into a defect detection model and extracted from a feature extraction network layer. Compared with training a defect detection model, the method can extract intermediate image features of several groups of new defect types through fewer new defect type image samples, and therefore, the method is equivalent to a recognition mode capable of quickly establishing the new defect types.
S140, correcting the defect type determined by the defect detection model according to the matching result; the method comprises the steps of presetting intermediate image features in an intermediate image feature library to be features extracted from at least one layer of feature extraction network after inputting a defect detection model based on object surface images marked with defect types.
In the embodiment of the application, the object surface image is required to be manually marked, and then the intermediate image features are output according to the defect detection model and stored in the preset intermediate image feature library, so that the intermediate image features stored in the preset intermediate image feature library have higher accuracy in the defect types related to the intermediate image features.
Specifically, matching the intermediate image features with a preset intermediate image feature library to obtain defect types and comparing the defect types with defect types determined by a defect detection model; and correcting the defect type determined by the defect detection model according to the comparison result. According to the method and the device for detecting the surface image defects of the object to be detected, after the defect types of the surface image of the object to be detected are determined through the defect detection model, detection results of the defect detection model are corrected through the preset intermediate image feature library, and the accuracy of the defect types of the surface image of the object to be detected is effectively improved.
According to the technical scheme, the defect type of the surface image of the object to be detected is identified through the defect detection model, and the intermediate image characteristics output by at least one layer of characteristic extraction network in the identification process of the defect detection model are extracted; matching the intermediate image features with intermediate image features in a preset intermediate image feature library; and correcting the defect type determined by the defect detection model according to the matching result. According to the embodiment of the application, after the defect type of the surface image of the object to be detected is determined through the defect detection model, the detection result of the defect detection model is corrected through the preset intermediate image feature library, so that the accuracy of the defect type of the surface image of the object to be detected is effectively improved; when the new defect types are identified, a large number of image samples of the new defect types do not need to be collected, a large amount of time is spent on retraining the defect detection model, and the defect detection model can be directly matched through a preset intermediate image feature library, so that the detection efficiency is further improved.
The embodiment of the application also provides a preferred implementation mode of the method for detecting the surface defects of the object, and the surface images of the object to be detected can be rapidly determined according to the abnormal points. Fig. 2 is a flow chart of another method for detecting surface defects of an article according to an embodiment of the present application. The method specifically comprises the following steps:
s210, identifying abnormal points in the surface image of the original object to be detected based on an abnormal point identification algorithm.
When the initial position of the defect is determined conventionally, the sliding window mode is adopted, the image position is continuously traversed in the surface image of the original object to be detected, and a plurality of pseudo windows can be found in the traversing result, so that the calculated amount of the whole detection process is increased, and the detection process is complex. In order to solve the problem, the embodiment of the application adopts an abnormal point identification algorithm to generate a rectangular frame of which the position is to be accurately determined according to the known defect types; and identifying abnormal points in the surface image of the original object to be detected according to the determined rectangular frame.
S220, determining candidate defect positions according to the identified abnormal point positions.
In the embodiment of the application, an image containing abnormal point positions in the surface image of the original object to be detected is sent to a detection network for learning by using a deep learning supervision technology, so that the regressive detection frame positions (namely candidate defect positions) are obtained. Wherein, a surface image of the original object to be measured may include at least one detection frame position. As shown in fig. 4, wherein the odd columns are defect images and the even columns are detection positions for determining defect images.
S230, shearing the surface image of the original object to be detected according to the candidate defect positions to obtain the surface image of the object to be detected; the surface image of the object to be measured comprises at least one outlier.
In the embodiment of the application, in order to avoid the problem of complex detection process caused by simultaneous detection of a plurality of defect types in the surface image of the original object to be detected, the surface image of the original object to be detected is sheared to obtain a plurality of surface images of the object to be detected, so that the detection efficiency can be effectively improved. In addition, by cutting, the size of the defect image can be matched with the input size of the defect detection model.
According to the embodiment of the application, based on an abnormal point identification algorithm, abnormal points are positioned in the surface image of the original object to be detected through the determined detection frame, candidate defect positions are determined according to the abnormal point positions, and then the surface image of the original object to be detected is sheared according to the candidate defect positions, so that the surface image of the object to be detected is obtained. The problem of low recognition efficiency caused by the fact that defects are directly recognized in the surface image of the object to be detected in the traditional mode can be avoided, and the recognition speed of the defects of the surface image of the object to be detected is effectively increased.
S240, inputting the surface image of the object to be detected into a defect detection model to identify and determine the defect type; the defect detection model is a deep learning model and comprises at least two layers of feature extraction networks.
S250, extracting intermediate image features output by at least one layer of feature extraction network in the identification process of the defect detection model.
And S260, matching the intermediate image features with intermediate image features in a preset intermediate image feature library.
S270, correcting the defect type determined by the defect detection model according to the matching result; the method comprises the steps of presetting intermediate image features in an intermediate image feature library to be features extracted from at least one layer of feature extraction network after inputting a defect detection model based on object surface images marked with defect types.
Optionally, the defect detection model is obtained by training based on the object surface images corresponding to at least two defect types, and the marking defect type is the same as or different from the defect type used for training. The different defect type is a newly added defect type.
In the embodiment of the application, the object surface images corresponding to at least two defect types are used as training samples, and the defect detection model is trained to obtain the defect detection model; wherein the defect types of the object surface image are obtained based on manual labeling; according to the method and the device for training the defect detection model, the defect detection model is trained according to the surface images of the plurality of articles with known defect types, and the universality of the trained defect detection model can be effectively improved. Optionally, the defect detection model at least includes a convolutional neural network model, and when performing defect identification on a cut image of a new defect type, the defect detection model can directly determine the characteristics of a middle-level image according to the input cut image and match with the defect type in a preset middle-level image characteristic library, so as to rapidly determine the defect type of the cut image of the new defect type.
Optionally, after S240, the method further includes:
extracting primary image features from the surface image of the object to be detected; and matching the primary image features with the primary image features in a preset primary image feature library.
Accordingly, S270 includes: and correcting the defect type determined by the defect detection model according to the intermediate-level matching result and the primary matching result.
In the embodiment of the application, the primary image features are basic features of the surface image of the object to be detected; optionally, the primary image features include gray distribution features and/or moment features of the surface image of the object to be detected, and effective features of the surface image of the object to be detected can be simply and integrally expressed through the primary image features, so that the matching efficiency of defect types in a preset primary image feature library is improved. The gray distribution feature is, for example, a statistical histogram of the gray levels of pixels of an image of the surface of the object.
In a preset primary image feature library, matching defect types related to primary image features extracted from the surface image of an object to be detected, and correcting the defect types determined by the defect detection model according to the defect types obtained by the primary matching result and the defect types obtained by the intermediate matching result; the method comprises the steps that a primary image feature library and a middle image feature library are preset, and a user can be supported to flexibly configure and change; the accuracy of the defect types determined by the defect detection model can be effectively detected, so that the defect types with more accurate surface images of the to-be-detected object can be obtained.
Optionally, S270 includes:
if the matching result is that the defect type corresponding to the matched intermediate image features is consistent with the defect type determined by the identification, the defect type determined by the identification is confirmed to be the final defect type;
if the matching result is that the defect type corresponding to the matched intermediate image feature is inconsistent with the defect type determined by recognition, confirming that the matched defect type is the final defect type or reporting an abnormal recognition phenomenon.
In the embodiment of the application, various defect types of the surface image of the object to be detected can exist, however, the defect types related to the training sample of the defect detection model are marked only by manual operation, so the defect types have larger limitation; when the defect types matched by the intermediate image features are inconsistent with the defect types identified by the defect detection model, the defect types of the surface image of the object to be detected are possibly new defect types, and the matching result of the intermediate image features is compared with the identification result of the defect detection model if the defect types are not in the training sample of the defect detection model, so that the problem that the error of the detection result is large due to the defect types determined by the defect detection model can be effectively avoided, and the detection efficiency is effectively improved.
Optionally, after inputting the surface image of the object to be measured into the defect detection model, the method further includes: outputting the profile coordinates with defects.
In the embodiment of the application, the surface image of the to-be-detected object is input, and the outline coordinates with defects can be output through the defect detection model so as to map the outline coordinates of the defects into the surface image of the to-be-detected original object, and the defect coordinates in the surface image of the to-be-detected original object can be rapidly determined.
Optionally, S210 includes:
determining an anomaly detection window for each defect type, wherein the anomaly detection window comprises a lateral window and/or a longitudinal window;
traversing the surface image of the original object to be detected by adopting an abnormal detection sliding window, and identifying abnormal points in the abnormal detection sliding window based on an abnormal point identification algorithm.
In the embodiment of the application, the abnormal defect type corresponds to the defect type, so that the problem of occurrence of a pseudo window in the traditional detection method is solved, and the calculated amount and the target window position are reduced. Specifically, the size and aspect ratio of the abnormality detection window correspond to the defect type. FIG. 3 is a schematic view of an anomaly detection sliding window for initial localization of defect types according to an embodiment of the present application; for example, if five defect types are known, five corresponding abnormal detection sliding window sizes and length-width ratios may exist, and the five sliding windows are traversed through the surface image of the original object to be detected five times to locate abnormal points; wherein the anomaly detection sliding window may include both lateral sliding and vertical sliding. By utilizing the determined abnormal detection sliding window, abnormal points in the abnormal detection sliding window are identified based on an abnormal point identification algorithm, so that the detection rate and accuracy of the abnormal points can be effectively improved.
Optionally, the outlier recognition algorithm includes at least one of: an abnormal gray distribution algorithm, an abnormal edge recognition algorithm and an abnormal corner recognition algorithm.
In the embodiment of the application, the adopted abnormal point identification algorithm is not limited to the identification of the edge key points, and can also comprise identification based on gray distribution and abnormal angular point identification, so that the abnormal points can be rapidly and accurately identified according to different types of surface images of the original object to be detected.
Optionally, in this embodiment, the article is a piece of cloth, and before S240, the method further includes:
continuously performing image shooting and acquisition on the moving cloth through at least one shooting device to acquire images of all articles to be detected;
and respectively setting electronic identifiers for the images of the objects to be detected according to the shooting equipment and the shooting sequence.
Correspondingly, after S270, further includes: and storing the determined defect type corresponding to the electronic identifier.
In the embodiment of the application, when the cloth works normally, the shooting equipment (such as a camera) is adopted to shoot the cloth uninterruptedly; for example, the shooting scene can be about 1.5 m-2 m cloth width, 7 industrial cameras are used for collecting together, and the movement speed of cloth is estimated to be 0.5 m-1 m/s; each camera acquires a data format of about 512 x 512; a region of about 20cm x 20cm corresponding to the cloth; the moving speed can finish the test length of at least 30m per minute, which is twice the efficiency of manual cloth inspection. For a plurality of cameras, the cameras need to be numbered, for example, the numbers acquired by the N cameras can be 1,2, … and N respectively; the camera shooting order number Q is 1,2, …, Q. If one image of the object to be detected is obtained through the 3 rd camera and the 5 th shooting of the third camera, the electronic mark of the image of the object to be detected is (3, 5); each object image to be detected corresponds to a group of defect type codes, and the defect type codes can be used as electronic identifications of the object images to be detected and used for classifying and managing different pieces of cloth, so that the management efficiency is effectively improved.
Optionally, continuously performing image capturing and collecting on the moving cloth by at least one capturing device includes:
continuously performing image shooting and acquisition on the moving cloth through at least two shooting devices; wherein, shooting equipment sets up side by side, and the direction that sets up side by side is perpendicular with the direction of movement of cloth.
In this embodiment, in order to avoid shooting equipment to be difficult to effectual carry out accurate problem of gathering to moving cloth, through setting up at least two shooting equipment of placing side by side, carry out continuous shooting to moving cloth and gather, prevent effectively that the missing from gathering, can be comparatively complete gather the article image that awaits measuring.
Optionally, after storing the determined defect type corresponding to the electronic identifier, the method further includes:
determining the position of the corresponding defect type in the cloth according to each electronic mark in the cloth, and carrying out statistical analysis according to the position and the defect type to determine the abnormality of the weaving equipment of the cloth or the cutting scheme of the subsequent cloth cutting equipment.
Specifically, the image numbers (N) of the cloth collected by each camera can be respectively compiled (Q) for the collected images according to the collection time sequence, and then the recognized defect type numbers (M) are determined according to the recognition result, and various numbers are added into the electronic tag of the cloth. For example, as shown in table 1 below:
TABLE 1
Video camera numbering Acquisition sequence number Defect type
N Q M
In the embodiment of the application, the electronic identification, defect type, defect number and defect rule in the production line are counted on different production lines, and the production line defect statistical information is generated, so that the quality inspection of the weaving production line, the cutting of the subsequent production line for cloth and the classification of different production lines for cloth are conveniently guided; meanwhile, the quality inspection mark is used, equipment is not stopped, and production efficiency is greatly improved.
Fig. 5 is a schematic structural diagram of a device for detecting surface defects of an article according to an embodiment of the present application, where the embodiment is applicable to a case of detecting surface defects of an article, and the device is configured in an electronic device, and the device 500 for detecting surface defects of an article according to any embodiment of the present application specifically includes:
a defect type identifying module 510, configured to input a surface image of the object to be detected into a defect detection model, so as to identify and determine a defect type; the defect detection model is a deep learning model and comprises at least two layers of feature extraction networks;
the image feature extraction module 520 is configured to extract intermediate image features output by at least one layer of feature extraction network during the identification process of the defect detection model;
An image feature matching module 530, configured to match the intermediate image feature with an intermediate image feature in a preset intermediate image feature library;
a defect type correction module 540, configured to correct the defect type determined by the defect detection model according to the matching result; the intermediate image features in the preset intermediate image feature library are features extracted from at least one layer of feature extraction network after the input of the defect detection model based on the object surface image marked with the defect type.
Optionally, the defect detection model is obtained by training based on the object surface images corresponding to at least two defect types, and the marking defect types are the same as or different from the defect types to be trained.
Optionally, the defect detection model includes at least a convolutional neural network model.
Optionally, the method further comprises:
the image feature extraction module 520 is further configured to extract primary image features from the surface image of the object to be detected;
the image feature matching module 530 is further configured to match the primary image feature with a primary image feature in a preset primary image feature library;
accordingly, the defect type correction module 540 is specifically configured to:
And correcting the defect type determined by the defect detection model according to the intermediate matching result and the primary matching result.
Optionally, the primary image features include gray scale distribution features and/or moment features of the surface image of the object to be measured.
Optionally, the defect type correction module 540 is further specifically configured to:
if the matching result is that the defect type corresponding to the matched intermediate image features is consistent with the defect type determined by the identification, the defect type determined by the identification is confirmed to be the final defect type;
if the matching result is that the defect type corresponding to the matched intermediate image feature is inconsistent with the defect type determined by recognition, confirming that the matched defect type is the final defect type or reporting an abnormal recognition phenomenon.
Optionally, the method further comprises:
and the coordinate output module is used for outputting the outline coordinates with the defects.
Optionally, the method further comprises:
the abnormal point identification module is used for identifying abnormal points in the surface image of the original object to be detected based on an abnormal point identification algorithm.
A defect position determining module, configured to determine a candidate defect position according to the identified abnormal point position;
the image shearing module is used for shearing the surface image of the original object to be detected according to the candidate defect position so as to acquire the surface image of the object to be detected; the surface image of the object to be measured comprises at least one abnormal point.
Optionally, the abnormal point identifying module is specifically configured to:
determining an anomaly detection window for each defect type, wherein the anomaly detection window comprises a lateral window and/or a longitudinal window;
traversing the surface image of the original object to be detected by adopting the abnormal detection sliding window, and identifying abnormal points in the abnormal detection sliding window based on an abnormal point identification algorithm.
Optionally, the outlier identification algorithm includes at least one of: an abnormal gray distribution algorithm, an abnormal edge recognition algorithm and an abnormal corner recognition algorithm.
Optionally, the method further comprises:
the image acquisition module is used for continuously carrying out image shooting acquisition on the moving cloth through at least one shooting device so as to acquire images of all articles to be detected;
the electronic identification setting module is used for respectively setting electronic identifications for the images of the articles to be detected according to the shooting equipment and the shooting sequence;
and the defect type storage module is used for storing the determined defect type corresponding to the electronic identifier.
Optionally, the image acquisition module is specifically configured to:
continuously performing image shooting and acquisition on the moving cloth through at least two shooting devices; the shooting equipment is arranged side by side, and the direction of the side by side arrangement is perpendicular to the moving direction of the cloth.
Optionally, the method further comprises:
the statistical analysis module is used for determining the position of the corresponding defect type in the cloth according to each electronic mark in the cloth, and carrying out statistical analysis according to the position and the defect type so as to determine the abnormality of the weaving equipment of the cloth or the cutting scheme of the subsequent cloth cutting equipment.
According to the technical scheme, the accuracy of the image defect types on the surface of the object to be detected is effectively improved; when the newly added defect types are identified, the defect detection model does not need to be updated, and the newly added defect types can be directly matched through a preset intermediate image feature library, so that the detection efficiency is further improved.
According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.
As shown in fig. 6, a block diagram of an electronic device of a method for detecting surface defects of an article according to an embodiment of the application is shown. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.
As shown in fig. 6, the electronic device includes: one or more processors 601, memory 602, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 601 is illustrated in fig. 6.
The memory 602 is a non-transitory computer readable storage medium provided by the present application. The storage stores instructions executable by at least one processor to enable the at least one processor to execute the method for detecting the surface defects of the object. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to execute the method for detecting surface defects of an article provided by the present application.
The memory 602 is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the method for detecting surface defects of an article according to the embodiments of the present application. The processor 601 executes various functional applications of the server and data processing by running non-transitory software programs, instructions and modules stored in the memory 602, i.e., implements the method for detecting surface defects of an article in the above-described method embodiments.
The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for a function; the storage data area may store data created from the use of the electronic device in accordance with the detection of the surface defect of the article, or the like. In addition, the memory 602 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 602 may optionally include memory remotely located relative to processor 601, which may be connected to the electronics for detection of surface defects of the article via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the method for detecting surface defects of an article may further include: an input device 603 and an output device 604. The processor 601, memory 602, input device 603 and output device 604 may be connected by a bus or otherwise, for example in fig. 6.
The input device 603 may receive input numeric or character information as well as key signal inputs related to user settings and function control of the electronic device that generate detection of surface defects of the article, such as input devices for a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, and the like. The output means 604 may include a display device, auxiliary lighting means (e.g., LEDs), tactile feedback means (e.g., vibration motors), and the like. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme provided by the embodiment of the application, the accuracy of the surface image defect types of the object to be detected is effectively improved; when the newly added defect types are identified, the defect detection model does not need to be updated, and the newly added defect types can be directly matched through a preset intermediate image feature library, so that the detection efficiency is further improved.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed embodiments are achieved, and are not limited herein.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.

Claims (15)

1. A method for detecting surface defects of an article, comprising:
inputting the surface image of the object to be detected into a defect detection model to identify and determine the defect type; the defect detection model is a deep learning model and comprises at least two layers of feature extraction networks;
extracting intermediate image features output by at least one layer of feature extraction network in the identification process of the defect detection model;
Matching the intermediate image features with intermediate image features in a preset intermediate image feature library;
if the matching result is that the defect type corresponding to the matched intermediate image features is consistent with the defect type determined by the identification, the defect type determined by the identification is confirmed to be the final defect type;
if the matching result is that the defect type corresponding to the matched intermediate image feature is inconsistent with the defect type determined by recognition, confirming that the matched defect type is the final defect type or reporting an abnormal recognition phenomenon;
the intermediate image features in the preset intermediate image feature library are features extracted from at least one layer of feature extraction network after the input of the defect detection model based on the object surface image marked with the defect type.
2. The method according to claim 1, wherein the defect detection model is obtained by training based on the object surface images corresponding to at least two defect types, and the labeling defect type is the same as or different from the defect type to be trained.
3. The method of claim 2, wherein the defect detection model comprises at least a convolutional neural network model.
4. The method of claim 1, wherein inputting the surface image of the object to be inspected into a defect detection model to identify the determined defect type, further comprises:
extracting primary image features from the surface image of the object to be detected;
matching the primary image features with primary image features in a preset primary image feature library;
correspondingly, correcting the defect type determined by the defect detection model according to the matching result comprises the following steps:
and correcting the defect type determined by the defect detection model according to the intermediate matching result and the primary matching result.
5. The method of claim 4, wherein the primary image features comprise gray scale distribution features and/or moment features of the surface image of the item under test.
6. The method of claim 1, further comprising, after inputting the surface image of the object to be inspected into the defect inspection model:
outputting the profile coordinates with defects.
7. The method of any one of claims 1-6, wherein prior to inputting the surface image of the object to be inspected into the defect inspection model to identify the determined defect type, further comprising:
Identifying abnormal points in the surface image of the original object to be detected based on an abnormal point identification algorithm;
determining candidate defect positions according to the identified abnormal point positions;
shearing the surface image of the original object to be detected according to the candidate defect position to obtain the surface image of the object to be detected; the surface image of the object to be measured comprises at least one abnormal point.
8. The method of claim 7, wherein identifying outliers in the surface image of the original object to be measured based on an outlier identification algorithm comprises:
determining an anomaly detection window for each defect type, wherein the anomaly detection window comprises a lateral window and/or a longitudinal window;
traversing the surface image of the original object to be detected by adopting the abnormal detection sliding window, and identifying abnormal points in the abnormal detection sliding window based on an abnormal point identification algorithm.
9. The method of claim 7, wherein the outlier identification algorithm comprises at least one of: an abnormal gray distribution algorithm, an abnormal edge recognition algorithm and an abnormal corner recognition algorithm.
10. The method according to any one of claims 1 to 6, wherein the article is a cloth, and before inputting the surface image of the article to be tested into the defect detection model to identify the determined defect type, the method comprises:
Continuously performing image shooting and acquisition on the moving cloth through at least one shooting device to acquire images of all articles to be detected;
respectively setting electronic identifiers for the images of the to-be-detected objects according to the shooting equipment and the shooting sequence;
correspondingly, after correcting the defect type determined by the defect detection model according to the matching result, the method further comprises the steps of;
and storing the determined defect type corresponding to the electronic identifier.
11. The method of claim 10, wherein continuously image capturing the moving web with at least one capturing device comprises:
continuously performing image shooting and acquisition on the moving cloth through at least two shooting devices; the shooting equipment is arranged side by side, and the direction of the side by side arrangement is perpendicular to the moving direction of the cloth.
12. The method of claim 10, wherein after storing the determined defect type corresponding to the electronic identification, further comprising:
determining the position of the corresponding defect type in the cloth according to each electronic mark in the cloth, and carrying out statistical analysis according to the position and the defect type to determine the abnormality of the cloth weaving equipment or the cutting scheme of the subsequent cloth cutting equipment.
13. A device for detecting surface defects of an article, comprising:
the defect type identification module is used for inputting the surface image of the object to be detected into the defect detection model so as to identify and determine the defect type; the defect detection model is a deep learning model and comprises at least two layers of feature extraction networks;
the image feature extraction module is used for extracting intermediate image features output by at least one layer of feature extraction network in the identification process of the defect detection model;
the image feature matching module is used for matching the intermediate image features with intermediate image features in a preset intermediate image feature library;
the defect type correction module is specifically configured to confirm that the identified defect type is a final defect type if the matching result is that the defect type corresponding to the matched intermediate image feature is consistent with the identified defect type; if the matching result is that the defect type corresponding to the matched intermediate image feature is inconsistent with the defect type determined by recognition, confirming that the matched defect type is the final defect type or reporting an abnormal recognition phenomenon; the intermediate image features in the preset intermediate image feature library are features extracted from at least one layer of feature extraction network after the input of the defect detection model based on the object surface image marked with the defect type.
14. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.
15. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-12.
CN202010538153.7A 2020-06-12 2020-06-12 Method, device, equipment and medium for detecting surface defects of article Active CN111738322B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010538153.7A CN111738322B (en) 2020-06-12 2020-06-12 Method, device, equipment and medium for detecting surface defects of article

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010538153.7A CN111738322B (en) 2020-06-12 2020-06-12 Method, device, equipment and medium for detecting surface defects of article

Publications (2)

Publication Number Publication Date
CN111738322A CN111738322A (en) 2020-10-02
CN111738322B true CN111738322B (en) 2023-09-01

Family

ID=72649050

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010538153.7A Active CN111738322B (en) 2020-06-12 2020-06-12 Method, device, equipment and medium for detecting surface defects of article

Country Status (1)

Country Link
CN (1) CN111738322B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112532838B (en) * 2020-11-25 2023-03-07 努比亚技术有限公司 Image processing method, mobile terminal and computer storage medium
CN113222967A (en) * 2021-05-28 2021-08-06 长江存储科技有限责任公司 Wafer detection method and system
CN113744268B (en) * 2021-11-04 2022-04-22 深圳市城市交通规划设计研究中心股份有限公司 Crack detection method, electronic device and readable storage medium
CN114708484B (en) * 2022-03-14 2023-04-07 中铁电气化局集团有限公司 Pattern analysis method suitable for identifying defects
CN114663403B (en) * 2022-03-25 2022-11-18 北京城建设计发展集团股份有限公司 Prefabricated part assembling surface local defect identification method based on dense scanning data
CN114897810A (en) * 2022-05-06 2022-08-12 阿里巴巴达摩院(杭州)科技有限公司 Image detection method, image detection device, storage medium and electronic equipment
CN115187927B (en) * 2022-07-27 2024-03-26 上海志远生态园林工程有限公司 Remote monitoring and management method and system for sightseeing seat
CN116258714B (en) * 2023-05-12 2023-08-01 苏州苏映视图像软件科技有限公司 Defect identification method and device, electronic equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109342456A (en) * 2018-09-14 2019-02-15 广东工业大学 A kind of welding point defect detection method, device, equipment and readable storage medium storing program for executing
CN109598721A (en) * 2018-12-10 2019-04-09 广州市易鸿智能装备有限公司 Defect inspection method, device, detection device and the storage medium of battery pole piece
CN110726724A (en) * 2019-10-22 2020-01-24 北京百度网讯科技有限公司 Defect detection method, system and device
WO2020092509A1 (en) * 2018-10-30 2020-05-07 University of North Texas System Reconfigurable fabric inspection system
CN111160451A (en) * 2019-12-27 2020-05-15 中山德著智能科技有限公司 Flexible material detection method and storage medium thereof

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109342456A (en) * 2018-09-14 2019-02-15 广东工业大学 A kind of welding point defect detection method, device, equipment and readable storage medium storing program for executing
WO2020092509A1 (en) * 2018-10-30 2020-05-07 University of North Texas System Reconfigurable fabric inspection system
CN109598721A (en) * 2018-12-10 2019-04-09 广州市易鸿智能装备有限公司 Defect inspection method, device, detection device and the storage medium of battery pole piece
CN110726724A (en) * 2019-10-22 2020-01-24 北京百度网讯科技有限公司 Defect detection method, system and device
CN111160451A (en) * 2019-12-27 2020-05-15 中山德著智能科技有限公司 Flexible material detection method and storage medium thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
姚明海 等.基于深度主动学习的磁片表面缺陷检测.《计算机测量与控制》.2018,第26卷(第9期),第29-33页. *

Also Published As

Publication number Publication date
CN111738322A (en) 2020-10-02

Similar Documents

Publication Publication Date Title
CN111738322B (en) Method, device, equipment and medium for detecting surface defects of article
CN111784663B (en) Method and device for detecting parts, electronic equipment and storage medium
CN111768381A (en) Part defect detection method and device and electronic equipment
Deng et al. Building an automatic defect verification system using deep neural network for pcb defect classification
CN108346144B (en) Automatic bridge crack monitoring and identifying method based on computer vision
US11699283B2 (en) System and method for finding and classifying lines in an image with a vision system
CN111833303A (en) Product detection method and device, electronic equipment and storage medium
CN112507949A (en) Target tracking method and device, road side equipment and cloud control platform
KR20190075707A (en) Method for sorting products using deep learning
CN111860319A (en) Method for determining lane line, method, device and equipment for evaluating positioning accuracy
CN108318773B (en) Transmission conductor strand breakage detection method and system
CN110533654A (en) The method for detecting abnormality and device of components
CN110555838A (en) Image-based part fault detection method and device
CN113091757B (en) Map generation method and device
CN112288699B (en) Method, device, equipment and medium for evaluating relative definition of image
CN111179250A (en) Industrial product defect detection system based on multitask learning
US20120170835A1 (en) Determining the Uniqueness of a Model for Machine Vision
CN111062934A (en) Real-time detection method for fabric image defects
CN111783639A (en) Image detection method and device, electronic equipment and readable storage medium
CN111967490A (en) Model training method for map detection and map detection method
CN108460344A (en) Dynamic area intelligent identifying system in screen and intelligent identification Method
CN111444819B (en) Cut frame determining method, network training method, device, equipment and storage medium
CN110458809B (en) Yarn evenness detection method based on sub-pixel edge detection
KR20240058827A (en) System and method for finding and classifying lines in an image with a vision system
CN111696095B (en) Method and device for detecting surface defects of object

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant