CN114529529B - Lace cloth surface defect detection method and device based on image simulation enhancement - Google Patents
Lace cloth surface defect detection method and device based on image simulation enhancement Download PDFInfo
- Publication number
- CN114529529B CN114529529B CN202210156089.5A CN202210156089A CN114529529B CN 114529529 B CN114529529 B CN 114529529B CN 202210156089 A CN202210156089 A CN 202210156089A CN 114529529 B CN114529529 B CN 114529529B
- Authority
- CN
- China
- Prior art keywords
- image
- defect
- network
- cloth
- lace cloth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 175
- 239000004744 fabric Substances 0.000 title claims abstract description 140
- 238000001514 detection method Methods 0.000 title claims abstract description 85
- 238000004088 simulation Methods 0.000 title claims abstract description 47
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 54
- 238000000034 method Methods 0.000 claims abstract description 39
- 230000002950 deficient Effects 0.000 claims abstract description 29
- 238000004519 manufacturing process Methods 0.000 claims abstract description 19
- 238000007689 inspection Methods 0.000 claims abstract description 16
- 238000003709 image segmentation Methods 0.000 claims description 22
- 230000006870 function Effects 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 11
- 230000005540 biological transmission Effects 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 8
- 238000011282 treatment Methods 0.000 claims description 4
- 238000011276 addition treatment Methods 0.000 claims description 2
- 238000009776 industrial production Methods 0.000 abstract description 6
- 238000012805 post-processing Methods 0.000 abstract description 4
- 230000008901 benefit Effects 0.000 description 6
- 230000011218 segmentation Effects 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000003416 augmentation Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000004873 anchoring Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30124—Fabrics; Textile; Paper
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The application relates to the technical field of product surface defect detection, in particular to an image simulation enhancement-based unsupervised lace cloth surface defect detection method and device, wherein the method comprises the following steps: collecting continuous non-defective sample pictures on a lace cloth production or quality inspection assembly line by using an industrial camera, wherein collected picture data is used as a training data set; performing simulation defect addition and image enhancement on the acquired data set; training a convolutional neural network using the simulated defect image; and collecting cloth sample graphs on a lace cloth production or quality inspection assembly line on line, detecting by using a trained convolutional neural network model, and performing post-processing on output to obtain a detection result of the region where the defect is located. Therefore, the defect that the convolutional neural network depends on a large number of marked defect patterns for training is avoided, and the robustness is good. Therefore, the problems of insufficient number of defective samples, long collection period and the like in an industrial production environment are solved.
Description
Technical Field
The application relates to the technical field of data-driven product surface defect detection, in particular to an unsupervised lace cloth surface defect detection method and device based on image simulation enhancement.
Background
The detection of surface defects of industrial products is an important part of enterprise production quality control. At present, the defect detection of the toilet surface of an enterprise is often carried out manually, which can cause the problems of improvement of operation and management cost, unstable detection quality, occurrence of missing detection phenomena, visual damage of workers and the like.
In the related art, the method for detecting the surface defects by using the traditional computer vision technology mainly comprises a structure-based method, a frequency domain-based method, a statistics-based method, a model-based method and the like, and the traditional methods often have the defects of dependence on super parameter setting, difficulty in adapting to different textures, poor detection performance for complex texture fabrics and the like. The lace fabric has complex texture and long texture repetition period, and the traditional method is difficult to obtain good detection effect.
In recent years, CNN (Convolutional Neural Network ) has demonstrated significant advantages in solving image-related problems. Compared with the detection by the traditional method, the method has the advantages that the convolutional neural network is utilized for supervised learning, the dependence of the algorithm on super parameters is reduced, and the method has good adaptability to the automatic learning of different lace texture fabrics through network parameters. However, in practical applications, supervised learning requires a large amount of defect pattern data with good labels. In the actual production process, the collection and labeling of the defect samples is a long process, which greatly limits the application of the convolutional neural network method in the actual production, and needs to be solved.
Disclosure of Invention
The application provides an image simulation enhancement-based unsupervised lace cloth surface defect detection method and device, which are used for solving the problems of insufficient number of defect samples, long collection period and the like in an industrial production environment.
An embodiment of a first aspect of the present application provides an image simulation enhancement-based method for detecting surface defects of an unsupervised lace cloth, including the following steps: collecting flawless lace cloth images to construct a cloth sample data set; performing defect enhancement and simulated image enhancement on the lace cloth images in the cloth sample data set to obtain a defect image training data set; performing convolutional neural network training through a defect image training data set to obtain a defect detection network model for detecting the surface defects of the lace cloth; and acquiring lace cloth images on line, and detecting the online lace cloth images through a defect detection network model to obtain defect detection results of the online lace cloth images.
Optionally, in one embodiment of the present application, the cloth sample dataset is a flawless lace cloth image on a production or quality inspection line, and the cloth sample dataset does not contain labeling information.
Optionally, in an embodiment of the present application, the performing defect augmentation on the lace cloth image in the cloth sample data set and the simulating image augmentation to obtain a defect image training data set further includes: selecting a deformed small image of the flawless lace cloth image from the large image of the flawless lace cloth image through transmission transformation, and performing HSV space enhancement treatment, gaussian noise addition treatment and Gaussian filtering treatment on the small image to obtain a flawless lace cloth image for training; and adding artificial simulation defects to the flawless lace cloth image to obtain a flawed training data set of the flawed image.
Optionally, in one embodiment of the present application, the defect detection network model includes an image reconstruction network, an image segmentation network and an image registration network based on a convolutional neural network, in which the defect detection network model combines the image registration network with the image reconstruction network and the image segmentation network, and performs defect detection by using texture information of an image; wherein the image segmentation network extracts image features using the convolutional neural network, restores image resolution layer by layer using upsampling, maintains image detail information using long connections, and uses BCE loss functions as parameter updates for the training process; the image reconstruction network uses pixel errors or texture loss as a loss function; the image registration network extracts image features of the image blocks and the reference image by using a convolutional neural network, calculates correlation coefficients of feature images, and obtains a registration relationship by decoding the convolutional neural network.
Optionally, in one embodiment of the present application, the defect detection network model uses the image reconstruction network to recover a non-defective sample map, and uses residuals between the recovered non-defective sample map and the input image to obtain a region where a defect may occur through a preset threshold; obtaining probability that pixel points are possibly located in a defect area by using the image segmentation network, and segmenting the defect area through a preset threshold value; and obtaining the relative position of the image to be detected in the reference image by using an image registration network, and detecting the defect by using the relative position information.
An embodiment of a second aspect of the present application provides an unsupervised lace cloth surface defect detection apparatus based on image simulation enhancement, including: the data acquisition module is used for acquiring the flaccid cloth images without defects to construct a cloth sample data set; the data enhancement module is used for carrying out defect enhancement and simulation image enhancement on the lace cloth images in the cloth sample data set to obtain a defect image training data set; the network training module is used for performing convolutional neural network training through the defect image training data set to obtain a defect detection network model for performing lace cloth surface defect detection; and the defect detection module is used for acquiring the lace cloth image on line, and detecting the online lace cloth image through the defect detection network model to obtain a defect detection result of the online lace cloth image.
Optionally, in one embodiment of the present application, the cloth sample dataset is a flawless lace cloth image on a production or quality inspection line, and the cloth sample dataset does not contain labeling information.
Optionally, in one embodiment of the present application, the data enhancing module is specifically configured to select, by transmission transformation, a deformed small image of the flawless lace cloth image from large images of flawless lace cloth images, and perform HSV spatial enhancement processing, gaussian noise addition processing, and gaussian filtering processing on the small image to obtain a flawless lace cloth image for training; and adding artificial simulation defects to the flawless lace cloth image to obtain a flawed training data set of the flawed image.
Optionally, in one embodiment of the present application, the defect detection network model includes an image reconstruction network, an image segmentation network and an image registration network based on a convolutional neural network, in which the defect detection network model combines the image registration network with the image reconstruction network and the image segmentation network, and performs defect detection by using texture information of an image; wherein the image segmentation network extracts image features using the convolutional neural network, restores image resolution layer by layer using upsampling, maintains image detail information using long connections, and uses BCE loss functions as parameter updates for the training process; the image reconstruction network uses pixel errors or texture loss as a loss function; the image registration network extracts image features of the image blocks and the reference image by using a convolutional neural network, calculates correlation coefficients of feature images, and obtains a registration relationship by decoding the convolutional neural network.
Optionally, in one embodiment of the present application, the defect detection network model uses the image reconstruction network to recover a non-defective sample map, and uses residuals between the recovered non-defective sample map and the input image to obtain a region where a defect may occur through a preset threshold; obtaining probability that pixel points are possibly located in a defect area by using the image segmentation network, and segmenting the defect area through a preset threshold value; and obtaining the relative position of the image to be detected in the reference image by using an image registration network, and detecting the defect by using the relative position information.
Therefore, the application has at least the following beneficial effects:
collecting continuous non-defective sample pictures on a lace cloth production or quality inspection assembly line by using an industrial camera, wherein the lace cloth is ensured not to have surface defects in the process, and the collected picture data is used as a training data set; performing simulation defect addition and image enhancement on the acquired defect-free sample data set, wherein the simulation defect addition comprises defects such as simulation holes, greasy dirt, mistakes and the like, and the image enhancement means comprises transmission transformation, HSV space transformation, gaussian noise, gaussian filtering and the like; training a convolutional neural network by using the simulated defect image, wherein the convolutional neural network adopts a segmentation model; and collecting cloth sample graphs on a lace cloth production or quality inspection assembly line on line, detecting by using a trained convolutional neural network model, and performing post-processing on output to obtain a detection result of the region where the defect is located. The method can effectively avoid the problems of insufficient number of defective samples and long collection period in an industrial production environment, can adapt to image quality changes caused by different shooting angles and environmental brightness, and can quickly adapt to different pattern textures through automatic learning of a neural network. Therefore, the problems of insufficient number of defective samples, long collection period and the like in an industrial production environment are solved.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a flowchart of an image simulation enhancement-based method for detecting surface defects of an unsupervised lace cloth according to an embodiment of the present application;
FIG. 2 is a schematic diagram of the execution logic of an image simulation enhancement-based unsupervised lace cloth surface defect detection algorithm according to one embodiment of the present application;
fig. 3 is an exemplary diagram of an unsupervised lace cloth surface defect detection apparatus based on image simulation enhancement according to an embodiment of the present application.
Reference numerals illustrate: the system comprises a data acquisition module-100, a data enhancement module-200, a network training module-300 and a defect detection module-400.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present application and are not to be construed as limiting the present application.
An image simulation enhancement-based unsupervised lace cloth surface defect detection method, device, electronic equipment and storage medium are described below with reference to the accompanying drawings. Aiming at the problems in the background art, the application provides an image simulation enhancement-based non-supervision lace cloth surface defect detection method, in which the image simulation enhancement-based non-supervision lace cloth surface defect detection device can realize automatic detection of lace cloth surface defects and save enterprise operation and management cost; the algorithm can train only by using a defect-free sample, so that the defect that a convolutional neural network depends on a large number of marked defect patterns for training can be avoided; the trained neural network has good robustness to various interference possibly occurring in the data acquisition process by performing image enhancement means such as transmission transformation, HSV space enhancement, gaussian blur, gaussian noise addition and the like on the data. Therefore, the problems of insufficient number of defective samples, long collection period and the like in an industrial production environment are solved.
Specifically, fig. 1 is a flowchart of an image simulation enhancement-based method for detecting surface defects of an unsupervised lace cloth according to an embodiment of the present application.
As shown in FIG. 1, the method for detecting the surface defects of the non-supervision lace cloth based on image simulation enhancement comprises the following steps:
in step S101, a non-defective lace cloth image is acquired to construct a cloth sample dataset.
The industrial camera is used for collecting a cloth sample data set on a lace cloth production or quality inspection assembly line, and in the collecting process, the lace cloth is ensured to have no surface defects, and the collected non-defective cloth sample data set is used as a training data set.
Optionally, in one embodiment of the present application, the training dataset is a defect-free sample picture on a production or quality inspection line, and the training dataset itself does not contain defective samples and labeling information.
It will be appreciated that as shown in fig. 2, embodiments of the present application first take a photograph by an industrial camera deployed on a lace cloth production or quality inspection line, thereby facilitating the acquisition of a defect-free sample dataset and using it as a training dataset.
In step S102, defect augmentation and simulated image augmentation are performed on the lace cloth image in the cloth sample data set to obtain a defect image training data set.
Specifically, simulation image enhancement is carried out on the acquired non-defective cloth sample data set, wherein the simulation image enhancement comprises image enhancement and simulation defect addition, and a simulation defect image is obtained.
Optionally, in an embodiment of the present application, performing simulated image enhancement on the acquired non-defective cloth sample dataset, including image enhancement and simulated defect addition, to obtain a simulated defect image, further including: selecting a deformed non-defective sample small image from the non-defective sample large image through transmission transformation, and performing HSV space enhancement processing, gaussian noise addition processing and Gaussian filtering processing on the small image to obtain a non-defective sample image for training; and adding artificial simulation defects to the defect-free sample graph to obtain a defective sample graph.
In the process of collecting the sample, as the view angle of the camera is not necessarily fixed, the overall brightness of the environment is changed, the problems of noise and the like to a certain extent and the influence of definition change and the like exist in the image collecting process, and a certain image enhancement means is needed to enhance the robustness of the follow-up neural network to the input data. Specifically, the image enhancement means used include transmission transform, HSV space transform enhancement, gaussian noise addition, gaussian filtering, and the like.
Meanwhile, the possible defects of the lace cloth sample have a certain priori knowledge. For example, a hole is represented as a low-pixel area, and oil stains do not damage the texture of the cloth itself, so that the color of the cloth changes. Specifically, the hole is simulated using a low pixel value region, the oil stain is simulated using a color patch having a certain transparency, and the staggering moves a certain portion of the texture to form an uncoordinated region.
In step S103, convolutional neural network training is performed through the defect image training data set, so as to obtain a defect detection network model for detecting the surface defects of the lace cloth.
Specifically, training a convolutional neural network by using a simulated defect image to obtain a trained convolutional neural network model, and determining a surface defect detection network model
Optionally, in an embodiment of the present application, the convolutional neural network is an image reconstruction network, an image segmentation network and an image registration network based on the convolutional neural network, and in the defect detection network model, the image registration network is combined with the image reconstruction network and the image segmentation network, and defect detection is performed by using texture information of the image; wherein the segmentation network extracts image features using a convolutional neural network, restores image resolution layer by layer using upsampling, maintains image detail information using long connections, and uses BCE loss functions as parameter updates for the training process; the image reconstruction network uses pixel errors or texture loss as a loss function; the image registration network uses a convolutional neural network to extract image features of the image blocks and the reference image, calculates correlation coefficients of feature images, and obtains a registration relationship through decoding of the convolutional neural network.
Optionally, in an embodiment of the present application, training the convolutional neural network using the simulated defect image to obtain a trained convolutional neural network model includes: collecting continuous non-defective sample pictures on a lace cloth production or quality inspection assembly line; after the simulation data are enhanced, the simulation enhanced data are used for training the convolutional neural network; obtaining supervision information according to the defect-free sample simulation enhancement data set, and building a training model based on a convolutional neural network; training the training model according to the defect-free sample simulation enhancement data set to obtain a surface defect detection network model.
It should be noted that in the embodiments of the present application, the convolutional neural network described above includes a reconstruction model, a segmentation model, and a registration model. The reconstruction model outputs a restored defect-free image, and the segmentation model outputs a probability map that pixel points are possibly located in a defect area.
Specifically, residual errors between the defect-free image and the image to be detected recovered by the reconstruction model are utilized, a region with larger deviation is obtained by setting a threshold value, and the region is considered to be a region with larger difference between the image to be detected and the defect-free image and is used as a defect detection result. Similarly, by setting a threshold value (the threshold value is generally selected to be 0.5) on the probability map output by the segmentation model, a region with a high probability of defect in the image to be detected is obtained and is used as a defect detection result. For the target detection model, the output defect position anchoring frame is a square calibration of the area where the defect position is located, and can be directly output as a detection result.
In step S104, the lace cloth image is collected online, and the online lace cloth image is detected by the defect detection network model, so as to obtain a defect detection result of the online lace cloth image.
Specifically, the number of cloth samples on a lace cloth production or quality inspection assembly line is collected on line, the surface defect detection network model is used for detection according to the number of cloth samples, and the output is subjected to post-processing to obtain a detection result of the region where the defect is located.
According to the method for detecting the surface defects of the non-supervision lace cloth based on image simulation enhancement, which is provided by the embodiment of the application, continuous non-defect sample pictures on a lace cloth production or quality inspection assembly line are collected by using an industrial camera, the lace cloth is ensured not to have surface defects in the process, and the collected picture data is used as a training data set; performing simulation defect addition and image enhancement on the acquired defect-free sample data set, wherein the simulation defect addition comprises defects such as simulation holes, greasy dirt, mistakes and the like, and the image enhancement means comprises transmission transformation, HSV space transformation, gaussian noise, gaussian filtering and the like; training a convolutional neural network by using the simulated defect image, wherein the convolutional neural network adopts a segmentation model; and collecting cloth sample graphs on a lace cloth production or quality inspection assembly line on line, detecting by using a trained convolutional neural network model, and performing post-processing on output to obtain a detection result of the region where the defect is located. The method and the device can effectively avoid the problems of insufficient number of defective samples and long collection period in an industrial production environment, can adapt to image quality changes caused by different shooting angles and environmental brightness, and can quickly adapt to different pattern textures through neural network automatic learning.
Next, an unsupervised lace cloth surface defect detection device based on image simulation enhancement according to an embodiment of the present application will be described with reference to the accompanying drawings.
Fig. 3 is a schematic block diagram of an image simulation enhancement-based unsupervised lace cloth surface defect detection device in accordance with an embodiment of the present application.
As shown in fig. 3, the image simulation enhancement-based unsupervised lace cloth surface defect detection apparatus 10 includes: a data acquisition module 100, a data enhancement module 200, a network training module 300, and a defect detection module 400.
The data acquisition module 100 is used for acquiring flaccid cloth images without defects to construct a cloth sample data set. The data enhancement module 200 is used for carrying out defect enhancement on the lace cloth image in the cloth sample data set and enhancing the simulation image to obtain a defect image training data set. The network training module 300 is configured to perform convolutional neural network training through the defect image training data set, so as to obtain a defect detection network model for performing surface defect detection on the lace cloth. The defect detection module 400 is configured to collect the lace cloth image online, and detect the online lace cloth image through the defect detection network model, so as to obtain a defect detection result of the online lace cloth image.
Optionally, in one embodiment of the present application, the cloth sample dataset is a non-defective lace cloth image on a production or quality inspection line, and the cloth sample dataset does not contain labeling information.
Optionally, in an embodiment of the present application, the data enhancing module is specifically configured to select, by transmission transformation, a deformed small image of the flawless lace cloth image from the large images of flawless lace cloth images, and perform HSV space enhancement processing, gaussian noise addition processing, and gaussian filtering processing on the small image to obtain a flawless lace cloth image for training; and adding artificial simulation defects to the flawless lace cloth images to obtain a flawed defect image training data set.
Optionally, in an embodiment of the present application, the defect detection network model includes an image reconstruction network, an image segmentation network and an image registration network based on a convolutional neural network, in the defect detection network model, the image registration network is combined with the image reconstruction network and the image segmentation network, and defect detection is performed by using texture information of the image; wherein the image segmentation network extracts image features using a convolutional neural network, restores image resolution layer by layer using upsampling, maintains image detail information using long connections, and uses BCE loss functions as parameter updates for the training process; the image reconstruction network uses pixel errors or texture loss as a loss function; the image registration network uses a convolutional neural network to extract image features of the image blocks and the reference image, calculates correlation coefficients of feature images, and obtains a registration relationship through decoding of the convolutional neural network.
Optionally, in one embodiment of the present application, the defect detection network model includes recovering a non-defective sample map using an image reconstruction network, obtaining a region where a defect may occur through a preset threshold value using a residual error between the recovered non-defective sample map and the input image; obtaining probability that a pixel point is possibly positioned in a defect area by using an image segmentation network, and segmenting the defect area through a preset threshold value; and obtaining the relative position of the image to be detected in the reference image by using the image registration network, and detecting the defect by using the relative position information.
It should be noted that the foregoing explanation of the embodiment of the method for detecting surface defects of an unsupervised lace cloth based on image emulation enhancement is also applicable to the device for detecting surface defects of an unsupervised lace cloth based on image emulation enhancement of this embodiment, and will not be repeated here.
According to the device for detecting the surface defects of the non-supervision lace cloth based on the image simulation enhancement, a large amount of defective image data do not need to be collected and marked, and the convolutional neural network is trained in a simulation data enhancement mode. The method has the advantages of being capable of exerting the advantage of strong self-learning adaptability of the convolutional neural network, and effectively avoiding dependence on the labeling data.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless explicitly defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more N executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Claims (4)
1. The method for detecting the surface defects of the non-supervision lace cloth based on the image simulation enhancement is characterized by comprising the following steps of:
collecting flawless lace cloth images to construct a cloth sample data set;
performing defect enhancement and simulated image enhancement on the lace cloth images in the cloth sample data set to obtain a defect image training data set;
performing convolutional neural network training through the defect image training data set to obtain a defect detection network model for detecting the surface defects of the lace cloth;
collecting lace cloth images on line, and detecting the online lace cloth images through the defect detection network model to obtain defect detection results of the online lace cloth images;
performing defect addition and simulated image enhancement on the lace cloth images in the cloth sample data set to obtain a defect image training data set, and further comprising:
selecting a deformed small image of the flawless lace cloth image from the large image of the flawless lace cloth image through transmission transformation, and performing HSV space enhancement treatment, gaussian noise addition treatment and Gaussian filtering treatment on the small image to obtain a flawless lace cloth image for training;
adding artificial simulation defects to the flawless lace cloth image to obtain a flawed training data set of the flawed image;
the defect detection network model comprises an image reconstruction network, an image segmentation network and an image registration network based on a convolutional neural network, wherein in the defect detection network model, the image registration network is combined with the image reconstruction network and the image segmentation network, and defect detection is carried out by utilizing texture information of an image; wherein the image segmentation network extracts image features using the convolutional neural network, restores image resolution layer by layer using upsampling, maintains image detail information using long connections, and uses BCE loss functions as parameter updates for the training process; the image reconstruction network uses pixel errors or texture loss as a loss function; the image registration network extracts image features of the image blocks and the reference image by using a convolutional neural network, calculates correlation coefficients of feature images, and obtains a registration relationship by decoding the convolutional neural network;
the defect detection network model uses the image reconstruction network to recover a non-defective sample image, and obtains a region where defects possibly occur through a preset threshold value by utilizing residual errors between the recovered non-defective sample image and an input image; obtaining probability that pixel points are possibly located in a defect area by using the image segmentation network, and segmenting the defect area through a preset threshold value; and obtaining the relative position of the image to be detected in the reference image by using the image registration network, and detecting the defect by using the relative position information.
2. The method of claim 1, wherein the cloth sample dataset is a flawless lace cloth image on a production or quality inspection line, and the cloth sample dataset does not contain labeling information.
3. An image simulation enhancement-based unsupervised lace cloth surface defect detection device is characterized by comprising:
the data acquisition module is used for acquiring the flaccid cloth images without defects to construct a cloth sample data set;
the data enhancement module is used for carrying out defect enhancement and simulation image enhancement on the lace cloth images in the cloth sample data set to obtain a defect image training data set;
the network training module is used for performing convolutional neural network training through the defect image training data set to obtain a defect detection network model for performing lace cloth surface defect detection;
the defect detection module is used for acquiring lace cloth images on line, detecting the online lace cloth images through the defect detection network model, and obtaining defect detection results of the online lace cloth images;
the data enhancement module is specifically used for selecting a deformed small image of the flawless lace cloth image from the large image of the flawless lace cloth image through transmission transformation, and performing HSV space enhancement processing, gaussian noise addition processing and Gaussian filtering processing on the small image to obtain a flawless lace cloth image for training; adding artificial simulation defects to the flawless lace cloth image to obtain a flawed training data set of the flawed image;
the defect detection network model comprises an image reconstruction network, an image segmentation network and an image registration network based on a convolutional neural network, wherein in the defect detection network model, the image registration network is combined with the image reconstruction network and the image segmentation network, and defect detection is carried out by utilizing texture information of an image; wherein the image segmentation network extracts image features using the convolutional neural network, restores image resolution layer by layer using upsampling, maintains image detail information using long connections, and uses BCE loss functions as parameter updates for the training process; the image reconstruction network uses pixel errors or texture loss as a loss function; the image registration network extracts image features of the image blocks and the reference image by using a convolutional neural network, calculates correlation coefficients of feature images, and obtains a registration relationship by decoding the convolutional neural network;
the defect detection network model uses the image reconstruction network to recover a non-defective sample image, and obtains a region where defects possibly occur through a preset threshold value by utilizing residual errors between the recovered non-defective sample image and an input image; obtaining probability that pixel points are possibly located in a defect area by using the image segmentation network, and segmenting the defect area through a preset threshold value; and obtaining the relative position of the image to be detected in the reference image by using the image registration network, and detecting the defect by using the relative position information.
4. A device according to claim 3, wherein the cloth sample dataset is a flawless lace cloth image on a production or quality inspection line, and the cloth sample dataset does not contain labeling information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210156089.5A CN114529529B (en) | 2022-02-21 | 2022-02-21 | Lace cloth surface defect detection method and device based on image simulation enhancement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210156089.5A CN114529529B (en) | 2022-02-21 | 2022-02-21 | Lace cloth surface defect detection method and device based on image simulation enhancement |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114529529A CN114529529A (en) | 2022-05-24 |
| CN114529529B true CN114529529B (en) | 2024-04-09 |
Family
ID=81624546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210156089.5A Active CN114529529B (en) | 2022-02-21 | 2022-02-21 | Lace cloth surface defect detection method and device based on image simulation enhancement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114529529B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019233166A1 (en) * | 2018-06-04 | 2019-12-12 | 杭州海康威视数字技术股份有限公司 | Surface defect detection method and apparatus, and electronic device |
| CN111553929A (en) * | 2020-05-12 | 2020-08-18 | 重庆邮电大学 | Mobile phone screen defect segmentation method, device and equipment based on converged network |
| WO2021232613A1 (en) * | 2020-05-22 | 2021-11-25 | 五邑大学 | Liquor bottle surface defect inspection method, electronic device, and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7170605B2 (en) * | 2019-09-02 | 2022-11-14 | 株式会社東芝 | Defect inspection device, defect inspection method, and program |
| EP3916635B1 (en) * | 2020-05-26 | 2023-05-10 | Fujitsu Limited | Defect detection method and apparatus |
-
2022
- 2022-02-21 CN CN202210156089.5A patent/CN114529529B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019233166A1 (en) * | 2018-06-04 | 2019-12-12 | 杭州海康威视数字技术股份有限公司 | Surface defect detection method and apparatus, and electronic device |
| CN111553929A (en) * | 2020-05-12 | 2020-08-18 | 重庆邮电大学 | Mobile phone screen defect segmentation method, device and equipment based on converged network |
| WO2021232613A1 (en) * | 2020-05-22 | 2021-11-25 | 五邑大学 | Liquor bottle surface defect inspection method, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114529529A (en) | 2022-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108961217B (en) | Surface defect detection method based on regular training | |
| WO2019104767A1 (en) | Fabric defect detection method based on deep convolutional neural network and visual saliency | |
| CN110648349A (en) | Weld defect segmentation method based on background subtraction and connected region algorithm | |
| TWI743837B (en) | Training data increment method, electronic apparatus and computer-readable medium | |
| KR20220022091A (en) | Automatic optimization of an examination recipe | |
| JP7169393B2 (en) | Generating training sets that can be used to inspect semiconductor specimens | |
| CN113284154B (en) | Steel coil end face image segmentation method and device and electronic equipment | |
| CN112801962B (en) | Semi-supervised industrial product flaw detection method and system based on positive sample learning | |
| CN114565552A (en) | Method and device for detecting surface defects of optical element and electronic equipment | |
| CN113706464A (en) | Printed matter appearance quality detection method and system | |
| CN112200790B (en) | Cloth defect detection method, device and medium | |
| Peng et al. | Non-uniform illumination image enhancement for surface damage detection of wind turbine blades | |
| JP7170605B2 (en) | Defect inspection device, defect inspection method, and program | |
| CN113962951B (en) | Training method and device for detecting segmentation model, and target detection method and device | |
| CN111784645A (en) | Crack detection method for filling pipeline | |
| CN115578326A (en) | Road disease identification method, system, equipment and storage medium | |
| CN114529529B (en) | Lace cloth surface defect detection method and device based on image simulation enhancement | |
| CN113516652A (en) | Battery surface defect and adhesive detection method, device, medium and electronic equipment | |
| CN111028250A (en) | Real-time intelligent cloth inspecting method and system | |
| CN117233192A (en) | Defect inspection apparatus | |
| CN108961384B (en) | Three-dimensional image reconstruction method | |
| Singh et al. | Segmentation technique for the detection of Micro cracks in solar cell using support vector machine | |
| KR20230036650A (en) | Defect detection method and system based on image patch | |
| CN114494931A (en) | Intelligent classification processing method and system for video image faults | |
| Huang et al. | Research on pipe crack detection based on image processing algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |