CN110619619A - Defect detection method and device and electronic equipment - Google Patents
Defect detection method and device and electronic equipment Download PDFInfo
- Publication number
- CN110619619A CN110619619A CN201810563652.4A CN201810563652A CN110619619A CN 110619619 A CN110619619 A CN 110619619A CN 201810563652 A CN201810563652 A CN 201810563652A CN 110619619 A CN110619619 A CN 110619619A
- Authority
- CN
- China
- Prior art keywords
- image
- detected
- deep learning
- value
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 145
- 238000001514 detection method Methods 0.000 title claims abstract description 61
- 238000013135 deep learning Methods 0.000 claims abstract description 90
- 238000010586 diagram Methods 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims description 69
- 238000000034 method Methods 0.000 claims description 43
- 238000012545 processing Methods 0.000 claims description 32
- 230000002950 deficient Effects 0.000 claims description 26
- 230000011218 segmentation Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 description 17
- 239000004744 fabric Substances 0.000 description 13
- 238000010801 machine learning Methods 0.000 description 7
- 235000019557 luminance Nutrition 0.000 description 6
- 238000012805 post-processing Methods 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/20—Linear translation of whole images or parts thereof, e.g. panning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention provides a defect detection method, a defect detection device and electronic equipment, wherein the defect detection method comprises the following steps: acquiring an image to be detected; inputting an image to be detected into a pre-trained deep learning network model to obtain a multi-value diagram corresponding to the image to be detected; and adjusting pixels in the multi-value image according to the image to be detected and the multi-value image to obtain a defect area in the image to be detected. By the scheme, the accuracy of defect detection can be improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a defect detection method and apparatus, and an electronic device.
Background
Defects such as spots, pits, scratches, color differences, and defects on the surface of an article directly affect the properties such as the appearance and performance of the article, and therefore, it is necessary to detect defects in the article. The defect detection refers to identifying whether defects exist on the surface of an article, determining the positions of defect areas and the types of the defects, and the traditional defect detection is realized by observing through human eyes. The defect detection mode not only consumes a large amount of time manually, but also is easy to generate detection errors. With the development of machine vision detection technology, the deep learning method replaces manual detection, and the defect area in the image to be detected is obtained by carrying out deep learning operation on the image to be detected obtained by shooting the object and analyzing the image to be detected, so that the surface defect of the object can be automatically detected by a machine.
Deep learning is an emerging field in machine learning, and data are analyzed by establishing a deep learning network and simulating a mechanism of a human brain. In the defect detection method based on deep learning, the image to be detected is input into a deep learning network obtained in advance based on sample image training, and information such as the type of the defect and the position of a defect area in the image to be detected can be obtained through the operation and analysis of the deep learning network.
However, in an actual scene, the difference between the defect feature and the feature of other regions may be small, and the regions with small feature difference cannot be distinguished through the deep learning network, so that the accuracy of defect detection is low.
Disclosure of Invention
The embodiment of the invention aims to provide a defect detection method, a defect detection device and electronic equipment so as to improve the accuracy of defect detection. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a defect detection method, where the method includes:
acquiring an image to be detected;
inputting the image to be detected into a pre-trained deep learning network model to obtain a multi-value diagram corresponding to the image to be detected;
and adjusting pixels in the multi-value image according to the image to be detected and the multi-value image to obtain a defect area in the image to be detected.
Optionally, the training mode of the deep learning network model includes:
acquiring an original sample image, wherein the original sample image comprises a plurality of defective sample images and a plurality of non-defective sample images;
processing the original sample image to obtain an expanded sample image;
and taking the original sample image and the extended sample image as training samples, and training to obtain a deep learning network model.
Optionally, the processing the original sample image to obtain an extended sample image includes:
and performing enhancement operation on any original sample image to obtain an expanded sample image, wherein the enhancement operation comprises at least one of rotation operation, random brightness conversion operation and turning operation.
Optionally, before the training obtains the deep learning network model, the method further includes:
acquiring image data in a preset data set;
pre-training a deep learning network model by using the image data in the preset training set as training samples;
the training of the original sample image and the extended sample image as training samples to obtain a deep learning network model includes:
and adjusting the pre-trained deep learning network model according to the original sample image and the extended sample image to obtain the deep learning network model.
Optionally, the pixel value of each pixel in the multi-value image is fixed; the pixel values of the pixels are obtained by performing semantic segmentation processing on the image to be detected;
according to the image to be detected and the multi-value image, adjusting the pixels in the multi-value image to obtain the defect area in the image to be detected, comprising:
acquiring an incidence relation between adjacent pixels in the image to be detected;
adjusting the pixel value of each pixel in the multi-value image based on the association relation;
and determining a defect area in the image to be detected according to the adjusted pixel value of each pixel.
In a second aspect, an embodiment of the present invention provides a defect detection apparatus, where the apparatus includes:
the acquisition module is used for acquiring an image to be detected;
the operation module is used for inputting the image to be detected into a pre-trained deep learning network model to obtain a multi-value diagram corresponding to the image to be detected;
and the adjusting module is used for adjusting the pixels in the multi-value image according to the image to be detected and the multi-value image to obtain the defect area in the image to be detected.
Optionally, the obtaining module is further configured to: acquiring an original sample image, wherein the original sample image comprises a plurality of defective sample images and a plurality of non-defective sample images;
the device further comprises:
the processing module is used for processing the original sample image to obtain an expanded sample image;
and the training module is used for training the original sample image and the extended sample image to obtain a deep learning network model.
Optionally, the processing module is specifically configured to:
and performing enhancement operation on any original sample image to obtain an expanded sample image, wherein the enhancement operation comprises at least one of rotation operation, random brightness conversion operation and turning operation.
Optionally, the obtaining module is further configured to: acquiring image data in a preset data set;
the device further comprises:
the pre-training module is used for pre-training the deep learning network model by taking the image data in the preset training set as a training sample;
the training module is further configured to: and adjusting the pre-trained deep learning network model according to the original sample image and the extended sample image to obtain the deep learning network model.
Optionally, the pixel value of each pixel in the multi-value image is fixed; the pixel values of the pixels are obtained by performing semantic segmentation processing on the image to be detected;
the adjusting module is specifically configured to:
acquiring an incidence relation between adjacent pixels in the image to be detected;
adjusting the pixel value of each pixel in the multi-value image based on the association relation;
and determining a defect area in the image to be detected according to the adjusted pixel value of each pixel.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, wherein,
the memory is used for storing a computer program;
the processor is configured to implement any step of the defect detection method provided in the first aspect of the embodiment of the present invention when executing the computer program stored in the memory.
According to the defect detection method, the defect detection device and the electronic equipment, the image to be detected is obtained, the image to be detected is input into the depth learning network trained in advance, the multi-value image corresponding to the image to be detected is obtained, and the pixels in the multi-value image are adjusted according to the image to be detected and the multi-value image, so that the defect area in the image to be detected is obtained. The defect characteristics in the image to be detected and other area characteristics are possibly slightly different, but pixels between the defect characteristics and other area characteristics are obviously different, so that the pixels in the multi-value image can be adjusted according to the image to be detected and the multi-value image, and the defect area in the multi-value image and the area with small characteristic difference are distinguished through the pixel adjusting process, so that the detection accuracy of the defect area in the image to be detected is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a simplified flow diagram of a defect detection method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a defect detection method according to an embodiment of the invention;
FIG. 3a is a schematic view of a cloth according to an embodiment of the present invention having a wrong yarn defect;
FIG. 3b is a schematic view of a cloth of an embodiment of the present invention showing a broken needle defect;
FIG. 3c is a schematic view of a cloth having a defect of a scutching line according to an embodiment of the present invention;
FIG. 3d is a schematic diagram illustrating a cloth having a hole defect according to an embodiment of the present invention;
FIG. 4 is a schematic view showing the defects of chopsticks according to the embodiment of the present invention;
FIG. 5a is a schematic diagram of a predetermined deep learning network model according to an embodiment of the present invention;
FIG. 5b is a schematic diagram of a predetermined deep learning network model according to another embodiment of the present invention;
FIG. 5c is a schematic diagram of a predetermined deep learning network model according to another embodiment of the present invention;
FIG. 5d is a schematic diagram of a predetermined deep learning network model according to yet another embodiment of the present invention;
FIG. 6 is a multi-value graph according to an embodiment of the present invention;
FIG. 7a is a schematic diagram of a cloth defect in accordance with an embodiment of the present invention;
FIG. 7b is a schematic diagram of the defect detection result obtained in FIG. 7a according to an embodiment of the present invention;
FIG. 7c is a schematic view of a cloth defect according to another embodiment of the present invention;
FIG. 7d is a schematic diagram of the defect detection result obtained in FIG. 7c according to an embodiment of the present invention;
FIG. 7e is a schematic diagram of a cloth defect according to yet another embodiment of the present invention;
FIG. 7f is a schematic diagram of the defect detection result obtained in FIG. 7e according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a training process of a deep learning network model according to an embodiment of the present invention;
FIG. 9 is a flowchart illustrating an exemplary defect detection method according to an embodiment of the invention;
FIG. 10 is a schematic structural diagram of a defect detection apparatus according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to improve the accuracy of defect detection, the embodiment of the invention provides a defect detection method, a defect detection device and electronic equipment.
The terms in the examples of the present invention are explained as follows:
full convolution network: the full convolution network is one type of convolution network, and the convolution network is one branch of the deep learning network. Convolutional networks are generally composed of convolutional layers, pooling layers, nonlinear layers, and fully-connected layers, etc., whereas a fully-convolutional network refers to a convolutional network that does not contain fully-connected layers.
Multi-value graph: the pixel values of the pixels in the image are a plurality of fixed values.
Deep learning: is a technology for realizing machine learning.
Markov chain: a model employed in a machine learning algorithm.
Bayesian algorithm: a machine learning algorithm is an algorithm for classifying by using probability statistical knowledge.
Caffe: a clear, high-readability and fast deep learning framework.
Tensorflow: another deep learning framework.
First, a defect detection method provided in an embodiment of the present invention is described below.
An execution subject of the defect detection method provided by the embodiment of the invention can be an electronic device, the electronic device is used for realizing functions such as image processing, target identification and the like, and the electronic device at least comprises a chip capable of completing logic processing. The defect detection method provided by the embodiment of the invention can be realized by at least one of software, hardware circuit and logic circuit arranged in the execution main body.
The embodiment of the invention provides a defect detection method, and a flow diagram of the defect detection method is shown in FIG. 1. The method mainly comprises the following steps: acquiring an image to be detected; inputting an image to be detected into a deep learning network model to obtain a multi-value image; post-processing the multivalued graph; based on the post-processing result, a defective region is acquired.
In the following, the defect detection method shown in fig. 1 is described in more detail, and as shown in fig. 2, the surface defect detection method may include the following steps:
s201, acquiring an image to be detected.
The image to be detected is an image which needs to be subjected to defect detection, and the image to be detected can be an image of an article which is stored in a database or an image obtained by shooting the article in real time. Exemplarily, the article to be defect detected is a piece of cloth, the image to be detected is an image obtained by shooting the piece of cloth, and the piece of cloth in the image to be detected may have defects as shown in fig. 3a (wrong yarn), fig. 3b (broken needle), fig. 3c (open line) or fig. 3d (broken hole); or, the article to be detected is a chopstick, the image to be detected is an image obtained by shooting the chopstick, and the chopstick in the image to be detected may have the defect shown in fig. 4.
S202, inputting the image to be detected into a pre-trained deep learning network model to obtain a multi-value image corresponding to the image to be detected.
The deep learning network model is obtained based on training of a defect sample image and a non-defect sample image, the confidence coefficient of each pixel in an image to be detected as a defect can be obtained through the deep learning network model, and a multi-value image corresponding to the image to be detected can be established based on the confidence coefficient of each pixel, namely the deep learning network model is a network model capable of obtaining the multi-value image. The pixel values of the pixels in the multi-value map are fixed values, for example, if the multi-value map is a binary map, there are only two pixel values (0 and 1) in the binary map, where 0 represents that the pixel is defect-free and 1 represents that the pixel is defect-free, and the size of the multi-value map is in direct proportion to the size of the image to be detected, and the proportion value is a real number less than or equal to 1, such as 1, 1/2, 1/4, 1/6, etc.
The preset deep learning network model may be a full convolutional network (as shown in fig. 5 a), a plurality of full convolutional networks connected in series (as shown in fig. 5 b), a plurality of full convolutional networks connected in parallel (as shown in fig. 5 c), or a combination of series connection and parallel connection of a plurality of full convolutional networks (as shown in fig. 5 d).
The deep learning network model is a network model capable of realizing region division based on pixels in an image to be detected, the pixel values of all pixels in the obtained multi-value image are fixed, and specifically, the pixel values of a plurality of pixels in the multi-value image can be obtained by performing semantic segmentation processing on the image to be detected. The image semantic segmentation processing can automatically segment and identify the content in the image, and the pixel values in the target areas with the same attribute are the same, for example, for the image of one person riding a motorcycle, through the image semantic segmentation processing, a multi-value graph is generated, wherein the human area is pink, the motorcycle area is dark green, and the background area is black.
When the image to be detected is subjected to the semantic segmentation processing, the obtained multivalued graph can be a binary graph, i.e. 0 in the binary graph indicates that the pixel is defect-free, and 1 indicates that the pixel is defective, for example, the image to be detected shown in fig. 4 is subjected to the semantic segmentation processing, and the obtained multivalued graph is shown in fig. 6, wherein a white area is a defective area, and a black area is a non-defective area. The multi-value map may also have the defective regions in one color (e.g., pink), the non-defective regions in another color (e.g., blackish green), and the background regions in yet another color (e.g., black).
S203, adjusting the pixels in the multi-value image according to the image to be detected and the multi-value image to obtain a defect area in the image to be detected.
However, because the boundary of the defect region is often blurred during defect identification, an error usually exists between the boundary of the defect region in the multi-value image and an actual situation, and even more, because the defect is not obvious, the difference between the defect region and other regions may not be obviously shown in the multi-value image, and the pixel information in the image to be detected can truly reflect the difference between different target regions, therefore, the pixels in the multi-value image can be adjusted according to the image to be detected and the multi-value image, so that the difference between the defect region and other regions can be obviously reflected by the adjusted multi-value image, and the detection accuracy of the defect region in the image to be detected can be improved.
The process of adjusting the pixels in the multi-value map may be to adjust the pixel values of the pixels, and the pixel values with larger differences may be set for the target areas with different attributes, or of course, the target areas with different attributes may have larger differences in the multi-value map by adjusting the brightness, the contrast, and the like of the pixels. Since the size of the multivalued map is proportional to the size of the image to be detected, the ratio between the image to be detected and the multivalued map also needs to be considered when adjusting the multivalued map pixels. For example, if the ratio is 1/6, the pixels of the upper left pixel block of the multi-value image are adjusted based on the pixel information of the upper left 6 pixel blocks of the image to be detected.
Optionally, the pixel value of each pixel in the multi-value image is fixed; the pixel values of the plurality of pixels are obtained by performing semantic segmentation processing on the image to be detected.
S203 may specifically be: acquiring an association relation between adjacent pixels in an image to be detected; adjusting the pixel value of each pixel in the multi-value image based on the association relation; and determining a defect area in the image to be detected according to the adjusted pixel value of each pixel.
The incidence relation between adjacent pixels in the image to be detected is the probability that the upper, lower, left, right and other adjacent pixels of one pixel in the image to be detected are a certain pixel. Specifically, the association relationship between adjacent pixels in the image to be detected may be obtained by a conventional machine learning method, for example, the conventional machine learning method may be a markov chain, a bayesian algorithm, or the like. Based on the incidence relation between adjacent pixels in the image to be detected, the incidence relation can be applied to the multi-value image, the pixels in the multi-value image are adjusted based on the incidence relation, the pixel values of part of the pixels in the multi-value image are changed, the effect of determining the defect area in the image to be detected is improved by further mining the information of the image to be detected, and the detected defect area is more consistent with the actual defect area.
In correspondence with the above embodiment, by detecting a defect in the cloth shown in fig. 7a, a defect detection result shown in fig. 7b can be obtained; by detecting a defect in the cloth as shown in FIG. 7c, a defect detection result as shown in FIG. 7d can be obtained; by detecting a defect in the cloth as shown in fig. 7e, a defect detection result as shown in fig. 7f can be obtained.
By applying the embodiment, the image to be detected is obtained, the image to be detected is input into the depth learning network trained in advance, the multi-value image corresponding to the image to be detected is obtained, and the pixels in the multi-value image are adjusted according to the image to be detected and the multi-value image, so that the defect area in the image to be detected is obtained. The defect characteristics in the image to be detected and other area characteristics are possibly slightly different, but pixels between the defect characteristics and other area characteristics are obviously different, so that the pixels in the multi-value image can be adjusted according to the image to be detected and the multi-value image, and the defect area in the multi-value image and the area with small characteristic difference are distinguished through the pixel adjusting process, so that the detection accuracy of the defect area in the image to be detected is improved.
As shown in fig. 8, which is a schematic diagram of a training process of a deep learning network model according to an embodiment of the present invention, the training of the deep learning network model may include the following steps:
s801, acquiring an original sample image, wherein the original sample image comprises a plurality of defective sample images and a plurality of non-defective sample images.
The defective sample image and the non-defective sample image may be previously collected original sample images with labels according to whether there is a defective mark in the images at the time of original sample image collection. If the label is defective, the sample image is a defective sample image; if the label is defect-free, the sample image is a defect-free sample image.
S802, processing the original sample image to obtain an expanded sample image.
In an actual application scene, the number of acquired original sample images is often limited, and all defects in a complex scene cannot be covered, so that an error exists between a detected defect area and an actual situation. Therefore, it is necessary to perform expansion processing on the acquired original sample image to obtain an expanded sample image.
Since the types of defects are fixed, mainly including scratches, dents, cracks, and the like, and the brightness, defect angle, and the like of the original sample image are changed a lot, the expanded sample image can be obtained by adjusting the brightness, angle, and the like of the original sample image.
Optionally, S802 may specifically be: and performing an enhancement operation on any original sample image to obtain an expanded sample image, wherein the enhancement operation can comprise at least one of a rotation operation, a random brightness conversion operation and a turning operation.
The rotation operation may be to rotate the original sample image by a preset angle, and the preset angle may be multiple, for example, 15 degrees, 30 degrees, 45 degrees, 60 degrees, 90 degrees, and the like, and according to the preset angle, the rotation operation may be performed on any original sample image. The random luminance transformation may be a random luminance transformation operation performed on the sample image according to a random luminance transformation strategy to maximally include sample images of various luminances. The flipping operation may be a flipping process of the original sample image based on the axis of symmetry. Of course, the enhancement operation may be performed on the original image by performing only one of the above-described rotation operation, random brightness operation, and inversion operation on the original image, or by performing any two or three operations simultaneously.
And S803, training the original sample image and the extended sample image as training samples to obtain the deep learning network model.
The original sample images and the extended sample images are used together to train a deep learning network model. The training process can be implemented on platforms such as a buffer (Convolutional neural network framework), a Tensorflow which realizes transmission of a complex data structure to an artificial intelligent neural network for analysis and processing, and the like.
In addition, the original sample images may be small in number, and the extended sample images obtained through the data enhancement operation cannot ensure complete coverage of various defects, so that the accuracy of the deep learning network model obtained through training of the original sample images and the extended sample images is low.
Optionally, before training to obtain the deep learning network model, the method may further include: acquiring image data in a preset data set; and pre-training the deep learning network model by using the image data in the preset training set as a training sample.
S803 may specifically be: and adjusting the pre-trained deep learning network model according to the original sample image and the extended sample image to obtain the deep learning network model.
Some specified data sets, such as coco (Common Objects in Context) data sets and the like, these data sets store conventional, labeled defect image data, based on which, the pre-training of the deep learning network model can be carried out, the network weight of the deep learning network model is initialized through the pre-training, thus, the depth learning network model after pre-training is adjusted based on the original sample image and the extended sample image, so as to obtain a more accurate depth learning network model, the process of adjusting the pre-trained deep learning network model according to the original sample image and the extended sample image may be to continue training the pre-trained deep learning network model by using the original sample image and the extended sample image as training samples.
In the training process of the deep learning network model, the original sample images are processed to obtain extended sample images, so that the number of the sample images is effectively extended; and based on the image data in the preset data set, the pre-training of the deep learning network model is firstly carried out, and then the pre-trained deep learning network model is adjusted by utilizing the original sample image and the extended sample image, so that the final deep learning network model can cover a larger range of defect types, and the accuracy of the deep learning network model in defect detection is improved.
For convenience of understanding, the defect detection method according to the embodiment of the present invention is described below with reference to specific examples. As shown in fig. 9, the method mainly includes the following steps:
step one, data preparation.
The prepared data may be image data, collecting labeled raw sample images, including defective sample images and non-defective sample images. And performing rotation operations of different angles on the collected original sample image to generate a new extended sample image with a label.
Optionally, the collected original sample image may be further subjected to operations such as rotation at different angles, random luminance transformation, symmetric inversion, or any combination of the operations, such as rotation at different angles, random luminance transformation, and symmetric inversion, to generate a new extended sample image with a label.
The extended sample image and the original sample image can be used as training samples together to train the deep learning network model; the original sample image can also be directly used as a training sample to train the deep learning network model.
And step two, training a deep learning network model.
And (4) directly training the deep learning network model by using the training sample prepared in the step one.
Optionally, the deep learning network model may be trained, or the deep learning network model may be pre-trained using image data with labels other than the prepared training sample in the step one, such as a coco data set, and then the pre-trained deep learning network model may be fine-tuned using the prepared training sample in the step one.
The training platform used may be Caffe, Tensorflow, etc. The deep learning network model is a full convolution network or a combination of a plurality of full convolution networks in series or in parallel.
And step three, inputting the image to be detected into the deep learning network model to obtain a multi-value image.
The multi-value graph refers to a plurality of fixed values of pixel values of pixels in the graph. If the binary image has two pixel values: 0 and 1, where 0 indicates that the pixel is defect free and 1 indicates that the pixel is defective. And, the size of the multi-valued map is proportional to the size of the image to be detected, and the proportional value is a real number less than or equal to 1, such as 1, 1/2, 1/4, 1/6, and the like.
And fourthly, carrying out post-processing on the multivalued graph by combining the image to be detected and the multivalued graph to obtain a defect area.
The post-processing changes the pixel values of some of the pixels in the multi-valued map by mining the relationships between adjacent pixels in the map to be examined and applying them to the multi-valued map. The post-processing method may employ a conventional machine learning method, such as a markov chain and a bayesian algorithm.
By the embodiment, the image to be detected is obtained, the image to be detected is input into the depth learning network trained in advance, the multi-value image corresponding to the image to be detected is obtained, and the pixels in the multi-value image are adjusted according to the image to be detected and the multi-value image, so that the defect area in the image to be detected is obtained. The defect characteristics in the image to be detected and other area characteristics are possibly slightly different, but pixels between the defect characteristics and other area characteristics are obviously different, so that the pixels in the multi-value image can be adjusted according to the image to be detected and the multi-value image, and the defect area in the multi-value image and the area with small characteristic difference are distinguished through the pixel adjusting process, so that the detection accuracy of the defect area in the image to be detected is improved. In the training process of the deep learning network model, the original sample images are processed to obtain extended sample images, so that the number of the sample images is effectively extended; and based on the image data in the preset data set, the pre-training of the deep learning network model is firstly carried out, and then the pre-trained deep learning network model is adjusted by utilizing the original sample image and the extended sample image, so that the final deep learning network model can cover a larger range of defect types, and the defect detection accuracy of the deep learning network model is further improved.
Corresponding to the above method embodiment, an embodiment of the present invention provides a defect detecting apparatus, as shown in fig. 10, including:
the obtaining module 1010 is configured to obtain an image to be detected.
And the operation module 1020 is configured to input the image to be detected into a pre-trained deep learning network model to obtain a multi-value image corresponding to the image to be detected.
And an adjusting module 1030, configured to adjust pixels in the multi-value map according to the image to be detected and the multi-value map, so as to obtain a defect area in the image to be detected.
Optionally, the obtaining module 1010 may be further configured to: an original sample image is acquired, the original sample image including a plurality of defective sample images and a plurality of non-defective sample images.
The apparatus may further include: the processing module is used for processing the original sample image to obtain an expanded sample image; and the training module is used for training the original sample image and the extended sample image to obtain a deep learning network model.
Optionally, the processing module may be specifically configured to: and performing enhancement operation on any original sample image to obtain an expanded sample image, wherein the enhancement operation comprises at least one of rotation operation, random brightness conversion operation and turning operation.
Optionally, the obtaining module 1010 may be further configured to: and acquiring image data in a preset data set.
The apparatus may further include: and the pre-training module is used for pre-training the deep learning network model by taking the image data in the preset training set as a training sample.
The training module may be further configured to: and adjusting the pre-trained deep learning network model according to the original sample image and the extended sample image to obtain the deep learning network model.
Optionally, the pixel value of each pixel in the multi-value image is fixed; and the pixel values of the plurality of pixels are obtained by performing semantic segmentation processing on the image to be detected.
The adjusting module 1030 may be specifically configured to: acquiring an incidence relation between adjacent pixels in the image to be detected; adjusting the pixel value of each pixel in the multi-value image based on the association relation; and determining a defect area in the image to be detected according to the adjusted pixel value of each pixel.
By applying the embodiment, the image to be detected is obtained, the image to be detected is input into the depth learning network trained in advance, the multi-value image corresponding to the image to be detected is obtained, and the pixels in the multi-value image are adjusted according to the image to be detected and the multi-value image, so that the defect area in the image to be detected is obtained. The defect characteristics in the image to be detected and other area characteristics are possibly slightly different, but pixels between the defect characteristics and other area characteristics are obviously different, so that the pixels in the multi-value image can be adjusted according to the image to be detected and the multi-value image, and the defect area in the multi-value image and the area with small characteristic difference are distinguished through the pixel adjusting process, so that the detection accuracy of the defect area in the image to be detected is improved.
In order to improve the accuracy of defect detection, an embodiment of the present invention further provides an electronic device, as shown in fig. 11, which includes a processor 1101 and a memory 1102, wherein,
the memory 1102 is used for storing computer programs;
the processor 1101 is configured to implement any one of the steps of the defect detection method described above when executing the computer program stored in the memory 1102.
The Memory may include a RAM (Random Access Memory) or an NVM (Non-Volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), an NP (Network Processor), and the like; but also a DSP (Digital Signal Processor), an ASIC (application specific Integrated Circuit), an FPGA (Field-Programmable gate array) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware component.
In this embodiment, the processor of the electronic device can read the computer program stored in the memory and run the computer program to implement: the method comprises the steps of obtaining an image to be detected, inputting the image to be detected into a pre-trained deep learning network to obtain a multi-value image corresponding to the image to be detected, and adjusting pixels in the multi-value image according to the image to be detected and the multi-value image to obtain a defect area in the image to be detected. The defect characteristics in the image to be detected and other area characteristics are possibly slightly different, but pixels between the defect characteristics and other area characteristics are obviously different, so that the pixels in the multi-value image can be adjusted according to the image to be detected and the multi-value image, and the defect area in the multi-value image and the area with small characteristic difference are distinguished through the pixel adjusting process, so that the detection accuracy of the defect area in the image to be detected is improved.
In addition, corresponding to the defect detection method provided in the foregoing embodiment, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements any step of the defect detection method.
In this embodiment, when running, the computer-readable storage medium executes the application program of the defect detection method provided in the embodiment of the present invention, so that the following can be implemented: the method comprises the steps of obtaining an image to be detected, inputting the image to be detected into a pre-trained deep learning network to obtain a multi-value image corresponding to the image to be detected, and adjusting pixels in the multi-value image according to the image to be detected and the multi-value image to obtain a defect area in the image to be detected. The defect characteristics in the image to be detected and other area characteristics are possibly slightly different, but pixels between the defect characteristics and other area characteristics are obviously different, so that the pixels in the multi-value image can be adjusted according to the image to be detected and the multi-value image, and the defect area in the multi-value image and the area with small characteristic difference are distinguished through the pixel adjusting process, so that the detection accuracy of the defect area in the image to be detected is improved.
For the embodiments of the electronic device and the computer-readable storage medium, since the contents of the related methods are substantially similar to those of the foregoing embodiments of the methods, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the embodiments of the methods.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus, the electronic device, and the computer-readable storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and in relation to the description, reference may be made to some portions of the description of the method embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (11)
1. A method of defect detection, the method comprising:
acquiring an image to be detected;
inputting the image to be detected into a pre-trained deep learning network model to obtain a multi-value diagram corresponding to the image to be detected;
and adjusting pixels in the multi-value image according to the image to be detected and the multi-value image to obtain a defect area in the image to be detected.
2. The method of claim 1, wherein the training of the deep learning network model comprises:
acquiring an original sample image, wherein the original sample image comprises a plurality of defective sample images and a plurality of non-defective sample images;
processing the original sample image to obtain an expanded sample image;
and taking the original sample image and the extended sample image as training samples, and training to obtain a deep learning network model.
3. The method of claim 2, wherein the processing the original sample image to obtain an extended sample image comprises:
and performing enhancement operation on any original sample image to obtain an expanded sample image, wherein the enhancement operation comprises at least one of rotation operation, random brightness conversion operation and turning operation.
4. The method of claim 2, wherein prior to the training resulting in a deep learning network model, the method further comprises:
acquiring image data in a preset data set;
pre-training a deep learning network model by using the image data in the preset training set as training samples;
the training of the original sample image and the extended sample image as training samples to obtain a deep learning network model includes:
and adjusting the pre-trained deep learning network model according to the original sample image and the extended sample image to obtain the deep learning network model.
5. The method of claim 1, wherein the pixel value of each pixel in the multi-valued map is fixed; the pixel values of the pixels are obtained by performing semantic segmentation processing on the image to be detected;
according to the image to be detected and the multi-value image, adjusting the pixels in the multi-value image to obtain the defect area in the image to be detected, comprising:
acquiring an incidence relation between adjacent pixels in the image to be detected;
adjusting the pixel value of each pixel in the multi-value image based on the association relation;
and determining a defect area in the image to be detected according to the adjusted pixel value of each pixel.
6. A defect detection apparatus, the apparatus comprising:
the acquisition module is used for acquiring an image to be detected;
the operation module is used for inputting the image to be detected into a pre-trained deep learning network model to obtain a multi-value diagram corresponding to the image to be detected;
and the adjusting module is used for adjusting the pixels in the multi-value image according to the image to be detected and the multi-value image to obtain the defect area in the image to be detected.
7. The apparatus of claim 6, wherein the obtaining module is further configured to: acquiring an original sample image, wherein the original sample image comprises a plurality of defective sample images and a plurality of non-defective sample images;
the device further comprises:
the processing module is used for processing the original sample image to obtain an expanded sample image;
and the training module is used for training the original sample image and the extended sample image to obtain a deep learning network model.
8. The apparatus of claim 7, wherein the processing module is specifically configured to:
and performing enhancement operation on any original sample image to obtain an expanded sample image, wherein the enhancement operation comprises at least one of rotation operation, random brightness conversion operation and turning operation.
9. The apparatus of claim 7, wherein the obtaining module is further configured to: acquiring image data in a preset data set;
the device further comprises:
the pre-training module is used for pre-training the deep learning network model by taking the image data in the preset training set as a training sample;
the training module is further configured to: and adjusting the pre-trained deep learning network model according to the original sample image and the extended sample image to obtain the deep learning network model.
10. The apparatus of claim 6, wherein the pixel value of each pixel in the multi-valued map is fixed; the pixel values of the pixels are obtained by performing semantic segmentation processing on the image to be detected;
the adjusting module is specifically configured to:
acquiring an incidence relation between adjacent pixels in the image to be detected;
adjusting the pixel value of each pixel in the multi-value image based on the association relation;
and determining a defect area in the image to be detected according to the adjusted pixel value of each pixel.
11. An electronic device comprising a processor and a memory, wherein,
the memory is used for storing a computer program;
the processor, when executing the computer program stored in the memory, is configured to perform the method steps of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810563652.4A CN110619619A (en) | 2018-06-04 | 2018-06-04 | Defect detection method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810563652.4A CN110619619A (en) | 2018-06-04 | 2018-06-04 | Defect detection method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110619619A true CN110619619A (en) | 2019-12-27 |
Family
ID=68919820
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810563652.4A Pending CN110619619A (en) | 2018-06-04 | 2018-06-04 | Defect detection method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110619619A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111093140A (en) * | 2019-12-11 | 2020-05-01 | 上海闻泰信息技术有限公司 | Method, device, equipment and storage medium for detecting defects of microphone and earphone dust screen |
| CN111242185A (en) * | 2020-01-03 | 2020-06-05 | 凌云光技术集团有限责任公司 | Defect rapid preliminary screening method and system based on deep learning |
| CN111681220A (en) * | 2020-06-04 | 2020-09-18 | 阿丘机器人科技(苏州)有限公司 | Defect detection model construction method, device and system and storage medium |
| CN111754470A (en) * | 2020-06-11 | 2020-10-09 | 厦门雨程户外运动用品有限公司 | Automatic cloth inspecting method and device, automatic cloth inspecting machine and storage medium |
| CN112001902A (en) * | 2020-08-19 | 2020-11-27 | 上海商汤智能科技有限公司 | Defect detection method and related device, equipment and storage medium |
| CN118261905A (en) * | 2024-04-30 | 2024-06-28 | 苏州辰瓴光学有限公司 | AI energized flexible circuit board defect detection method and detection equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105654121A (en) * | 2016-04-11 | 2016-06-08 | 李云栋 | Complex jacquard fabric defect detection method based on deep learning |
| CN106683099A (en) * | 2016-11-17 | 2017-05-17 | 南京邮电大学 | Product surface defect detection method |
| CN107194920A (en) * | 2017-05-19 | 2017-09-22 | 山东财经大学 | A kind of tire defect inspection method based on singular value decomposition |
| CN107768269A (en) * | 2017-10-30 | 2018-03-06 | 河北工业大学 | A kind of method of the dirty defects detection of polysilicon solar battery slice outward appearance |
| CN107833220A (en) * | 2017-11-28 | 2018-03-23 | 河海大学常州校区 | Fabric defect detection method based on depth convolutional neural networks and vision significance |
| CN107993219A (en) * | 2017-10-31 | 2018-05-04 | 无锡港湾网络科技有限公司 | A kind of deck of boat detection method of surface flaw based on machine vision |
-
2018
- 2018-06-04 CN CN201810563652.4A patent/CN110619619A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105654121A (en) * | 2016-04-11 | 2016-06-08 | 李云栋 | Complex jacquard fabric defect detection method based on deep learning |
| CN106683099A (en) * | 2016-11-17 | 2017-05-17 | 南京邮电大学 | Product surface defect detection method |
| CN107194920A (en) * | 2017-05-19 | 2017-09-22 | 山东财经大学 | A kind of tire defect inspection method based on singular value decomposition |
| CN107768269A (en) * | 2017-10-30 | 2018-03-06 | 河北工业大学 | A kind of method of the dirty defects detection of polysilicon solar battery slice outward appearance |
| CN107993219A (en) * | 2017-10-31 | 2018-05-04 | 无锡港湾网络科技有限公司 | A kind of deck of boat detection method of surface flaw based on machine vision |
| CN107833220A (en) * | 2017-11-28 | 2018-03-23 | 河海大学常州校区 | Fabric defect detection method based on depth convolutional neural networks and vision significance |
Non-Patent Citations (1)
| Title |
|---|
| 谢静 等；: "《基于MRF模型的金属工件红外热像表面缺陷检测》", 《石油矿场机械》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111093140A (en) * | 2019-12-11 | 2020-05-01 | 上海闻泰信息技术有限公司 | Method, device, equipment and storage medium for detecting defects of microphone and earphone dust screen |
| CN111242185A (en) * | 2020-01-03 | 2020-06-05 | 凌云光技术集团有限责任公司 | Defect rapid preliminary screening method and system based on deep learning |
| CN111242185B (en) * | 2020-01-03 | 2023-10-27 | 凌云光技术股份有限公司 | Defect rapid preliminary screening method and system based on deep learning |
| CN111681220A (en) * | 2020-06-04 | 2020-09-18 | 阿丘机器人科技(苏州)有限公司 | Defect detection model construction method, device and system and storage medium |
| CN111681220B (en) * | 2020-06-04 | 2024-02-13 | 阿丘机器人科技(苏州)有限公司 | Method, device, system and storage medium for constructing defect detection model |
| CN111754470A (en) * | 2020-06-11 | 2020-10-09 | 厦门雨程户外运动用品有限公司 | Automatic cloth inspecting method and device, automatic cloth inspecting machine and storage medium |
| CN112001902A (en) * | 2020-08-19 | 2020-11-27 | 上海商汤智能科技有限公司 | Defect detection method and related device, equipment and storage medium |
| CN118261905A (en) * | 2024-04-30 | 2024-06-28 | 苏州辰瓴光学有限公司 | AI energized flexible circuit board defect detection method and detection equipment |
| CN118261905B (en) * | 2024-04-30 | 2024-09-10 | 苏州辰瓴光学有限公司 | AI energized flexible circuit board defect detection method and detection equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110619619A (en) | Defect detection method and device and electronic equipment | |
| CN110060237B (en) | Fault detection method, device, equipment and system | |
| CN111553929B (en) | Mobile phone screen defect segmentation method, device and equipment based on converged network | |
| CN111612763B (en) | Mobile phone screen defect detection method, device and system, computer equipment and medium | |
| CN112348787B (en) | Training method of object defect detection model, object defect detection method and device | |
| CN109285139A (en) | A kind of x-ray imaging weld inspection method based on deep learning | |
| CN110619618A (en) | Surface defect detection method and device and electronic equipment | |
| CN109671058B (en) | Defect detection method and system for large-resolution image | |
| CN111667455A (en) | AI detection method for various defects of brush | |
| CN111582359B (en) | Image identification method and device, electronic equipment and medium | |
| CN111815564B (en) | Method and device for detecting silk ingots and silk ingot sorting system | |
| CN110889838A (en) | Fabric defect detection method and device | |
| CN110245697B (en) | Surface contamination detection method, terminal device and storage medium | |
| CN114581432B (en) | Tongue image segmentation method of tongue image instrument based on deep learning | |
| CN116012291A (en) | Industrial part image defect detection method and system, electronic equipment and storage medium | |
| CN112750113B (en) | Glass bottle defect detection method and device based on deep learning and linear detection | |
| TW201512649A (en) | Method of chip detects inspecting, system therefor, and computer program product thereof | |
| US20190272627A1 (en) | Automatically generating image datasets for use in image recognition and detection | |
| CN115861170A (en) | Surface defect detection method based on improved YOLO V4 algorithm | |
| CN116385380A (en) | Defect detection method, system, equipment and storage medium based on depth characteristics | |
| CN114140463A (en) | Welding defect identification method, device, equipment and storage medium | |
| CN116309526A (en) | Paper cup defect detection method and device | |
| CN116152191A (en) | Display screen crack defect detection method, device and equipment based on deep learning | |
| Bhardwaj et al. | An imaging approach for the automatic thresholding of photo defects | |
| CN115471494A (en) | Wo citrus quality inspection method, device, equipment and storage medium based on image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |