CN110020691B - Liquid crystal screen defect detection method based on convolutional neural network impedance type training - Google Patents
Liquid crystal screen defect detection method based on convolutional neural network impedance type training Download PDFInfo
- Publication number
- CN110020691B CN110020691B CN201910290488.9A CN201910290488A CN110020691B CN 110020691 B CN110020691 B CN 110020691B CN 201910290488 A CN201910290488 A CN 201910290488A CN 110020691 B CN110020691 B CN 110020691B
- Authority
- CN
- China
- Prior art keywords
- defect
- sample
- normal
- convolutional neural
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
Abstract
The invention relates to a liquid crystal screen defect detection method based on convolutional neural network impedance type training, and belongs to the technical field of screen detection. The method comprises the following steps: converting the defect sample into a normal sample by a frequency domain Gaussian filtering or time domain Gaussian blurring method, and suggesting the frequency domain Gaussian filtering, because the blurring mode has small damage degree to the normal background characteristic; in the antagonism training process, inputting the defect sample and the smoothed normal sample into a convolutional neural network in pairs; by setting the corresponding loss function, the network learns the feature difference of the defect region and the normal background which is not directly related, and the difference between the defect region and the normal background which is directly related. The algorithm provided by the invention adopts a antagonism training method, and the difference between the defect and the background can be quickly learned even under the condition of a small number of training samples.
Description
Technical Field
The invention belongs to the technical field of screen detection, and relates to a liquid crystal screen defect detection method based on convolutional neural network impedance type training.
Background
1. State of the art
The production links of the liquid crystal screen, including cutting, assembling, welding and the like, are basically automated, but manual detection is still used in the quality detection link. The greatest disadvantage of manual inspection is the high cost, so many module manufacturers are actively looking for the relevant AOI equipment.
The reason for restricting the mass production of the AOI equipment is that the algorithm detection has low first pass rate, the algorithm debugging difficulty is high, the training time is long, and the like. Many vision enterprises are actively developing such equipment, but how to reduce equipment cost, especially maintenance cost, on the premise of ensuring high accuracy is still a difficult problem.
2. Prior art solutions
1) Digital image processing based scheme
The scheme mainly adopts a defect enhancement or background reconstruction method and then detects through setting a threshold. The enhancement method mostly adopts frequency domain enhancement, and the background reconstruction method mostly adopts methods such as PCA and ICA.
2) Secondary judgment method based on pattern recognition or neural network
The method comprises the steps of firstly searching possible defect regions through digital image processing, extracting the regions for modeling, and making accurate judgment. Most of modeling algorithms are EM, SVM or BP networks.
3) Based on deep learning method,
Most of the methods collect a large number of training samples, and perform CNN or DBN training or RNN training.
3. The shortcomings of the existing scheme
1) The algorithm has high complexity and low detection speed.
2) More training samples are required.
3) The detection effect on the Mura defect which is not obvious is not good.
4) The Mura defect detection effect at the edge is not good.
5) The detection super-parameters are more, and the threshold parameters are difficult to unify for different types of templates.
Disclosure of Invention
In view of the above, the present invention aims to provide a method for detecting defects of a liquid crystal display based on convolutional neural network antagonistic training, which trains a convolutional neural network model for simultaneously detecting multiple types of defects (Mura defects, point defects, line defects, etc.) in a short time through a small number of training samples.
In order to achieve the purpose, the invention provides the following technical scheme:
the liquid crystal screen defect detection method based on convolutional neural network impedance training comprises the following steps:
collecting a defect-free sample image A on a normal module, then making a defect on the module and collecting a sample B containing the defect; purposefulLearning the defective region P in the B picture B And in graph A and P B Region P of the same position A The difference in (a);
the defect sample is converted into a normal sample by a frequency domain Gaussian filtering or time domain Gaussian blurring method, and the blurring mode has small damage degree to the normal background characteristic;
in the antagonism training process, inputting the defect sample and the smoothed normal sample into a convolutional neural network in pairs;
by setting a corresponding loss function, the network learns the characteristic difference between the defect area and the normal background which is not directly related and the difference between the defect area and the normal background which is directly related; the loss function of the network is as follows:
wherein, sample ng And Sample ok Respectively representing a defect sample and a converted normal sample;
representing the confidence loss of the feature map corresponding to the defect sample, including a defect region and a normal region;
representing the coordinate regression loss in the defect sample feature map;
representing the confidence loss of the region corresponding to the defect sample characteristic diagram in the normal sample characteristic diagram, and only calculating the normal region;
further, the backbone network of the method is based on the SSD classification network architecture, the number of layers of the backbone network and the convolution mode are changed to a certain extent in consideration of factors such as the shape and the area of the defect, and the model structure is as follows:
the model structure has 36448 detection frames, and deletion is carried out according to actual conditions.
The invention has the beneficial effects that:
1. by adopting the convolutional neural network, the image characteristics do not need to be changed, and the problem that a background model needs to be established in the early stage to eliminate the uneven brightness of the image is solved. Meanwhile, the image edge characteristics cannot be damaged in the preprocessing process, and the problem that edge defects are easy to miss detection in a background algorithm can be solved.
2. The convolutional neural network can simultaneously detect various defects by one model, different algorithms do not need to be designed most aiming at different defect types, the universality of the algorithms is realized, and the number of the hyper-parameters is further reduced.
3. Aiming at the flexible production mode of enterprises, the enterprises need new models which can be quickly brought online. This not only requires a short training time for the new model, but also requires a small number of training samples. The algorithm provided by the invention adopts an antagonism training method, and can quickly learn the difference between the defect and the background even under the condition of a small amount of training samples.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.
Drawings
For purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 is a training and detection flow diagram;
FIG. 2 is a resistance training process;
FIG. 3 is a detection backbone network;
FIG. 4 illustrates modules and alternative modules in a backbone network; FIG. 4 (a) is a Conv module, FIG. 4 (b) is an Attrous module, and FIG. 4 (c) is a ResAttrous module;
FIG. 5 is a graph of accuracy versus sample number.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.
Wherein the showings are for the purpose of illustration only and not for the purpose of limiting the invention, shown in the drawings are schematic representations and not in the form of actual drawings; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there is an orientation or positional relationship indicated by the terms "upper", "lower", "left", "right", "front", "rear", etc., based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore the terms describing the positional relationship in the drawings are only used for illustrative purposes and are not to be construed as limiting the present invention, and the specific meaning of the terms described above will be understood by those skilled in the art according to the specific circumstances.
As shown in fig. 1, similar to comparative experiments in biology: under the same environment, only one factor is changed, and the influence degree of the change of the factor on the result is observed, so that the importance of the factor is visually shown. In view of this, the simplest and most effective way to distinguish between defects and normal background is: a sample A without defects is collected on a normal die set, and then a defect is made on the die set and a sample B with defects is collected. Purposeful learning of defective regions P in B picture B And in graph A and P B Region P of the same position A The difference in (a).
However, it is not feasible to realize such a destructive way of collecting a sample. However, if the reverse: since a defective block like Mura, point defect, line defect, etc., if its area is small, it can be blurred until eliminated. The defect sample can be converted into a normal sample by a frequency domain Gaussian filtering or time domain Gaussian blurring method, and the blurring mode has small damage degree to the normal background feature. The algorithm recommends the use of frequency domain gaussian filtering to eliminate the defect (if the defect area is large, the defect can be processed with the appearance difference or the brightness is not uniform).
As shown in fig. 2, in the course of the antagonism training, the defect samples and the smoothed normal samples are input into the convolutional neural network in pairs. By setting the corresponding loss function, the network can learn not only the feature difference between the defect region and the normal background which is not directly related, but also the difference between the defect region and the normal background which is directly related. The loss function of the network is as follows:
wherein, sample ng And Sample ok Respectively, a defect sample and a transformed normal sample.
And representing the confidence loss of the feature map corresponding to the defect sample, including the defect region and the normal region.
Representing the coordinate regression loss in the defect sample feature map.
And (4) representing the confidence loss of the region corresponding to the defect sample feature map in the normal sample feature map, and only calculating the normal region.
The backbone network of the algorithm is based on an SSD classification network architecture, the number of layers of the backbone network and the convolution mode are changed to a certain extent in consideration of factors such as the shape and the area of defects, and a model structure diagram is shown in FIG. 3.
In fig. 4, the blocks of fig. (a) and (b) correspond to fig. 3, and if it is necessary to deepen the network, the residual block of fig. 4 (c) can be used. The model has 36448 detection frames, and can be deleted according to actual conditions.
The total number of training samples of the algorithm is 650, which comes from 3 liquid crystal module production enterprises, and is 300 pieces of 7-inch TFT-LCD module samples, 150 pieces of 5.5-inch OLED modules and 200 pieces of 4.5-inch TFT-LCD modules respectively. The relationship between the detection accuracy and the number of samples is shown in FIG. 5.
Therefore, on the premise of less sample size, the method for antagonism training can obviously improve the accuracy of detection.
The model has the following characteristics:
1. the number of required training samples is small, and under 650 training samples, the model missing rate is less than 0.3%, and the false positive rate is less than 1.2%. The requirement of enterprises on accuracy is met.
2. The model volume is about 4M, the detection time is less than 1.8s/p (Intel E3-12263.31 Ghz)
3. Edge defects and relatively shallow Mura-type defects can be well detected.
Finally, the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all that should be covered by the claims of the present invention.
Claims (2)
1. The liquid crystal screen defect detection method based on convolutional neural network impedance training is characterized by comprising the following steps of: the method comprises the following steps:
collecting a defect-free sample image A on a normal module, then making a defect on the module and collecting a sample B containing the defect; purposeful learning of defective regions P in B picture B And in graph A and P B Region P of the same position A A difference of (a);
the defect sample is converted into a normal sample by a frequency domain Gaussian filtering or time domain Gaussian blurring method, and the blurring mode has small damage degree to the normal background characteristic;
in the antagonism training process, inputting the defect sample and the smoothed normal sample into a convolutional neural network in pairs;
by setting a corresponding loss function, the network learns the characteristic difference between the defect area and the normal background which is not directly related and the difference between the defect area and the normal background which is directly related; the loss function of the network is as follows:
wherein, sample ng And Sample ok Respectively representing a defect sample and a converted normal sample;
representing the confidence loss of the feature map corresponding to the defect sample, including the defect region and the normal region;
representing the coordinate regression loss in the defect sample feature map;
and (4) representing the confidence loss of the region corresponding to the defect sample feature map in the normal sample feature map, and only calculating the normal region.
2. The liquid crystal screen defect detection method based on convolutional neural network impedance training as claimed in claim 1, characterized in that: the backbone network of the method is based on an SSD classification network architecture, the number of layers of the backbone network and the convolution mode are changed to a certain extent in consideration of factors such as the shape and the area of defects, and the model structure is as follows:
the model structure has 36448 detection frames, and deletion is carried out according to actual conditions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910290488.9A CN110020691B (en) | 2019-04-11 | 2019-04-11 | Liquid crystal screen defect detection method based on convolutional neural network impedance type training |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910290488.9A CN110020691B (en) | 2019-04-11 | 2019-04-11 | Liquid crystal screen defect detection method based on convolutional neural network impedance type training |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110020691A CN110020691A (en) | 2019-07-16 |
| CN110020691B true CN110020691B (en) | 2022-10-11 |
Family
ID=67191185
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910290488.9A Active CN110020691B (en) | 2019-04-11 | 2019-04-11 | Liquid crystal screen defect detection method based on convolutional neural network impedance type training |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110020691B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110717378A (en) * | 2019-08-26 | 2020-01-21 | 苏州感知线智能科技有限公司 | Method and device for detecting conductive particles based on neural network algorithm |
| CN111476759B (en) * | 2020-03-13 | 2022-03-25 | 深圳市鑫信腾机器人科技有限公司 | Screen surface detection method and device, terminal and storage medium |
| CN111524120B (en) * | 2020-04-22 | 2023-04-25 | 征图新视(江苏)科技股份有限公司 | Deep learning-based detection method for fine scratches and overprinting deviation of printed matter |
| CN113255590A (en) * | 2021-06-25 | 2021-08-13 | 众芯汉创（北京）科技有限公司 | Defect detection model training method, defect detection method, device and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016201947A1 (en) * | 2015-06-16 | 2016-12-22 | 华南理工大学 | Method for automated detection of defects in cast wheel products |
| CN109559298A (en) * | 2018-11-14 | 2019-04-02 | 电子科技大学中山学院 | Emulsion pump defect detection method based on deep learning |
-
2019
- 2019-04-11 CN CN201910290488.9A patent/CN110020691B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016201947A1 (en) * | 2015-06-16 | 2016-12-22 | 华南理工大学 | Method for automated detection of defects in cast wheel products |
| CN109559298A (en) * | 2018-11-14 | 2019-04-02 | 电子科技大学中山学院 | Emulsion pump defect detection method based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 基于BP神经网络和均值差分的TFT-LCD MURA缺陷检测方法;曾毅等;《湖南理工学院学报(自然科学版)》;20170315(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110020691A (en) | 2019-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110020691B (en) | Liquid crystal screen defect detection method based on convolutional neural network impedance type training | |
| CN110175982B (en) | Defect detection method based on target detection | |
| CN108562589B (en) | Method for detecting surface defects of magnetic circuit material | |
| CN106875381B (en) | Mobile phone shell defect detection method based on deep learning | |
| US10803573B2 (en) | Method for automated detection of defects in cast wheel products | |
| CN109191457B (en) | Pathological image quality validity identification method | |
| CN105067638B (en) | Tire fetal membrane face character defect inspection method based on machine vision | |
| US20230039805A1 (en) | Method and device for testing product quality | |
| CN109671058B (en) | Defect detection method and system for large-resolution image | |
| CN102735695B (en) | Rapid lens flaw detection method and apparatus thereof | |
| CN104992449A (en) | Information identification and surface defect on-line detection method based on machine visual sense | |
| CN109615609A (en) | A kind of solder joint flaw detection method based on deep learning | |
| CN106875373A (en) | Mobile phone screen MURA defect inspection methods based on convolutional neural networks pruning algorithms | |
| CN112233090B (en) | Film flaw detection method based on improved attention mechanism | |
| CN108932523B (en) | Image classification and classified data creation system and method, and storage medium | |
| CN112686833B (en) | Industrial product surface defect detection and classification device based on convolutional neural network | |
| WO2022236876A1 (en) | Cellophane defect recognition method, system and apparatus, and storage medium | |
| CN105044122A (en) | Copper part surface defect visual inspection system and inspection method based on semi-supervised learning model | |
| CN111476307B (en) | Lithium battery surface defect detection method based on depth field adaptation | |
| US20220076404A1 (en) | Defect management apparatus, method and non-transitory computer readable medium | |
| CN112132828A (en) | Film flaw detection method based on deep learning | |
| CN114926407A (en) | Steel surface defect detection system based on deep learning | |
| CN115601355A (en) | Method and device for detecting and classifying product surface defects and storage medium | |
| CN110567967B (en) | Display panel detection method, system, terminal device and computer readable medium | |
| TW202127014A (en) | Intelligent Production Line Monitoring System and Implementation Method Thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |