CN106875373B - Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm - Google Patents
Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm Download PDFInfo
- Publication number
- CN106875373B CN106875373B CN201611154333.5A CN201611154333A CN106875373B CN 106875373 B CN106875373 B CN 106875373B CN 201611154333 A CN201611154333 A CN 201611154333A CN 106875373 B CN106875373 B CN 106875373B
- Authority
- CN
- China
- Prior art keywords
- neural network
- convolutional neural
- mobile phone
- phone screen
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30148—Semiconductor; IC; Wafer
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
A mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm comprises the following steps: 1) customizing a deep convolutional neural network, and training a neural network for detecting MURA defects of a mobile phone screen by using the existing training data; 2) the pruning operation of the convolutional neural network is carried out by using a self-adaptive template matching method, so that the network scale is reduced, and the operation time of the algorithm is shortened; 3) zooming mobile phone screen pictures shot by a high-resolution camera in different proportions to form a picture pyramid, dividing the pictures into small blocks by using a sliding window method for the pictures of each scale, and sending all the small blocks of pictures into a convolutional neural network together as a group; 4) and selecting all characteristic graphs of the middle layer as response graphs of the defects, and finally obtaining the positions of the MURA defect areas of the mobile phone screen by adopting a threshold segmentation method.
Description
Technical Field
The invention belongs to the field of target detection and identification, and relates to a method for detecting a specific target from an image, in particular to a method for detecting defects of a mobile phone screen.
Background
The traditional method for detecting the defects of the screen manually has a plurality of defects, and the method cannot meet the requirements of high efficiency and accuracy of the current industrial production at present when the industrial production is developed at a high speed. For mobile phone screen manufacturers, finding an efficient and accurate automatic detection device to replace a manual detection link is an urgent need. With the development of the fields of computer vision, image processing and the like, an automatic detection system based on machine vision becomes a good solution. According to the scheme, the high-resolution industrial camera is used for collecting the mobile phone screen image, and then the image information is processed in real time through the image analysis module, so that whether the mobile phone screen is qualified or not is judged.
The traditional screen defect detection algorithm based on machine vision is mostly designed aiming at one or more types of screen defects, and has no universality, so that a special algorithm needs to be written for special defects. Moire is inevitable on imaging when photographing due to the particularity of the screen, and the Moire problem cannot be well solved by the traditional algorithm. In addition, although the traditional algorithm can detect more obvious linear and point defects, the accuracy rate of the traditional algorithm is low for the imaging light-weight massive MURA defects. Finally, the conventional screen defect detection algorithm needs to adjust a large number of parameters, and especially when the screen product is remodeled, time is wasted by adjusting the large number of parameters. Therefore, it is of great practical value to design an algorithm with good generalization.
In recent years, deep learning methods have had a great impact in the field of computer vision. Deep learning adopts a multilayer network structure to simulate the hierarchical relationship and the transmission mode of a nervous system in a human brain, and the method is widely applied to a plurality of fields of pattern recognition and achieves good results. The method adopts a classification algorithm based on a deep convolutional neural network, uses a preprocessed local image block of a mobile phone screen as the input of a classifier, then extracts a feature map of the convolutional neural network as a detection result, and converts a defect detection problem into an image block classification problem. In addition, compared with the traditional algorithm, the convolutional neural network algorithm has less parameter setting, and the algorithm has good universality, so that the method is particularly suitable for rapid model changing of screen products, the model changing time is shortened, and the production line efficiency is improved.
Disclosure of Invention
The invention provides a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm, aiming at overcoming the defects of a screen defect detection algorithm based on machine vision.
In order to achieve the purpose, the mobile phone screen MURA defect detection method based on the convolutional neural network pruning algorithm comprises the following steps:
1) self-defining a convolutional neural network, and training the network through training data until convergence is achieved and the accuracy is high;
2) pruning the convolutional neural network by a self-adaptive template matching method, and reducing the network scale and network parameters;
3) collecting mobile phone screen picture data, generating a picture pyramid, dividing the picture pyramid into picture blocks, generating test phase data, and sending the test phase data into the pruned convolutional neural network obtained in the step 2) for operation;
4) and taking the sum of the characteristic graphs of the intermediate hidden layers as a response graph, and obtaining and enclosing the final position of the defect by adopting a threshold segmentation method, wherein the method is particularly used for detecting the MURA defect.
The pruning matched with the self-adaptive template in the step 2) is specifically as follows: corresponding the part with response of the characteristic diagram of the hidden layer in the middle of the network to an original image, taking the part as a foreground, taking the rest as a background, and calculating the average brightness difference between the foreground and the background; taking a plurality of feature maps with the largest difference for reservation; the remaining feature maps and their associated convolution kernels are pruned from the network.
The picture pyramid in step 3) is specifically: the original high resolution picture is reduced to pictures of different scales, which are collectively referred to as a set of picture pyramids. The purpose of using the picture pyramid is to detect flaws of different sizes on different scales.
The response graph in step 4) is specifically: the output of the 4 th convolutional layer is adopted, and the sum of the characteristic graphs of the layer of the network after pruning is used as a response graph of the defect.
The invention has the following beneficial effects:
the invention discloses a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm. Based on a convolutional neural network algorithm of deep learning, a self-adaptive template matching method is adopted for network pruning, and a network model and parameters are compressed, so that the effect close to real time is achieved. In addition, the convolution neural network algorithm can automatically learn background texture information, so that moire interference in imaging can be well processed. When the screen product is remodeled, only a small amount of training pictures are needed to carry out network fine tuning, and the accuracy of the network can be guaranteed to reach the standard again.
Compared with the traditional method, the method can more effectively detect the defects with light imaging, such as the nodular MURA defects. In addition, the traditional method needs to set a large number of parameters, and the accuracy of the algorithm can be ensured only by adjusting the parameters every time the model is changed. The neural network algorithm adopted by the invention has good self-adaption and generalization characteristics, can quickly change the screen product, and saves the deployment time of a production line.
Drawings
FIG. 1 is a framework diagram of the adaptive template matching pruning algorithm of the present invention.
FIG. 2 is a graphical representation of the inventive pyramid-to-tile segmentation of a picture.
Detailed Description
The technical solution of the present invention is clearly and completely explained and described below.
The invention provides a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm.
The invention discloses a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm, which comprises the following steps:
step 1, data acquisition in a training phase: and respectively collecting small blocks containing flaws and normal pictures, and marking (1 represents that the pictures contain flaws, and 0 represents that the pictures are normal). According to the following steps of 9: 1 into a training set and a validation set; and (3) self-defining the convolutional neural network, training the convolutional neural network until convergence by using the training data, and achieving higher classification accuracy on the verification set. By observing the feature map of the hidden layer in the middle of the visual convolutional neural network, whether the network well learns the flaw feature can be analyzed.
And 2, pruning the convolutional neural network by using a self-adaptive template matching method, and reducing the network scale and network parameters. Specifically, all N feature maps on the L convolutional layer will be the ith (c)i < N) the response position learned by the feature map corresponds to the original image, the partial position is used as a foreground template, the rest position of the original image is used as a background template, and the average brightness difference D between the foreground template and the background template is calculatedi. For all DiSorting in descending order, and taking the first K (K is less than N) average brightness differences DiAnd taking the sum of the maximum feature maps as a response map. The remaining feature map and its corresponding convolution kernel are pruned directly as shown in fig. 1. And the whole network is retrained after pruning, so that the network after pruning still has higher accuracy.
Step 3, generating test stage data: and acquiring the picture data of the mobile phone screen, wherein the picture comprises and only comprises the complete mobile phone screen position. And spreading the picture into an image pyramid by using the scaling scales of different proportions for multi-scale detection. For each scale of picture, the picture is divided into picture blocks with fixed sizes, and all the picture blocks are used as a group to serve as primary input data of the convolutional neural network, as shown in fig. 2.
And 4, selecting the sum of all feature maps of the 4 th convolution layer as a response map for each picture block, re-corresponding the response map to the original map, obtaining the final position of the flaw by a threshold segmentation method, and drawing out the flaw.
Claims (3)
1. A mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm comprises the following steps:
1) self-defining a convolutional neural network, and training the network through training data until convergence;
2) pruning the convolutional neural network by a self-adaptive template matching method, and reducing the network scale and network parameters; specifically, in all the N feature maps on the lth convolutional layer, the learned response position of the ith (i < N) feature map is corresponding to the original map, the partial position is used as a foreground template, the rest position of the original map is used as a background template, and the average brightness difference Di between the foreground template and the background template is calculated;
sorting all Di in descending order, taking the sum of the first K (K < N) feature graphs with the largest average brightness difference Di as a response graph, and directly pruning the rest feature graphs and the corresponding convolution kernels thereof;
3) collecting mobile phone screen picture data, generating a picture pyramid, dividing the picture pyramid into picture blocks, generating test phase data, and sending the test phase data into the pruned convolutional neural network obtained in the step 2) for operation;
4) and taking the sum of the characteristic graphs of the middle hidden layers as a response graph, and obtaining and enclosing the final position of the defect by adopting a threshold segmentation method, wherein the method is used for detecting the MURA defect.
2. The method of claim 1, wherein: the picture pyramid in step 3) is specifically: reducing the original high-resolution picture into pictures with different scales, wherein the pictures with different scales are collectively called a group of picture pyramids; the purpose of using the picture pyramid is to detect flaws of different sizes on different scales.
3. The method according to claim 1, wherein the response map of step 4) is specifically: the output of the 4 th convolutional layer is adopted, and the sum of the characteristic graphs of the layer of the network after pruning is used as a response graph of the defect.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611154333.5A CN106875373B (en) | 2016-12-14 | 2016-12-14 | Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611154333.5A CN106875373B (en) | 2016-12-14 | 2016-12-14 | Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106875373A CN106875373A (en) | 2017-06-20 |
| CN106875373B true CN106875373B (en) | 2019-12-20 |
Family
ID=59164615
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611154333.5A Active CN106875373B (en) | 2016-12-14 | 2016-12-14 | Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106875373B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7294768B2 (en) | 2017-08-04 | 2023-06-20 | オムロン株式会社 | Image processing system |
| CN109784129A (en) * | 2017-11-14 | 2019-05-21 | 北京京东尚科信息技术有限公司 | Information output method and device |
| CN107742294A (en) * | 2017-11-27 | 2018-02-27 | 歌尔股份有限公司 | Scratch detection method, apparatus and electronic equipment |
| CN108171707A (en) * | 2018-01-23 | 2018-06-15 | 武汉精测电子集团股份有限公司 | A kind of Mura defects level evaluation method and device based on deep learning |
| CN108492291B (en) * | 2018-03-12 | 2022-07-22 | 苏州天准科技股份有限公司 | CNN segmentation-based solar photovoltaic silicon wafer defect detection system and method |
| CN108802041B (en) * | 2018-03-16 | 2023-05-05 | 惠州旭鑫智能技术有限公司 | Method for rapidly changing small sample set of screen detection |
| CN108844966A (en) * | 2018-07-09 | 2018-11-20 | 广东速美达自动化股份有限公司 | A kind of screen detection method and detection system |
| CN109063834B (en) * | 2018-07-12 | 2021-07-20 | 浙江工业大学 | Neural network pruning method based on convolution characteristic response graph |
| CN109509172A (en) * | 2018-09-25 | 2019-03-22 | 无锡动视宫原科技有限公司 | A kind of liquid crystal display flaw detection method and system based on deep learning |
| US11568324B2 (en) | 2018-12-20 | 2023-01-31 | Samsung Display Co., Ltd. | Adversarial training method for noisy labels |
| CN109840901B (en) * | 2019-01-09 | 2020-08-25 | 武汉精立电子技术有限公司 | Quick judgment method for split screen Mura |
| CN109829895B (en) * | 2019-01-09 | 2021-08-17 | 武汉精立电子技术有限公司 | AOI defect detection method based on GAN |
| CN109858613B (en) * | 2019-01-22 | 2021-02-19 | 鹏城实验室 | Compression method and system of deep neural network and terminal equipment |
| CN109829914B (en) * | 2019-02-26 | 2021-09-03 | 视睿（杭州）信息科技有限公司 | Method and device for detecting product defects |
| CN110097084B (en) * | 2019-04-03 | 2021-08-31 | 浙江大学 | Knowledge fusion method for training multitask student network through projection characteristics |
| CN110211121B (en) * | 2019-06-10 | 2021-07-16 | 北京百度网讯科技有限公司 | Method and device for pushing model |
| CN110532859B (en) * | 2019-07-18 | 2021-01-22 | 西安电子科技大学 | Remote sensing image target detection method based on deep evolution pruning convolution net |
| CN110619391B (en) * | 2019-09-19 | 2023-04-18 | 华南理工大学 | Detection model compression method and device and computer readable storage medium |
| CN110728681B (en) * | 2019-12-19 | 2020-05-22 | 武汉精立电子技术有限公司 | Mura defect detection method and device |
| CN111080633A (en) * | 2019-12-20 | 2020-04-28 | Oppo（重庆）智能科技有限公司 | Screen defect detection method and device, terminal equipment and storage medium |
| CN111476759B (en) * | 2020-03-13 | 2022-03-25 | 深圳市鑫信腾机器人科技有限公司 | Screen surface detection method and device, terminal and storage medium |
| CN111583129A (en) * | 2020-04-09 | 2020-08-25 | 天津大学 | Screen shot image moire removing method based on convolutional neural network AMNet |
| CN111563883B (en) * | 2020-04-22 | 2023-05-05 | 惠州旭鑫智能技术有限公司 | Screen vision positioning method, positioning equipment and storage medium |
| CN111681229B (en) * | 2020-06-10 | 2023-04-18 | 创新奇智(上海)科技有限公司 | Deep learning model training method, wearable clothes flaw identification method and wearable clothes flaw identification device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036323A (en) * | 2014-06-26 | 2014-09-10 | 叶茂 | Vehicle detection method based on convolutional neural network |
| CN104850858A (en) * | 2015-05-15 | 2015-08-19 | 华中科技大学 | Injection-molded product defect detection and recognition method |
| CN104977313A (en) * | 2014-04-09 | 2015-10-14 | 四川省特种设备检验研究院 | Method and device for detecting and identifying X-ray image defects of welding seam |
| CN105118044A (en) * | 2015-06-16 | 2015-12-02 | 华南理工大学 | Method for automatically detecting defects of wheel-shaped cast product |
-
2016
- 2016-12-14 CN CN201611154333.5A patent/CN106875373B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104977313A (en) * | 2014-04-09 | 2015-10-14 | 四川省特种设备检验研究院 | Method and device for detecting and identifying X-ray image defects of welding seam |
| CN104036323A (en) * | 2014-06-26 | 2014-09-10 | 叶茂 | Vehicle detection method based on convolutional neural network |
| CN104850858A (en) * | 2015-05-15 | 2015-08-19 | 华中科技大学 | Injection-molded product defect detection and recognition method |
| CN105118044A (en) * | 2015-06-16 | 2015-12-02 | 华南理工大学 | Method for automatically detecting defects of wheel-shaped cast product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106875373A (en) | 2017-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106875373B (en) | Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm | |
| CN107123111B (en) | Deep residual error network construction method for mobile phone screen defect detection | |
| CN107437245B (en) | High-speed railway contact net fault diagnosis method based on deep convolutional neural network | |
| CN108960245B (en) | Tire mold character detection and recognition method, device, equipment and storage medium | |
| CN107330453B (en) | Pornographic image identification method based on step-by-step identification and fusion key part detection | |
| CN111126325B (en) | Intelligent personnel security identification statistical method based on video | |
| CN110717481B (en) | Method for realizing face detection by using cascaded convolutional neural network | |
| CN113052210B (en) | Rapid low-light target detection method based on convolutional neural network | |
| CN111553929A (en) | Mobile phone screen defect segmentation method, device and equipment based on converged network | |
| CN110675368B (en) | Cell image semantic segmentation method integrating image segmentation and classification | |
| CN113469953B (en) | Transmission line insulator defect detection method based on improved YOLOv4 algorithm | |
| CN109034184B (en) | Grading ring detection and identification method based on deep learning | |
| CN108960404B (en) | Image-based crowd counting method and device | |
| CN113554631B (en) | Chip surface defect detection method based on improved network | |
| CN111209858A (en) | Real-time license plate detection method based on deep convolutional neural network | |
| CN113592911B (en) | Apparent enhanced depth target tracking method | |
| CN111242026B (en) | Remote sensing image target detection method based on spatial hierarchy perception module and metric learning | |
| CN116385430A (en) | Machine vision flaw detection method, device, medium and equipment | |
| CN115375991A (en) | Strong/weak illumination and fog environment self-adaptive target detection method | |
| CN114972246A (en) | Die-cutting product surface defect detection method based on deep learning | |
| CN111160100A (en) | Lightweight depth model aerial photography vehicle detection method based on sample generation | |
| CN114549489A (en) | Carved lipstick quality inspection-oriented instance segmentation defect detection method | |
| CN108154199B (en) | High-precision rapid single-class target detection method based on deep learning | |
| CN110136098B (en) | Cable sequence detection method based on deep learning | |
| CN114596244A (en) | Infrared image identification method and system based on visual processing and multi-feature fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20180418 Address after: 516006 Huizhou, Zhongkai hi tech Zone, Guangdong, Chen Jiang Street, 36 North (two) fifth. Applicant after: Huizhou Xu Xin Intelligent Technology Co., Ltd. Address before: 310027 Hangzhou, Zhejiang Province, Xihu District, Zhejiang Road, No. 38, No. Applicant before: Zhejiang University |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |