CN106875373B - Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm - Google Patents

Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm Download PDF

Info

Publication number
CN106875373B
CN106875373B CN201611154333.5A CN201611154333A CN106875373B CN 106875373 B CN106875373 B CN 106875373B CN 201611154333 A CN201611154333 A CN 201611154333A CN 106875373 B CN106875373 B CN 106875373B
Authority
CN
China
Prior art keywords
neural network
convolutional neural
mobile phone
picture
phone screen
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201611154333.5A
Other languages
Chinese (zh)
Other versions
CN106875373A (en
Inventor
宋明黎
高鑫
沈红佳
邱画谋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huizhou Xu Xin Intelligent Technology Co., Ltd.
Original Assignee
Huizhou Xu Xin Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huizhou Xu Xin Intelligent Technology Co Ltd filed Critical Huizhou Xu Xin Intelligent Technology Co Ltd
Priority to CN201611154333.5A priority Critical patent/CN106875373B/en
Publication of CN106875373A publication Critical patent/CN106875373A/en
Application granted granted Critical
Publication of CN106875373B publication Critical patent/CN106875373B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computer systems based on biological models
    • G06N3/02Computer systems based on biological models using neural network models
    • G06N3/04Architectures, e.g. interconnection topology
    • G06N3/0454Architectures, e.g. interconnection topology using a combination of multiple neural nets
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computer systems based on biological models
    • G06N3/02Computer systems based on biological models using neural network models
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30148Semiconductor; IC; Wafer

Abstract

A mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm comprises the following steps: 1) customizing a deep convolutional neural network, and training a neural network for detecting MURA defects of a mobile phone screen by using the existing training data; 2) the pruning operation of the convolutional neural network is carried out by using a self-adaptive template matching method, so that the network scale is reduced, and the operation time of the algorithm is shortened; 3) zooming mobile phone screen pictures shot by a high-resolution camera in different proportions to form a picture pyramid, dividing the pictures into small blocks by using a sliding window method for the pictures of each scale, and sending all the small blocks of pictures into a convolutional neural network together as a group; 4) and selecting all characteristic graphs of the middle layer as response graphs of the defects, and finally obtaining the positions of the MURA defect areas of the mobile phone screen by adopting a threshold segmentation method.

Description

Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm
Technical Field
The invention belongs to the field of target detection and identification, and relates to a method for detecting a specific target from an image, in particular to a method for detecting defects of a mobile phone screen.
Background
The traditional method for detecting the defects of the screen manually has a plurality of defects, and the method cannot meet the requirements of high efficiency and accuracy of the current industrial production at present when the industrial production is developed at a high speed. For mobile phone screen manufacturers, finding an efficient and accurate automatic detection device to replace a manual detection link is an urgent need. With the development of the fields of computer vision, image processing and the like, an automatic detection system based on machine vision becomes a good solution. According to the scheme, the high-resolution industrial camera is used for collecting the mobile phone screen image, and then the image information is processed in real time through the image analysis module, so that whether the mobile phone screen is qualified or not is judged.
The traditional screen defect detection algorithm based on machine vision is mostly designed aiming at one or more types of screen defects, and has no universality, so that a special algorithm needs to be written for special defects. Moire is inevitable on imaging when photographing due to the particularity of the screen, and the Moire problem cannot be well solved by the traditional algorithm. In addition, although the traditional algorithm can detect more obvious linear and point defects, the accuracy rate of the traditional algorithm is low for the imaging light-weight massive MURA defects. Finally, the conventional screen defect detection algorithm needs to adjust a large number of parameters, and especially when the screen product is remodeled, time is wasted by adjusting the large number of parameters. Therefore, it is of great practical value to design an algorithm with good generalization.
In recent years, deep learning methods have had a great impact in the field of computer vision. Deep learning adopts a multilayer network structure to simulate the hierarchical relationship and the transmission mode of a nervous system in a human brain, and the method is widely applied to a plurality of fields of pattern recognition and achieves good results. The method adopts a classification algorithm based on a deep convolutional neural network, uses a preprocessed local image block of a mobile phone screen as the input of a classifier, then extracts a feature map of the convolutional neural network as a detection result, and converts a defect detection problem into an image block classification problem. In addition, compared with the traditional algorithm, the convolutional neural network algorithm has less parameter setting, and the algorithm has good universality, so that the method is particularly suitable for rapid model changing of screen products, the model changing time is shortened, and the production line efficiency is improved.
Disclosure of Invention
The invention provides a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm, aiming at overcoming the defects of a screen defect detection algorithm based on machine vision.
In order to achieve the purpose, the mobile phone screen MURA defect detection method based on the convolutional neural network pruning algorithm comprises the following steps:
1) self-defining a convolutional neural network, and training the network through training data until convergence is achieved and the accuracy is high;
2) pruning the convolutional neural network by a self-adaptive template matching method, and reducing the network scale and network parameters;
3) collecting mobile phone screen picture data, generating a picture pyramid, dividing the picture pyramid into picture blocks, generating test phase data, and sending the test phase data into the pruned convolutional neural network obtained in the step 2) for operation;
4) and taking the sum of the characteristic graphs of the intermediate hidden layers as a response graph, and obtaining and enclosing the final position of the defect by adopting a threshold segmentation method, wherein the method is particularly used for detecting the MURA defect.
The pruning matched with the self-adaptive template in the step 2) is specifically as follows: corresponding the part with response of the characteristic diagram of the hidden layer in the middle of the network to an original image, taking the part as a foreground, taking the rest as a background, and calculating the average brightness difference between the foreground and the background; taking a plurality of feature maps with the largest difference for reservation; the remaining feature maps and their associated convolution kernels are pruned from the network.
The picture pyramid in step 3) is specifically: the original high resolution picture is reduced to pictures of different scales, which are collectively referred to as a set of picture pyramids. The purpose of using the picture pyramid is to detect flaws of different sizes on different scales.
The response graph in step 4) is specifically: the output of the 4 th convolutional layer is adopted, and the sum of the characteristic graphs of the layer of the network after pruning is used as a response graph of the defect.
The invention has the following beneficial effects:
the invention discloses a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm. Based on a convolutional neural network algorithm of deep learning, a self-adaptive template matching method is adopted for network pruning, and a network model and parameters are compressed, so that the effect close to real time is achieved. In addition, the convolution neural network algorithm can automatically learn background texture information, so that moire interference in imaging can be well processed. When the screen product is remodeled, only a small amount of training pictures are needed to carry out network fine tuning, and the accuracy of the network can be guaranteed to reach the standard again.
Compared with the traditional method, the method can more effectively detect the defects with light imaging, such as the nodular MURA defects. In addition, the traditional method needs to set a large number of parameters, and the accuracy of the algorithm can be ensured only by adjusting the parameters every time the model is changed. The neural network algorithm adopted by the invention has good self-adaption and generalization characteristics, can quickly change the screen product, and saves the deployment time of a production line.
Drawings
FIG. 1 is a framework diagram of the adaptive template matching pruning algorithm of the present invention.
FIG. 2 is a graphical representation of the inventive pyramid-to-tile segmentation of a picture.
Detailed Description
The technical solution of the present invention is clearly and completely explained and described below.
The invention provides a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm.
The invention discloses a mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm, which comprises the following steps:
step 1, data acquisition in a training phase: and respectively collecting small blocks containing flaws and normal pictures, and marking (1 represents that the pictures contain flaws, and 0 represents that the pictures are normal). According to the following steps of 9: 1 into a training set and a validation set; and (3) self-defining the convolutional neural network, training the convolutional neural network until convergence by using the training data, and achieving higher classification accuracy on the verification set. By observing the feature map of the hidden layer in the middle of the visual convolutional neural network, whether the network well learns the flaw feature can be analyzed.
And 2, pruning the convolutional neural network by using a self-adaptive template matching method, and reducing the network scale and network parameters. Specifically, all N feature maps on the L convolutional layer will be the ith (c)i < N) the response position learned by the feature map corresponds to the original image, the partial position is used as a foreground template, the rest position of the original image is used as a background template, and the average brightness difference D between the foreground template and the background template is calculatedi. For all DiSorting in descending order, and taking the first K (K is less than N) average brightness differences DiAnd taking the sum of the maximum feature maps as a response map. The remaining feature map and its corresponding convolution kernel are pruned directly as shown in fig. 1. And the whole network is retrained after pruning, so that the network after pruning still has higher accuracy.
Step 3, generating test stage data: and acquiring the picture data of the mobile phone screen, wherein the picture comprises and only comprises the complete mobile phone screen position. And spreading the picture into an image pyramid by using the scaling scales of different proportions for multi-scale detection. For each scale of picture, the picture is divided into picture blocks with fixed sizes, and all the picture blocks are used as a group to serve as primary input data of the convolutional neural network, as shown in fig. 2.
And 4, selecting the sum of all feature maps of the 4 th convolution layer as a response map for each picture block, re-corresponding the response map to the original map, obtaining the final position of the flaw by a threshold segmentation method, and drawing out the flaw.

Claims (3)

1. A mobile phone screen MURA defect detection method based on a convolutional neural network pruning algorithm comprises the following steps:
1) self-defining a convolutional neural network, and training the network through training data until convergence;
2) pruning the convolutional neural network by a self-adaptive template matching method, and reducing the network scale and network parameters; specifically, in all the N feature maps on the lth convolutional layer, the learned response position of the ith (i < N) feature map is corresponding to the original map, the partial position is used as a foreground template, the rest position of the original map is used as a background template, and the average brightness difference Di between the foreground template and the background template is calculated;
sorting all Di in descending order, taking the sum of the first K (K < N) feature graphs with the largest average brightness difference Di as a response graph, and directly pruning the rest feature graphs and the corresponding convolution kernels thereof;
3) collecting mobile phone screen picture data, generating a picture pyramid, dividing the picture pyramid into picture blocks, generating test phase data, and sending the test phase data into the pruned convolutional neural network obtained in the step 2) for operation;
4) and taking the sum of the characteristic graphs of the middle hidden layers as a response graph, and obtaining and enclosing the final position of the defect by adopting a threshold segmentation method, wherein the method is used for detecting the MURA defect.
2. The method of claim 1, wherein: the picture pyramid in step 3) is specifically: reducing the original high-resolution picture into pictures with different scales, wherein the pictures with different scales are collectively called a group of picture pyramids; the purpose of using the picture pyramid is to detect flaws of different sizes on different scales.
3. The method according to claim 1, wherein the response map of step 4) is specifically: the output of the 4 th convolutional layer is adopted, and the sum of the characteristic graphs of the layer of the network after pruning is used as a response graph of the defect.
CN201611154333.5A 2016-12-14 2016-12-14 Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm Active CN106875373B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611154333.5A CN106875373B (en) 2016-12-14 2016-12-14 Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611154333.5A CN106875373B (en) 2016-12-14 2016-12-14 Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm

Publications (2)

Publication Number Publication Date
CN106875373A CN106875373A (en) 2017-06-20
CN106875373B true CN106875373B (en) 2019-12-20

Family

ID=59164615

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611154333.5A Active CN106875373B (en) 2016-12-14 2016-12-14 Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm

Country Status (1)

Country Link
CN (1) CN106875373B (en)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019028960A (en) 2017-08-04 2019-02-21 オムロン株式会社 Image processing system
CN109784129A (en) * 2017-11-14 2019-05-21 北京京东尚科信息技术有限公司 Information output method and device
CN107742294A (en) * 2017-11-27 2018-02-27 歌尔股份有限公司 Scratch detection method, apparatus and electronic equipment
CN108171707A (en) * 2018-01-23 2018-06-15 武汉精测电子集团股份有限公司 A kind of Mura defects level evaluation method and device based on deep learning
CN108802041A (en) * 2018-03-16 2018-11-13 浙江大学 A kind of method that the small sample set of screen detection is quickly remodeled
CN108844966A (en) * 2018-07-09 2018-11-20 广东速美达自动化股份有限公司 A kind of screen detection method and detection system
CN109063834A (en) * 2018-07-12 2018-12-21 浙江工业大学 A kind of neural networks pruning method based on convolution characteristic response figure
CN109509172A (en) * 2018-09-25 2019-03-22 无锡动视宫原科技有限公司 A kind of liquid crystal display flaw detection method and system based on deep learning
CN109840901B (en) * 2019-01-09 2020-08-25 武汉精立电子技术有限公司 Quick judgment method for split screen Mura
CN109829895A (en) * 2019-01-09 2019-05-31 武汉精立电子技术有限公司 A kind of AOI defect inspection method based on GAN
CN109858613B (en) * 2019-01-22 2021-02-19 鹏城实验室 Compression method and system of deep neural network and terminal equipment
CN110097084A (en) * 2019-04-03 2019-08-06 浙江大学 Pass through the knowledge fusion method of projection feature training multitask student network
CN110211121A (en) * 2019-06-10 2019-09-06 北京百度网讯科技有限公司 Method and apparatus for pushing model
CN110532859B (en) * 2019-07-18 2021-01-22 西安电子科技大学 Remote sensing image target detection method based on deep evolution pruning convolution net
CN110728681B (en) * 2019-12-19 2020-05-22 武汉精立电子技术有限公司 Mura defect detection method and device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104036323A (en) * 2014-06-26 2014-09-10 叶茂 Vehicle detection method based on convolutional neural network
CN104850858A (en) * 2015-05-15 2015-08-19 华中科技大学 Injection-molded product defect detection and recognition method
CN104977313A (en) * 2014-04-09 2015-10-14 四川省特种设备检验研究院 Method and device for detecting and identifying X-ray image defects of welding seam
CN105118044A (en) * 2015-06-16 2015-12-02 华南理工大学 Method for automatically detecting defects of wheel-shaped cast product

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104977313A (en) * 2014-04-09 2015-10-14 四川省特种设备检验研究院 Method and device for detecting and identifying X-ray image defects of welding seam
CN104036323A (en) * 2014-06-26 2014-09-10 叶茂 Vehicle detection method based on convolutional neural network
CN104850858A (en) * 2015-05-15 2015-08-19 华中科技大学 Injection-molded product defect detection and recognition method
CN105118044A (en) * 2015-06-16 2015-12-02 华南理工大学 Method for automatically detecting defects of wheel-shaped cast product

Also Published As

Publication number Publication date
CN106875373A (en) 2017-06-20

Similar Documents

Publication Publication Date Title
CN106127204B (en) A kind of multi-direction meter reading Region detection algorithms of full convolutional neural networks
US10803573B2 (en) Method for automated detection of defects in cast wheel products
Kundu et al. No-reference quality assessment of tone-mapped HDR pictures
CN106096561B (en) Infrared pedestrian detection method based on image block deep learning features
CN107145908B (en) A kind of small target detecting method based on R-FCN
CN108961235B (en) Defective insulator identification method based on YOLOv3 network and particle filter algorithm
CN103927534B (en) A kind of online visible detection method of coding character based on convolutional neural networks
WO2019104767A1 (en) Fabric defect detection method based on deep convolutional neural network and visual saliency
CN104050471B (en) Natural scene character detection method and system
CN104217404B (en) Haze sky video image clearness processing method and its device
CN105069472B (en) A kind of vehicle checking method adaptive based on convolutional neural networks
CN103593670B (en) A kind of copper plate/strip detection method of surface flaw based on online limit of sequence learning machine
CN106875381B (en) Mobile phone shell defect detection method based on deep learning
CN107561738B (en) Fast TFT-LCD surface defect detection method based on FCN
JP2014515587A (en) Learning image processing pipelines for digital imaging devices
CN108780508A (en) System and method for normalized image
CN105184271A (en) Automatic vehicle detection method based on deep learning
CN102221559B (en) Online automatic detection method of fabric defects based on machine vision and device thereof
CN105574550A (en) Vehicle identification method and device
CN107016405A (en) A kind of insect image classification method based on classification prediction convolutional neural networks
CN104077577A (en) Trademark detection method based on convolutional neural network
CN104992223A (en) Dense population estimation method based on deep learning
CN108108768B (en) Convolutional neural network-based photovoltaic glass defect classification method and device
CN104182772A (en) Gesture recognition method based on deep learning
CN104517122A (en) Image target recognition method based on optimized convolution architecture

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20180418

Address after: 516006 Huizhou, Zhongkai hi tech Zone, Guangdong, Chen Jiang Street, 36 North (two) fifth.

Applicant after: Huizhou Xu Xin Intelligent Technology Co., Ltd.

Address before: 310027 Hangzhou, Zhejiang Province, Xihu District, Zhejiang Road, No. 38, No.

Applicant before: Zhejiang University

Effective date of registration: 20180418

Address after: 516006 Huizhou, Zhongkai hi tech Zone, Guangdong, Chen Jiang Street, 36 North (two) fifth.

Applicant after: Huizhou Xu Xin Intelligent Technology Co., Ltd.

Address before: 310027 Hangzhou, Zhejiang Province, Xihu District, Zhejiang Road, No. 38, No.

Applicant before: Zhejiang University

GR01 Patent grant
GR01 Patent grant