CN109035226B

CN109035226B - Mura defect detection method based on LSTM model

Info

Publication number: CN109035226B
Application number: CN201810763443.4A
Authority: CN
Inventors: 陈武; 张胜森; 郑增强
Original assignee: Wuhan Jingce Electronic Group Co Ltd
Current assignee: Wuhan Jingce Electronic Group Co Ltd
Priority date: 2018-07-12
Filing date: 2018-07-12
Publication date: 2021-11-23
Anticipated expiration: 2038-07-12
Also published as: CN109035226A

Abstract

The invention discloses a Mura defect detection method based on an LSTM model, which comprises the following steps: 1) equally dividing the panel into m areas, and collecting qualified product samples and defective product samples of the m areas; 2) forming a sample sequence by the qualified product sample and the defective product sample of each region, and establishing an LSTM model for each sequence; 3) respectively carrying out LSTM model training on the sample sequences of the m areas; 4) inputting the pictures of m areas of the panel to be detected into corresponding LSTM models, and outputting qualified or defective classification results by the m LSTM models; 5) and (5) integrating the classification results of the m LSTM models, and judging whether the classification result of any region is a defect. According to the method, the defect detection model is established through the deep neural network, so that the limitations of background segmentation and parameter extraction of the traditional algorithm are effectively avoided, the labor cost is reduced, the Mura defect can be rapidly output, the defect is accurately detected, and the accuracy and the efficiency of the Mura defect detection are improved.

Description

Mura defect detection method based on LSTM model

Technical Field

The invention relates to the field of automatic defect detection of panels, in particular to deep learning and image processing, and specifically relates to a Mura defect detection method based on an LSTM model.

Background

In AOI (Automatic optical Inspection) defect detection, Mura defect detection is of great importance, and the final judgment results of defect reduction and the like of the LCD panel are directly influenced. The Mura defect has the characteristics of irregular shape, uneven size, unfixed position, uneven local or overall brightness and low contrast, and is limited by human eye perception and subjective factors, so that the Mura defect is difficult to quickly and accurately detect and evaluate quickly and accurately, and as can be seen from figure 1, the contrast of the Mura defect relative to the background is very low, the edge is fuzzy and is difficult to identify by naked eyes. Meanwhile, with the generation of new processes, various unpredictable Mura defects are also generated. At present, a stable and reliable detection system is not formed in the traditional algorithm, and a more stable and higher-precision Mura defect detection method needs to be found.

As shown in fig. 2, the conventional Mura defect detection process is generally divided into 4 processes of image acquisition, brightness correction, image enhancement and defect detection. Along with the development trend of large picture, light weight and high resolution of LCD, the occurrence probability of Mura defect is greatly increased, and the environment is more and more complex, and the repeated texture background, the non-uniform whole brightness, the fuzzy edge of the defect and the lower contrast ratio make the traditional brightness correction and picture enhancement algorithm in image processing difficult to directly detect the Mura defect. Therefore, a new set of methods must be developed to meet the existing industry requirements.

Disclosure of Invention

Aiming at the defects of the prior art, the invention aims to provide a Mura defect detection method based on an LSTM model, which establishes a defect detection model through a deep neural network, effectively avoids the limitations of background segmentation and parameter extraction of the traditional algorithm, reduces the labor cost, can quickly output Mura defects, accurately detects the defects and improves the accuracy and efficiency of Mura defect detection.

In order to achieve the above purpose, the invention provides a Mura defect detection method based on an LSTM model, which is characterized by comprising the following steps:

1) equally dividing the panel into m areas, and collecting qualified product samples and defective product samples of the m areas;

2) forming a sample sequence by the qualified product sample and the defective product sample of each region, and establishing an LSTM model for each sequence;

3) respectively carrying out LSTM model training on the sample sequences of the m regions, and extracting qualified product sample characteristics and defective product sample characteristics through deep learning by the LSTM model;

4) inputting the pictures of m areas of the panel to be detected into corresponding LSTM models, and outputting qualified or defective classification results by the m LSTM models;

5) and (4) integrating the classification results of the m LSTM models, judging whether the classification result of any region is a defect, if so, judging that the panel to be detected is a Mura defect, and positioning the position of the defect, otherwise, judging that the panel to be detected is a qualified product.

Preferably, the ratio of the qualified product sample to the defective product sample is (0.8-1.2): (0.8 to 1.2).

Preferably, m ═ a, a is a natural number. Most preferably, a is 10.

Preferably, the panels to be tested are n panel products of the same batch.

Preferably, the panels of the qualified product samples and the defective product samples and the panel to be detected are panel products of the same model.

The invention provides a complete Mura defect detection method, which forms a sample sequence according to panel production time, carries out memory training through an LSTM network model, and can quickly screen defect-removed areas and normal areas after the network model trains the sample sequence, thereby carrying out defect detection. By the method, the Mura defect of the panel can be accurately and specifically detected, the limitation of the traditional algorithm can be successfully avoided, and meanwhile, the labor cost is greatly reduced.

The invention has the advantages that:

1) the panel area is divided into a plurality of sub-areas, so that the sequence information of each sub-area is improved, and the LSTM detection is facilitated;

2) based on the LSTM neural network, a large number of samples are learned, the interference of a complex background is reduced, and the accuracy of Mura detection is improved;

3) the detection method is suitable for online detection, and replaces the detection process of personnel, so that the labor cost and the time cost are reduced.

Drawings

Fig. 1 is an LCD sample view.

Fig. 2 is a flowchart of conventional Mura detection.

FIG. 3 is a schematic diagram of the LSTM model.

FIG. 4 is a flow chart of the Mura defect detection method based on the LSTM model.

FIG. 5 is a schematic diagram of a sample sequence structure.

Detailed Description

The invention is described in further detail below with reference to the figures and specific embodiments.

The present invention detects Mura defects using the LSTM model, which is a specific form of RNN (Recurrent neural network) that is a generic term for a series of neural networks that can process sequence data.

At present, panel products are produced according to batches, all products in the same batch can be regarded as a sequence sample, the difference between normal samples is small, the difference between every two adjacent samples in a sequence is smaller, and once the obvious difference occurs, Mura defects are generated with high probability.

The LSTM model avoids long-term dependency problems through deliberate design. Keeping in mind that long-term information is in practice the default behavior of LSTM. The key to the LSTM model is the cell state, with horizontal lines running across the top of the graph. The unit state is similar to a conveyor belt. Run directly on the whole chain with only a few linear interactions. It is easy for information to remain unchanged in the stream above. The LSTM model has the ability to remove or add information to the cell state through a carefully designed structure called a "gate". The gate is a method for selectively passing information, and comprises a Sigmoid neural network layer and a pointwise multiplication operation. The Sigmoid layer outputs a value between 0 and 1 describing how much of each part can pass through. 0 means that no amount is allowed to pass, and 1 means that any amount is allowed to pass. The LSTM model has three gates to protect and control the cell states, and its structural principle is shown in fig. 3.

The LSTM model established by the invention finds the mutual relation among normal samples by learning a sequence of panel samples, controls the information needing to be reserved or deleted by a forgetting gate, and correspondingly outputs the normal samples and Mura defect samples by an output gate.

As shown in fig. 4, the specific steps of the Mura defect detection method based on the LSTM model provided by the present invention include:

1) and equally dividing the panel into m-a areas, and collecting qualified product samples and defective product samples of the m areas. In this embodiment, m is 100 and a is 10. Dividing each panel sample into 100 areas of 10 × 10; and labeled 1-100, as shown in FIG. 5-a, the number of panels in the same batch is labeled as n.

2) The non-defective samples and defective samples of each region constitute a sample sequence, as shown in fig. 5-b, and each sequence establishes an LSTM model.

3) And respectively carrying out LSTM model training on the sample sequences of the m regions, and extracting qualified product sample characteristics and defective product sample characteristics by the LSTM model through deep learning. The training samples are generally 500-2000 pieces under normal conditions, and the ratio of qualified product samples to defective product samples is about 1: 1.

And (3) respectively carrying out LSTM model training on 100 sequence samples, wherein each sample controls input, forgetting and output through a gate. The gate is actually a fully connected layer, the input is the eigenvector x, and the output is a real vector between 0 and 1. The formula is as follows:

g(x)＝σ(Wx+b)

in the formula, W: the weight matrix controls whether to reserve or forget by setting 0 or 1; b: the bias term of the gate, σ denotes the sigmoid function, is used as a threshold function for the neural network, maps variables between 0,1,

the LSTM calculates the unit state of the current input according to the last output and the current input by controlling an input gate according to the information of the learned sample sequence, and the input gate determines how much the input of the network is stored in the unit state at the current moment; the control of the forgetting door determines how much the unit state at the previous moment is reserved to the current moment; how much of the output gate control unit state is output to the current output value of the LSTM. Because the samples in the same batch have obvious sequence information, the LSTM can learn the previous useful information, and finally, the output gate can control how much information becomes the output value of the current LSTM; the LSTM establishes the feature rules of qualified products and defective products according to the features extracted by the LSTM, and then controls the retention quantity and the output quantity of the features extracted by each sample through three gates to establish a better model and realize Mura defect detection.

4) And inputting the pictures of the m areas of the panel to be detected into the corresponding LSTM models, and outputting qualified or defective classification results by the m LSTM models. In the LSTM learning process, 100 LSTM networks are generated, and when performing a test, a panel area is also divided into 10 × 10(100 areas), and each area is detected by using a corresponding model.

5) And (4) integrating the classification results of the m LSTM models, judging whether the classification result of any region is a defect, if so, judging that the panel to be detected is a Mura defect, and positioning the position of the defect, otherwise, judging that the panel to be detected is a qualified product. If one of the 100 output results is a Mura defect, the panel is judged as the Mura defect, and the defect position can be accurately positioned.

The foregoing is only a preferred embodiment of this invention and it should be noted that modifications can be devised by those skilled in the art without departing from the principles of the invention and these modifications should also be considered as within the scope of the invention.

Details not described in this specification are within the skill of the art that are well known to those skilled in the art.

Claims

1. A Mura defect detection method based on an LSTM model is characterized in that: the method comprises the following steps:

1) equally dividing the panel into m areas, and collecting qualified product samples and defective product samples of the m areas; the panel of the qualified product sample and the panel of the defective product sample and the panel to be detected are panel products of the same type;

2. The method of claim 1 for detecting Mura defect based on LSTM model, wherein: the proportion of the qualified product sample to the defective product sample is (0.8-1.2): (0.8 to 1.2).

3. The method of claim 1 for detecting Mura defect based on LSTM model, wherein: m is a, a is a natural number.

4. The method of claim 1 for detecting Mura defect based on LSTM model, wherein: the qualified product sample and the defective product sample are n panel products in the same batch.

5. The method of claim 1 for detecting Mura defect based on LSTM model, wherein: a is 10.