KR102342337B1 - System and method for diagnosising display panel using deep learning neural network - Google Patents

Abstract

The present technology discloses a display panel defect diagnosis system and method using a deep learning neural network. According to a specific example of the present technology, as the defect diagnosis of the display panel is automatically performed using a deep learning neural network during the manufacturing process of the display panel, the accuracy of the defect diagnosis of the corresponding display panel can be improved, and the human Errors can be prevented, human and cost loss can be prevented, and the accuracy and precision of defect determination as it determines whether the display panel is defective based on multi-channel image information obtained under various angles and lighting conditions The reliability of the judgment result of the display panel as a result of determining the final failure by setting the failure weight by the difference in the reliability of the failure learning value for the failure for each failure type, and assigning the set failure weight to each failure learning value can improve

Description

Display panel defect diagnosis system and method using deep learning neural network

The present invention relates to a system and method for diagnosing a display panel defect using a deep learning neural network, and more particularly, to a display panel defect diagnosis system based on multi-channel image information obtained from multiple angles and/or multi-illumination conditions through at least one camera. technology that can accurately diagnose

In a display device, if there is a disconnection or short circuit of a display signal line or the like or a defect in a pixel during a manufacturing process, they are pre-filtered through a predetermined inspection. Examples of such inspections include an array test, a visual inspection (VI) test, and a module test.

The array test is a test that applies a constant voltage before being separated into individual cells and checks whether the display signal line is disconnected through the presence or absence of an output voltage. Afterwards, it is a test to check whether the display signal line is disconnected by looking at it with the human eye. The module test is a test to determine whether the driving circuit is operating normally after the driving circuit is installed.

These various inspections are performed automatically in some parts, but since it is a display device, a human visual inspection must be finally performed.

For visual inspection, a flicker pattern or an afterimage pattern is displayed separately to finally inspect the state of the product. A test pattern implemented after connecting a computer and a display device through a board and applying a test signal to display the test pattern This is done through the process of checking

Defect inspection using such a test pattern is performed through a human visual inspection, so the inspection speed is low and the yield is low, and since the inspection is performed by a human visual inspection, the accuracy is lower than that of automation.

Accordingly, the applicant of the present application conducted learning based on a deep learning prediction model based on image information of multiple channels obtained from multiple angles and/or multiple lighting using a deep learning neural network to derive a defect type having a defective element and a defective pattern. We would like to propose a method of assigning weights to defective types and then judging the display panel as defective using the weighted failure learning value.

The present invention can improve the accuracy of the defect determination by automatically performing the defect determination of the display panel using a deep learning neural network, prevent human error due to visual inspection, and prevent loss of human and cost The purpose of this is to provide a display panel defect diagnosis system and method using a deep learning neural network that can do this.

Accordingly, the present invention can diagnose a display panel defect in real time, so it is possible to increase the productivity of the display pattern and reduce the defect rate, and thus a display panel defect diagnosis system using a deep learning neural network that can further improve product reliability and to provide a method.

The object of the present invention is not limited to the above-mentioned object, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly known by the embodiments of the present invention. Further, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof indicated in the appended claims.

A display panel defect diagnosis system using a deep learning neural network according to an embodiment of the present invention for achieving the above object,

a preprocessing unit that compresses image information of a plurality of channels obtained from at least one image acquisition device to a low resolution, derives a region of interest, and outputs image data for the derived region of interest;

a plurality of object detection units for detecting a plurality of defective images including defective elements of each channel based on a deep learning neural network with respect to the image data of the region of interest;

For each defective image of each channel, a plurality of defective images for each defective pattern included in each defective image are extracted based on a deep learning neural network, and then a defective type for each extracted defective pattern is classified, and each a plurality of semantic segment classification units for outputting defective type learning values; and

By reflecting the weight of each failure type of each channel optimized for the learning value of each failure type for each channel, the failure learning value for the failure type of each channel is derived, and a corresponding display as the derived failure learning value of each failure type It is characterized in that it includes a defect derivation unit for determining the final defect type of the panel.

Preferably, the failure derivation unit is provided to derive a final failure type by a weighted combination determination method by receiving a learning value and an optimized weight for each failure type of each channel of the plurality of semantic segment classifiers, and is initially derived with the same weight. a weight setting module for optimizing the weight of each failure type of each channel based on the failure learning value for the final failure type; an operation module for multiplying the weight of each failure type of each optimized channel and the learning value of each failure type for each channel, then summing them to output a failure learning value for each failure type; and a final failure determination module that determines a failure type having a maximum failure learning value among failure learning values for each failure type of each channel as a final failure type.

Preferably, the weight setting module may be provided to calculate the detection probability of the final failure type of each channel and optimize the weight of each failure type of each channel with the detection probability of the final failure type.

Preferably, the image information of the plurality of channels is a plurality of image information obtained from a plurality of angles or a plurality of lighting conditions.

A display panel defect diagnosis system using a deep learning neural network according to another embodiment of the present invention,

a preprocessing unit that compresses image information of a plurality of channels obtained from at least one image acquisition device to a low resolution, derives a region of interest, and outputs image data for the derived region of interest;

For the image data of the region of interest, a number of defective images including defective elements of each channel are detected based on a deep learning neural network, a defective pattern included in each defective image of each channel is extracted, and then the extracted defective pattern is a plurality of object detection and semantic segment classifiers for classifying a plurality of defective types for each type and outputting a learning value of each classified failure type; and

By reflecting the weight of each failure type of each channel optimized for the learning value of each failure type for each channel, the failure learning value for the failure type of each channel is derived, and a corresponding display as the derived failure learning value of each failure type Another feature is that it includes a defect derivation unit for determining the final defect type of the panel.

Preferably, the failure derivation unit is provided to derive the final failure type in a weighted combination determination method by receiving the learning values and optimized weights for each failure type of the plurality of semantic segment classification units, and the final failure type derived with the same initial weight a weight setting module for optimizing a weight for each failure type of each channel based on the failure learning value for an arithmetic module for multiplying the weight for each of the optimized failure types and the learning value of each failure type, then summing them to output a failure learning value for each failure type of each channel; and a final failure determination module that determines a failure type having a maximum failure learning value among failure learning values for each failure type of each channel as a final failure type.

Preferably, the weight setting module may be provided to calculate the detection probability of the final failure type of each channel and optimize the weight of each failure type of each channel with the detection probability of the final failure type.

Preferably, the image information of the plurality of channels is a plurality of image information obtained from a plurality of angles or a plurality of lighting conditions.

A display panel defect diagnosis method using a deep learning neural network according to another embodiment of the present invention,

A pre-processing step of compressing image information of a plurality of channels obtained from at least one image acquisition device to a low resolution, then deriving a region of interest and outputting image data for the derived region of interest; a plurality of object detection step of detecting a plurality of defective images including defective elements of each channel based on a deep learning neural network with respect to the image data of the region of interest; For each defective image of each channel, a plurality of defective images for each defective pattern included in each defective image are extracted based on a deep learning neural network, and then a defective type for each extracted defective pattern is classified, and each A plurality of semantic segment classification step of outputting a learning value of a defective type; and by reflecting the weight of each failure type of each channel optimized for the learning value of each failure type for each channel, deriving a failure learning value for the failure type of each channel, and corresponding to the derived failure learning value of each failure type and a defect derivation step of determining a final defect type of the display panel, wherein the defect derivation step includes optimizing the weight of each defect type of each channel based on the failure learning value for the final defect type derived with the same initial weight. ; outputting a failure learning value for each failure type by multiplying the optimized weight of each failure type of each channel and the learning value of each failure type for each channel, then summing them; and determining, as a final rejection type, a failure type having a maximum failure learning value among failure learning values for each failure type of each channel.

According to the present invention, in the manufacturing process of the display panel, by automatically performing the defect diagnosis of the display panel using a deep learning neural network, the accuracy of the defect diagnosis of the corresponding display panel can be improved, and human error caused by the visual inspection can be prevented. and can prevent human and financial loss.

According to the present invention, it is possible to reduce the computational complexity of the deep learning neural network by separately providing an object extraction unit for extracting a predictive failure element for each channel and a semantic segment classification unit for extracting a predictive failure pattern for each channel and deriving a failure type. can

In addition, by determining whether the display panel is defective based on multi-channel image information obtained according to various angles and lighting conditions, the accuracy and precision of the defect determination can be improved.

In addition, the reliability of the judgment result of the display panel can be improved by setting the bad weight by the difference in the reliability of the bad learning value for the bad for each defect type and assigning the set bad weight to each bad learning value to determine the final defect. have.

As the weights and failure weights set according to the present embodiment are set according to the characteristics and application fields of multi-channel image information, compatibility can be obtained not only for the display panel but also for various failure diagnosis systems.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and serve to further understand the technical spirit of the present invention together with the detailed description of the present invention to be described later, so the present invention is a matter described in such drawings should not be construed as being limited only to
1 is a block diagram of a display panel failure diagnosis system according to the present embodiment.
2 is an image diagram illustrating a defective element of the object detection unit according to the present embodiment.
3 is an exemplary diagram of an output image of the semantic segment classifier according to the present embodiment.
4 is a conceptual diagram of a weighted joint determination method applied to the system of the present embodiment.
5 is a detailed configuration diagram of a defect derivation unit of the system according to the present embodiment.
6 is a diagram for explaining a weight optimization process of the system according to the present embodiment.
7 is a block diagram of a display panel failure diagnosis system according to another exemplary embodiment.

Hereinafter, embodiments of the present invention will be described in more detail with reference to the drawings.

Advantages and features of the present invention and methods for achieving them will become apparent with reference to the embodiments described below in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

Terms used in this specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. Also, as used herein, the term “unit” refers to a hardware component such as software, FPGA, or ASIC, and “unit” performs certain roles. However, "part" is not meant to be limited to software or hardware. A “unit” may be configured to reside on an addressable storage medium and may be configured to refresh one or more processors.

Thus, by way of example, “part” includes components such as software components, object-oriented software components, class components and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functionality provided within components and “parts” may be combined into a smaller number of components and “parts” or further divided into additional components and “parts”.

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily implement them. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description will be omitted.

The deep learning neural network applied to this embodiment receives and processes the image data of each channel as input, but creates a predictive bad element and bad type as many times as a preset number (Sequence Length) to build a deep learning prediction model and build deep learning A defective element and a defective pattern are predicted from the multi-channel image data of the input region of interest based on the prediction model. A series of processes for predicting a defective element and a defective pattern based on such a deep learning prediction model can be understood by those of ordinary skill in the art related to an embodiment of the present invention.

This embodiment is provided to automatically determine a defect of the display panel based on a deep learning neural network. That is, for the image information of multiple channels obtained from multiple angles or multiple illuminations, the defective elements and defective patterns included in the image data of each channel are separately extracted based on the deep learning neural network, and the final defect detection probability with the previously extracted defective pattern. gives the weight of the defect type to a number of defect patterns included in the defective image of each channel, and determines the final defect type of the display panel with the given weight of each defect type in each channel and the learning value of each defect type in each channel. detect

In addition, the present embodiment calculates the detection probability of the final failure type of each channel and optimizes the weight of each failure type of each channel with the detection probability of the final failure type.

1 is a view showing a display panel failure diagnosis system using a deep learning neural network according to an embodiment of the present invention. Referring to FIG. 1 , a preprocessor 100, a deep learning neural network (DNN) prediction model 200, an object It may include a detection unit 300 , a semantic segment classification unit 400 , and a defect detection unit 500 .

The pre-processing unit 100 compresses image information of multiple channels acquired according to various angles and/or multiple illumination in at least one image acquisition device to a low resolution, and then derives a region of interest and image data for the derived region of interest to output

That is, the preprocessor 100 compresses and converts the image or sensor map data obtained from the multi-channel image acquisition device into a low-capacity compression format, and an image of an unnecessary region except for the object to be detected in the cropping block of the region of interest. remove the part

In this embodiment, a multiplexer that multiplexes image data of different input channels output from the preprocessor 100 and transmits the multiplexer to the object detection unit 300 may be additionally provided, which is a technical field related to an embodiment of the present invention Anyone with ordinary knowledge in

Then, the image data of the region of interest of the preprocessor 100 is transmitted to the object detection unit 300 based on the deep learning neural network. Here, the deep learning prediction model 200 is formed with a plurality of defective types having a plurality of defective elements and defective patterns included in the image data obtained for each channel by a preset number of times (Sequence Length) for the image data for the region of interest. is built

Accordingly, the object detection unit 300 performs a function of detecting all defective elements based on the image data of the region of interest from the built-in deep learning prediction model 200, and, for example, the object detection unit 300 is A defective image including a plurality of defective elements is detected from the image data of the ROI of each channel by using the running prediction model 200 . That is, a plurality of defective elements are detected in the image data of one channel. In this case, a plurality of defective images including each defective element are extracted in the form of a rectangular box block centered on the ROI.

That is, as shown in FIG. 1 , a plurality of defective images k having defective elements included in the image data of each channel output from the object detection unit 300 are defective image 1, defective image k, and Bad image K is included, bad image m of any channel m includes bad image 1, bad image m, and bad image M, and bad image h of any channel h is bad image 1, bad image h, and Bad image H is included.

The bad image k, the bad image m, and the bad image h of each of these channels k, m, and h are transmitted to the semantic segment classification unit 400 .

The semantic segment classification unit 400 uses the deep learning prediction model 200 constructed with the bad image k, the bad image m, and the bad image h of each channel k, m, and h as inputs to the multiple included in each bad image. to extract the bad pattern of At this time, since a plurality of defective patterns are included in each defective image of each channel, the semantic segment classification unit 400 detects a defect type for each extracted defective pattern.

For example, when there is no defect, defective image k, defective image m, and defective image h of each channel k, m, and h are all 0.

Meanwhile, the semantic segment classification unit 400 learns based on the deep learning prediction model 200 by inputting each defective image having a defective element of each channel as an input, and a plurality of defective patterns included in each defective image according to the learning result. to derive the defect type for each extracted defect pattern.

In this case, the defect types are scratches, dents, holes, NAs, and the like, and a learning value for each of these defect types is output from the semantic segment classification unit 400 .

In addition, the learning values for a plurality of defect types for each channel of the semantic segment classifying unit 400 are transmitted to the defect deriving unit 500 .

That is, the defective image detected for each channel may be prominently expressed in a specific channel or may not be expressed in other channels according to different angles and/or lighting conditions. Accordingly, the failure deriving unit 500 determines the final failure type of the corresponding display panel by reflecting the plurality of weights for each channel set in the plurality of learning values for each channel.

That is, the rejection derivation unit 500 determines the final rejection type in a weighted combination determination method based on the learning values for a plurality of rejection types of each channel and set weights.

The weighted combining method failure derivation unit 500 initially gives the same weight to the failure learning value for each channel, then calculates the probability that the final failure type for each channel is detected, and uses the calculated probability to give a plurality of failure types for each channel. Optimize each weight for The algorithm for optimizing such a weight may be provided with various algorithms such as an automatic tuning method (Grid Search), but is not limited thereto, and may be implemented in various ways within the scope of known technology.

2 and 3 are diagrams illustrating defective images output from the object detection unit 300 and the semantic segment classifying unit 400 , and as shown in FIGS. 2 and 3 , the defective images output from the object detection unit 300 . It can be confirmed that includes the defective element and the defective image output from the semantic segment classifying unit 400 includes the defective pattern.

In this embodiment, it is described that the object detection unit 300 and the semantic segment classification unit 400 are separately provided inside the deep learning neural network as an example, thereby reducing the computational complexity of the deep learning neural network.

Meanwhile, FIG. 4 is a conceptual diagram of a weighted coupling determination method of the defect deriving unit 500. Referring to FIG. 4 , the defect deriving unit 500 is a semantic segment classifying unit 400 for each channel type of failure. The failure learning value is multiplied by a preset weight (w1, w2, w3, w4) and then synthesized. Among the failure learning values for the synthesized failure type, a failure type exceeding a predetermined threshold is determined as the final failure type of the corresponding display panel. do.

FIG. 5 is a view showing a detailed configuration of the defect deriving unit 500 shown in FIG. 2 . Referring to FIG. 5 , a structure in which weights are combined in the present embodiment can be confirmed, and thus the defect deriving unit 500 is weighted It may include a setting module 510 , an operation module 520 , and a final failure determination module 530 .

The weight setting module 510 initially gives the same weight to the learning values for each defective type, then calculates the probability that the final defective type for each channel is detected, and uses the calculated probability to learn the learning values for a plurality of defective types for each channel. Optimize each weight.

6 is a diagram showing a learning value for a failure type for each channel derived with an optimized weight in the weight setting module 510. Referring to FIG. 6, the average of the conventional method is multiplied by a weight obtained by the grid search method. When averaging, it can be confirmed that the accuracy of the learning value for the defect type is higher. Accordingly, the weight setting module 510 learns the defect type (dent, scratch, hole, NA) for each channel derived with the same weight. The weight is optimized based on the ratio of values, and the optimized weight is provided to the operation module 520 .

On the other hand, the operation module 520 multiplies each learning value for a plurality of defective types for each channel and each weight for a plurality of defective types for each channel, and then sums them to obtain a failure learning value for each defective type for each channel. print out

과 1번째 채널, j 번째 채널, 및 J번째 채널의 예측 불량 유형 1에 대한 가중치

의 곱을 연산하는 예측 불량 유형 1에 대한 불량 학습값

을 출력한다.That is, the operation module 520 is a learning value for the 1st channel, the j-th channel, and the prediction failure type 1 of the J-th channel.

and weights for the 1st channel, the jth channel, and the Jth channel for bad prediction type 1

Bad learning value for predictive bad type 1 that computes the product of

to output

과 1번째 채널, j 번째 채널, 및 J번째 채널의 불량 유형 n에 대한 가중치

의 곱을 연산하는 예측 불량 유형 n에 대한 불량 학습값

을 출력한다.In addition, the operation module 520 is a learning value for the failure type n of the first channel, the j-th channel, and the J-th channel.

and weights for the 1st channel, the jth channel, and the defect type n of the Jth channel

Bad learning value for predictive bad type n that computes the product of

to output

과 은 1번째 채널, j 번째 채널, 및 J번째 채널의 불량 유형 N에 대한 가중치

의 곱을 연산하는 예측 불량 유형 N에 대한 학습값

을 출력한다. 전술한 다수의 예측 불량유형 1, n, N 에 대한 불량 학습값

,

,

을 도출하는 일련의 과정은 각 채널 별 모든 예측 불량 유형에 도달할 때까지 반복 수행된다. And the operation module 520 is a learning value for the failure type N of the first channel, the j-th channel, and the J-th channel

and are the weights for the 1st channel, the jth channel, and the defect type N of the Jth channel.

Learning value for bad prediction type N that computes the product of

to output Bad learning values for the above-described multiple prediction rejection types 1, n, and N

,

,

A series of processes for deriving ? is iteratively performed until all prediction failure types for each channel are reached.

Accordingly, the failure learning value for each prediction failure type of the operation module 520 is transmitted to the final failure deriving module 530 .

,

,

중 최대 학습값을 최종 불량 학습값

으로 도출하고 도출된 최종 불량 학습값

과 매칭되는 예측 불량 유형을 해당 디스플레이 패널의 최종 불량으로 판정한다. The final failure derivation module 530 is a failure learning value for each predicted failure type.

,

,

The maximum learning value among the final bad learning values

The final bad learning value derived from

The predicted defect type matching with the . is determined as the final defect of the corresponding display panel.

For example, it is assumed that there are two defective types of scratch and dent among the defect types and the semantic segment classifier 400, and the ratio of the learning value of the scratch and the dent output from the first semantic segment classifier 400 is 0.8: Assuming that the ratio of the learning value of the scratch and the dent output from the second semantic segment classification unit 400 is 0.2: 0.8, and the ratio of the weight of the optimized scratch and the dent is 0.9: 0.1, the final defect derivation module 530 determines the final defect type as scratch.

Accordingly, in the manufacturing process of the display panel according to the present embodiment, by using a deep learning neural network to automate the diagnosis of display panel defects, the accuracy of defect diagnosis can be improved, and human error caused by visual inspection can be prevented. , human and cost loss can be prevented, and as a semantic segment classifier that extracts a bad element for each channel and a semantic segment classifier that extracts a bad pattern for each channel and derives a bad type is provided as a separate device, deep learning The computational complexity of the neural network can be reduced, and by analyzing the bad learning value for each defect type, the bad weight is set as a difference in the reliability of the bad judgment result, and the set bad weight is given to the learning value of each bad to determine the final bad. According to the determination, reliability according to the determination result of the display panel may be improved.

By setting the weights set according to the present embodiment differently according to the characteristics and application fields of the channel image, compatibility can be obtained not only for the display panel but also for the failure diagnosis system.

According to another embodiment of the present invention, there is provided a display panel defect diagnosis system in which an object detection unit and a semantic segment classification unit included in a deep learning neural network are integrated into one, so that it is possible to automatically accurately determine the best defect type of the display panel.

, a display panel defect diagnosis system in which an object detection unit and a semantic segment classification unit included in a deep learning neural network are integrated into one, it is possible to accurately determine the optimal type of display panel automatically.

7 is a view showing the configuration of a display panel failure diagnosis system using a deep learning neural network according to another embodiment of the present invention, the preprocessor 10, the DNN prediction model 20, the object and semantic segment classification unit 30 , and a defect derivation unit 50, and the function of each configuration is the above-described preprocessor 100, DNN prediction model 200, object detection unit 300, semantic segment classification unit 400, and defect derivation unit 500 ), so a redundant description of the function will be omitted below.

The preprocessor 10 compresses image information of a plurality of channels obtained by the image acquisition device to a low resolution, then derives a region of interest, outputs image data for the derived region of interest, and performs the pre-processing of the above-described preprocessor 100 . perform the function

Then, the object detection and semantic segment classification unit 30 extracts a bad image for each channel including a bad element of each channel from the DNN prediction model 20 built in advance with respect to the image data of the region of interest, and then extracts each extracted channel. Extracts the defective pattern included in the defective image of each channel for the defective image of 300) and the semantic segment classifier 400 classify a plurality of defective types including a defective image including a defective element and a defective pattern included in the defective image from the image data for each channel of each channel, and learn about each classified defective type It performs the function of deriving a value.

Then, each learning value for a plurality of defect types of each defect pattern is transmitted to the defect deriving unit 40 .

The failure derivation unit 50 reflects the weight for each failure type optimized to the received learning value for each failure type, derives a failure learning value for the failure type of each channel, and displays the corresponding failure learning value as the derived failure learning value. It determines the final defect type of the panel, and performs a function of determining the final defect type of the above-described defect derivation unit 400 .

Accordingly, according to this embodiment, by using a deep learning neural network in the manufacturing process of the display panel to automate the diagnosis of defects in the display panel, it is possible to improve the accuracy in diagnosis of defects and to prevent human errors due to visual inspection. In addition, it is possible to prevent human and cost loss, analyze the bad learning value for each defect by type of defect, set the defect weight as a difference in the reliability of the defect determination result, and assign the set defect weight to the learning value of each defect. As the final defect is determined, reliability according to the determination result of the display panel may be improved.

Meanwhile, according to another embodiment of the present invention, image information of a plurality of channels acquired from at least one image acquisition device is compressed to a low resolution, and then a preprocessing method for deriving a region of interest and outputting image data for the derived region of interest step; a plurality of object detection step of detecting a plurality of defective images including defective elements of each channel based on a deep learning neural network with respect to the image data of the region of interest; For each defective image of each channel, a plurality of defective images for each defective pattern included in each defective image are extracted based on a deep learning neural network, and then a defective type for each extracted defective pattern is classified, and each A plurality of semantic segment classification step of outputting a learning value of a defective type; and by reflecting the weight of each failure type of each channel optimized for the learning value of each failure type for each channel, deriving a failure learning value for the failure type of each channel, and corresponding to the derived failure learning value of each failure type and a defect derivation step of determining a final defect type of the display panel, wherein the defect derivation step includes optimizing the weight of each defect type of each channel based on the failure learning value for the final defect type derived with the same initial weight. ; outputting a failure learning value for each failure type by multiplying the optimized weight of each failure type of each channel and the learning value of each failure type for each channel, then summing them; and determining, as a final failure type, a failure type having a maximum failure learning value among failure learning values for each failure type of each channel, wherein each step of the display panel failure diagnosis method using the deep learning neural network is described above. This function is performed by one preprocessor 100 , the DNN prediction module 200 , the object detection unit 300 , the semantic segment classification unit 400 , and the defect derivation unit 500 , and detailed references are omitted.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the following claims as well as the claims and equivalents.

In the manufacturing process of the display panel, by using a deep learning neural network to automate the diagnosis of display panel defects, the accuracy of defect diagnosis can be improved, human error caused by visual inspection can be prevented, and human and cost losses can be achieved. can prevent and reduce the computational complexity of deep learning neural networks as the object extraction unit that extracts the bad elements for each channel and the semantic segment classifier that extracts the bad pattern for each channel and derives the bad type are provided as separate devices By analyzing the bad learning value for the defect for each type of defect, the defect weight is set as the difference in the reliability of the defect determination result, and the set defect weight is given to the learning value of each defect to determine the final defect. The accuracy and reliability of the display panel defect diagnosis system using a deep learning neural network that can improve the reliability according to the judgment result, and furthermore, in terms of performance efficiency, can bring a very big improvement, and the defect of the device in various applications It is an invention with industrial applicability because the possibility of commercialization or business of the diagnostic system is sufficient, and it is a degree that can be clearly implemented in reality.

Claims (11)

a preprocessing unit that compresses image information of a plurality of channels obtained from at least one image acquisition device to a low resolution, derives a region of interest, and outputs image data for the derived region of interest;
a plurality of object detectors for detecting a plurality of defective images including defective elements of each channel based on a deep learning neural network through a deep learning prediction model for the image data of the region of interest;
For each defective image of each channel, a plurality of defective images for each defective pattern included in each defective image are extracted based on a deep learning neural network, and then a defective type for each extracted defective pattern is classified, and each a plurality of semantic segment classification units for outputting defective type learning values; and
By reflecting the weights for each defect type of each channel optimized for the learning value of each defect type of each channel, the bad learning value for the bad type of each channel is derived, and corresponding to the derived bad learning value for each bad type It includes a defect derivation unit for determining the final defect type of the display panel,
The preprocessor is
The image information of multiple channels is compressed into a low-resolution compression format, and the region of interest is extracted by removing unnecessary image regions except for the object to be detected from the cropping block of the region of interest.
A multiplexer for multiplexing image data of a plurality of channels and transmitting the multiplexer to the object detection unit is further provided,
The object detection unit,
The deep learning prediction model is constructed by detecting a plurality of defective elements including image data obtained for each channel as much as a preset number of times (Squence Length) for the extracted image data of the region of interest and a plurality of defective types having a defective pattern. A display panel defect diagnosis system using a deep learning neural network, characterized in that it is provided to do so. According to claim 1, wherein the defect derivation unit,
Display panel failure diagnosis using a deep learning neural network, characterized in that the plurality of semantic segment classification units are provided with a learning value and an optimized weight for each failure type of each channel, and are provided to derive the final failure type using a weighted combination determination method system. The method of claim 2, wherein the defect derivation unit,
a weight setting module for optimizing weights for each defective type of each channel based on the failure learning value for the final defective type derived with the same initial weight;
an arithmetic module for multiplying the optimized weight for each failure type of each channel and a learning value for each failure type of each channel, then summing them to output a failure learning value for each failure type; and
A display panel failure diagnosis system using a deep learning neural network, comprising a final failure determination module for judging a failure type having a maximum failure learning value among failure learning values for each failure type of each channel as a final failure type. The method of claim 3, wherein the weight setting module comprises:
A display panel defect diagnosis system using a deep learning neural network, characterized in that it is provided to calculate the detection probability of the final defect type of each channel and optimize the weight of each defect type of each channel with the detection probability of the final defect type. [Claim 5] The display panel defect using a deep learning neural network according to any one of claims 1 to 4, wherein the image information of the plurality of channels is a plurality of image information obtained from a plurality of angles or a plurality of lighting conditions. diagnostic system. a preprocessing unit that compresses image information of a plurality of channels obtained from at least one image acquisition device to a low resolution, derives a region of interest, and outputs image data for the derived region of interest;
For the image data of the region of interest, a number of defective images including defective elements of each channel are detected based on a deep learning neural network through a deep learning prediction model, and a defective pattern included in each defective image of each channel is extracted and then extracted a plurality of object detection and semantic segment classification unit for classifying a plurality of defective types for each defective pattern and outputting a learning value for each classified defective type; and
By reflecting the weights for each defective type of each channel optimized for the learning value for each defective type for each channel, a failure learning value for each failure type of each channel is derived, and a failure learning value for each derived failure type is reflected. Including a defect derivation unit that determines the final defect type of the display panel with
The preprocessor is
The image information of multiple channels is compressed into a low-resolution compression format, and the region of interest is extracted by removing unnecessary image regions except for the object to be detected from the cropping block of the region of interest.
The image data of a plurality of channels is multiplexed and transmitted to the object detection and semantic segment classification unit,
The object detection and semantic segment classification unit
The deep learning prediction model is constructed by detecting a plurality of defective elements including image data acquired for each channel as much as a preset number of times (Squence Length) for the extracted image data of the region of interest and a plurality of defective types having a defective pattern. A display panel defect diagnosis system using a deep learning neural network, characterized in that it is provided to do so. The method of claim 6, wherein the defect derivation unit,
A display panel defect diagnosis system using a deep learning neural network, characterized in that it is provided with a learning value and an optimized weight for each defect type of a plurality of semantic segment classification units to derive a final defect type by a weighted combination determination method. The method of claim 7, wherein the defect derivation unit,
a weight setting module for optimizing the weight of each failure type of each channel based on the failure learning value for the final failure type derived with the same initial weight;
an arithmetic module for multiplying the weight of each optimized failure type and a learning value for each failure type, then summing them to output a failure learning value for each failure type of each channel; and
A display panel failure diagnosis system using a deep learning neural network, comprising a final failure determination module for judging a failure type having a maximum failure learning value among failure learning values for each failure type of each channel as a final failure type. The method of claim 8, wherein the weight setting module
A display panel defect diagnosis system using a deep learning neural network, characterized in that it is provided to calculate the detection probability of the final defect type of each channel and optimize the weight of each defect type of each channel with the detection probability of the final defect type. The display panel defect using a deep learning neural network according to any one of claims 6 to 9, wherein the image information of the plurality of channels is a plurality of image information obtained from a plurality of angles or a plurality of lighting conditions. diagnostic system. A pre-processing step of compressing image information of a plurality of channels obtained from at least one image acquisition device to a low resolution, then deriving a region of interest and outputting image data for the derived region of interest;
a plurality of object detection step of detecting a plurality of defective images including defective elements of each channel based on a deep learning neural network through a deep learning prediction model for the image data of the region of interest;
For each defective image of each channel, a plurality of defective images for each defective pattern included in each defective image are extracted based on a deep learning neural network, and then the defective type of each extracted defective pattern is classified, and each classified defective A plurality of semantic segment classification step of outputting a learning value for the type; and
By reflecting the weights for each defective type of each channel optimized for the learning value for each defective type for each channel, the failure learning value for the failure type of each channel is derived, and the failure learning value for each of the derived failure types is reflected. including a defect derivation step of determining the final defect type of the corresponding display panel with
The pre-processing step is
The image information of multiple channels is compressed into a low-resolution compression format, and unnecessary image regions except for the object to be detected are removed from the cropping block of the region of interest to extract the region of interest and multiplex the image data of multiple channels. become,
The step of detecting the plurality of objects is
The deep learning of a plurality of defective types having a plurality of defective elements and defective patterns including image data obtained for each channel by successively performing the multiplexed image data of the region of interest in the preprocessor step for a set number of times (Squence Length) It is provided to build a predictive model,
The step of deriving the defect is
optimizing the weight of each failure type of each channel based on the failure learning value for the final failure type derived with the same initial weight;
outputting a failure learning value for each failure type by multiplying the optimized weight for each failure type of each channel and a learning value for each failure type for each channel, then summing them; and
A method for diagnosing a display panel failure using a deep learning neural network, comprising the step of determining a failure type having the maximum failure learning value among failure learning values for each failure type of each channel as the final failure type.

Images