CN113781429A

CN113781429A - Defect classification method and device for liquid crystal panel, electronic equipment and storage medium

Info

Publication number: CN113781429A
Application number: CN202111057409.3A
Authority: CN
Inventors: 马政; 张伟
Original assignee: Shenzhen Sensetime Technology Co Ltd
Current assignee: Shenzhen Sensetime Technology Co Ltd
Priority date: 2021-09-09
Filing date: 2021-09-09
Publication date: 2021-12-10

Abstract

The disclosure relates to a defect classification method and apparatus for a liquid crystal panel, an electronic device, and a storage medium. The method comprises the following steps: acquiring an image to be classified of the liquid crystal panel; performing defect classification on the image to be classified to obtain a first defect classification result corresponding to the image to be classified; in response to that the first defect classification result at least indicates that the image to be classified belongs to a class to be rechecked, obtaining a gray value of a first region and a gray value of a second region of the image to be classified based on the image to be classified, wherein the first region contains a defect region, and the second region does not contain the defect region; and determining a second defect classification result of the image to be classified according to the gray value of the first region and the gray value of the second region.

Description

Defect classification method and device for liquid crystal panel, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to a method and an apparatus for classifying defects of a liquid crystal panel, an electronic device, and a storage medium.

Background

A Liquid Crystal panel (LCD) is a material that determines the brightness, contrast, color, viewing angle, etc. of a Liquid Crystal Display. The quality of the liquid crystal panel and the quality of the technology are related to the overall performance of the liquid crystal display. The intelligent industrial quality inspection is an important problem in the field of computer vision and industrial quality inspection and an important development direction of industrial 4.0 technical strategy. In the information age, it is of great significance to accurately classify defects of liquid crystal panels.

Disclosure of Invention

The present disclosure provides a defect classification technical scheme of a liquid crystal panel.

According to an aspect of the present disclosure, there is provided a defect classification method of a liquid crystal panel, including:

acquiring an image to be classified of the liquid crystal panel;

performing defect classification on the image to be classified to obtain a first defect classification result corresponding to the image to be classified;

in response to that the first defect classification result at least indicates that the image to be classified belongs to a class to be rechecked, obtaining a gray value of a first region and a gray value of a second region of the image to be classified based on the image to be classified, wherein the first region contains a defect region, and the second region does not contain the defect region;

and determining a second defect classification result of the image to be classified according to the gray value of the first region and the gray value of the second region.

Acquiring an image to be classified of a liquid crystal panel, performing defect classification on the image to be classified to obtain a first defect classification result corresponding to the image to be classified, responding to the first defect classification result and at least indicating that the image to be classified belongs to a class to be rechecked, acquiring a gray value of a first region containing a defect region and a gray value of a second region not containing the defect region based on the image to be classified, and determining a second defect classification result of the image to be classified according to the gray value of the first region and the gray value of the second region, so that the image to be classified is rechecked of the class of defects based on the gray value of the first region containing the defect region and the gray value of the second region not containing the defect region under the condition that the first defect classification result at least indicates that the image to be classified belongs to the class to be rechecked, thereby improving the accuracy of defect classification of the liquid crystal panel.

In a possible implementation manner, the obtaining, in response to the first defect classification result indicating that the image to be classified at least belongs to a class to be rechecked, a gray value of a first region and a gray value of a second region of the image to be classified based on the image to be classified includes:

and in response to the fact that the first defect classification result indicates that the image to be classified belongs to the class to be rechecked and the confidence coefficient of the image to be classified belonging to the class to be rechecked is smaller than or equal to a first preset threshold value, obtaining the gray value of a first region and the gray value of a second region of the image to be classified based on the image to be classified.

In the implementation mode, the image to be classified is rechecked for the defect category by responding to the first defect classification result indicating that the image to be classified belongs to the category to be rechecked and the confidence coefficient of the image to be classified, which indicates that the image to be classified belongs to the category to be rechecked, is less than or equal to the first preset threshold, so that the efficiency of rechecking the defect category for the image to be classified can be improved.

In a possible implementation manner, the obtaining a gray value of a first region and a gray value of a second region of the image to be classified based on the image to be classified includes:

carrying out smoothing treatment on the image to be classified to obtain a smooth image corresponding to the image to be classified;

and obtaining the gray value of the first area and the gray value of the second area of the image to be classified from the smooth image.

In the implementation mode, the smooth image corresponding to the image of the defect area in the image to be classified can be enlarged by smoothing the image to be classified, so that the effect of classifying weak defects is improved.

In a possible implementation manner, the smoothing processing on the image to be classified to obtain a smoothed image corresponding to the image to be classified includes:

and smoothing the image to be classified by adopting a 2 x 2 or 3 x 3 Gaussian convolution core to obtain a smooth image corresponding to the image to be classified.

In this implementation, by using a smaller gaussian convolution kernel, information of defective pixels in the image to be classified can be retained.

In one possible implementation form of the method,

the obtaining of the gray value of the first region and the gray value of the second region of the image to be classified based on the image to be classified includes: obtaining gray values of a plurality of first areas and gray values of a plurality of second areas of the image to be classified based on the image to be classified;

determining a second defect classification result of the image to be classified according to the gray value of the first region and the gray value of the second region, wherein the determining comprises the following steps: and determining a second defect classification result of the image to be classified according to the gray values of the plurality of first areas and the gray values of the plurality of second areas.

In the implementation mode, the second defect classification result of the image to be classified is determined according to the gray values of the plurality of first areas and the gray values of the plurality of second areas, so that the accuracy of defect category review of the image to be classified is improved.

In a possible implementation manner, the determining a second defect classification result of the image to be classified according to the gray-scale values of the plurality of first regions and the gray-scale values of the plurality of second regions includes:

determining a first gray value and/or a second gray value from the gray values of the plurality of first regions, wherein the first gray value represents the maximum gray value of the gray values of the plurality of first regions, and the second gray value represents the minimum gray value of the gray values of the plurality of first regions;

determining a third gray value and/or a fourth gray value from the gray values of the plurality of second areas, wherein the third gray value represents the maximum gray value of the gray values of the plurality of second areas, and the fourth gray value represents the minimum gray value of the gray values of the plurality of second areas;

and determining a second defect classification result of the image to be classified according to at least one of the first gray value and the second gray value and at least one of the third gray value and the fourth gray value.

In this implementation manner, according to different application scenes of defect classification, the second defect classification result of the image to be classified may be determined according to at least one of the first gray value and the second gray value (i.e., the maximum gray value and/or the minimum gray value of the plurality of first regions including the defect region), and at least one of the third gray value and the fourth gray value (i.e., the maximum gray value and/or the minimum gray value of the plurality of second regions not including the defect region), so that the accuracy of defect classification and review of the image to be classified may be improved.

In one possible implementation, the category to be reviewed includes a highlight category;

determining a second defect classification result of the image to be classified according to at least one of the first gray value and the second gray value and at least one of the third gray value and the fourth gray value, including:

under the condition that the first defect classification result indicates that the image to be classified belongs to the bright point category, determining that a second defect classification result of the image to be classified belongs to an unobvious bright point category in response to the second gray value, the third gray value and the fourth gray value meeting a first preset condition.

According to the implementation mode, the category correction of the non-obvious bright point defects which are wrongly classified into the bright point categories can be effectively and accurately carried out.

In one possible implementation manner, the first preset condition includes:

the difference value between the fourth gray value and the second gray value is greater than a second preset threshold value, and the second gray value is smaller than a third preset threshold value;

alternatively, the first and second electrodes may be,

the difference value between the fourth gray value and the second gray value is greater than the second preset threshold, and the difference value between the third gray value and the second gray value is greater than the fourth preset threshold.

According to the implementation mode, the accuracy of classifying the non-obvious bright spot defects can be further improved.

In one possible implementation, the image to be classified includes images of a plurality of channels;

the determining that the image to be classified belongs to the category of non-obvious bright points in response to the second gray value, the third gray value and the fourth gray value meeting a first preset condition includes:

determining that a second defect classification result of the image to be classified belongs to an unobvious bright spot category in response to the second gray scale value, the third gray scale value and the fourth gray scale value of the image of any one of the plurality of channels satisfying a first preset condition.

According to this implementation, the effect of classifying weak defects (non-distinct bright spots) can be improved.

In one possible implementation, the to-be-rechecked category includes a dark point category;

and under the condition that the first defect classification result indicates that the image to be classified belongs to the dark spot category, determining that a second defect classification result of the image to be classified belongs to an unobvious bright spot category in response to that the first gray value and the third gray value meet a second preset condition.

According to the implementation mode, the category correction of the non-obvious bright point defects which are classified into dark point categories can be effectively and accurately carried out.

In one possible implementation manner, the second preset condition includes:

the difference value between the third gray value and the first gray value is greater than a fifth preset threshold, and the first gray value is greater than a sixth preset threshold.

the determining that the second defect classification result of the image to be classified belongs to the category of non-obvious bright spots in response to the first gray value and the third gray value meeting a second preset condition includes:

determining that a second defect classification result of the image to be classified belongs to an unobvious bright point category in response to the first gray value and the third gray value of the image of any one of the plurality of channels satisfying a second preset condition.

In a possible implementation manner, the performing defect classification on the image to be classified to obtain a first defect classification result corresponding to the image to be classified includes:

and carrying out defect classification on the image to be classified through the trained first neural network to obtain a first defect classification result corresponding to the image to be classified.

In the implementation mode, the trained first neural network is adopted to classify the defects of the image to be classified, so that the accuracy and the speed of classifying the defects of the image to be classified can be improved.

In one possible implementation manner, before the training of the trained first neural network to classify the defect of the image to be classified, the method further includes:

training a second neural network by adopting a training image set, wherein training images in the training image set comprise defect type marking data;

in response to the completion of the training of the second neural network, pruning the second neural network to obtain the first neural network;

and training the first neural network by adopting the training image set.

In this implementation, the second neural network can be made to learn the ability to identify the defect class in the image by training the second neural network with a set of training images having defect class annotation data. The second neural network is pruned to obtain the first neural network by responding to the completion of the training of the second neural network, and the first neural network is finely adjusted by adopting the training image set, so that the precision and the speed of defect classification of the image to be classified can be further improved.

In one possible implementation, the second neural network employs a lightweight neural network.

In this implementation, by using the lightweight neural network, a better training effect, that is, a better small sample learning effect, can be achieved in the case that the number of training images in the training image set is small. Since the probability of the liquid crystal panel having a defect is low, it is difficult to obtain a defective sample (i.e., a training image having a defect), and therefore, the probability of the occurrence of overfitting can be reduced by the lightweight neural network.

In one possible implementation, the second neural network adopts a network structure of ResNet-18, and the first neural network adopts a network structure with the first pooling layer of ResNet-18 removed.

Because the number of pixels occupied by the defect area is usually small in the defect image of the liquid crystal panel, the removal of the first pooling layer of ResNet-18 is beneficial to keeping the information of the defect pixel point and the information around the defect pixel point, so that the information loss can be reduced.

In a possible implementation manner, the acquiring an image to be classified of a liquid crystal panel includes:

detecting the defects of the image to be processed of the liquid crystal panel;

in response to the defect area being detected from the image to be processed, cutting out an image with a preset size from the image to be processed as the image to be classified by taking the defect area as a geometric center.

In this implementation, by performing defect detection on an image to be processed of the liquid crystal panel, in response to detecting a defect region from the image to be processed, cutting out an image of a preset size from the image to be processed as an image to be classified with the defect region as a geometric center, and performing defect classification of the liquid crystal panel based on the image to be classified thus obtained, it is helpful to improve accuracy of defect classification.

According to an aspect of the present disclosure, there is provided a defect classification apparatus of a liquid crystal panel, including:

the acquisition module is used for acquiring an image to be classified of the liquid crystal panel;

the first defect classification module is used for classifying the defects of the image to be classified to obtain a first defect classification result corresponding to the image to be classified;

an obtaining module, configured to obtain, in response to the first defect classification result indicating that the image to be classified at least belongs to a class to be rechecked, a gray value of a first region and a gray value of a second region of the image to be classified based on the image to be classified, where the first region includes a defect region, and the second region does not include the defect region;

and the second defect classification module is used for determining a second defect classification result of the image to be classified according to the gray value of the first area and the gray value of the second area.

In one possible implementation, the obtaining module is configured to:

In one possible implementation form of the method,

the obtaining module is configured to: obtaining gray values of a plurality of first areas and gray values of a plurality of second areas of the image to be classified based on the image to be classified;

the second defect classification module is configured to: and determining a second defect classification result of the image to be classified according to the gray values of the plurality of first areas and the gray values of the plurality of second areas.

In one possible implementation, the second defect classification module is configured to:

the second defect classification module is configured to:

In one possible implementation manner, the first preset condition includes:

alternatively, the first and second electrodes may be,

the second defect classification module is configured to:

In one possible implementation manner, the second preset condition includes:

the second defect classification module is configured to:

In one possible implementation, the first defect classification module is configured to:

In one possible implementation, the apparatus further includes:

the first training module is used for training a second neural network by adopting a training image set, wherein training images in the training image set comprise defect type marking data;

the pruning module is used for responding to the completion of the training of the second neural network, and pruning the second neural network to obtain the first neural network;

and the second training module is used for training the first neural network by adopting the training image set.

In one possible implementation manner, the obtaining module is configured to:

detecting the defects of the image to be processed of the liquid crystal panel;

According to an aspect of the present disclosure, there is provided an electronic device including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the memory-stored executable instructions to perform the above-described method.

According to an aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.

In the embodiment of the disclosure, an image to be classified of a liquid crystal panel is obtained, the image to be classified is subjected to defect classification to obtain a first defect classification result corresponding to the image to be classified, in response to that the first defect classification result at least indicates that the image to be classified belongs to a category to be rechecked, a gray value of a first region including a defect region and a gray value of a second region not including the defect region are obtained based on the image to be classified, and the second defect classification result of the image to be classified is determined according to the gray value of the first region and the gray value of the second region, so that the image to be classified can be rechecked of the category of defects based on the gray value of the first region including the defect region and the gray value of the second region not including the defect region under the condition that the first defect classification result at least indicates that the image to be classified belongs to the category to be rechecked, thereby improving the accuracy of defect classification of the liquid crystal panel.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

Fig. 1 shows a flowchart of a defect classification method of a liquid crystal panel provided by an embodiment of the present disclosure.

Fig. 2 is a schematic diagram illustrating a plurality of first regions are cut from a smooth image corresponding to an image to be classified in the defect classification method for a liquid crystal panel provided by the embodiment of the present disclosure.

Fig. 3 is a schematic diagram illustrating a plurality of second regions are cut from a smooth image corresponding to an image to be classified in the defect classification method for a liquid crystal panel provided by the embodiment of the present disclosure.

Fig. 4 shows a block diagram of a defect classification apparatus of a liquid crystal panel provided by an embodiment of the present disclosure.

Fig. 5 illustrates a block diagram of an electronic device 800 provided by an embodiment of the disclosure.

Fig. 6 shows a block diagram of an electronic device 1900 provided by an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.

In the related art, since the accuracy of defect classification of the liquid crystal panel is difficult to meet the requirement, a quality inspector still needs to perform secondary recheck on a defect classification result by naked eyes.

The following describes a defect classification method for a liquid crystal panel according to an embodiment of the present disclosure in detail with reference to the accompanying drawings.

Fig. 1 shows a flowchart of a defect classification method of a liquid crystal panel provided by an embodiment of the present disclosure. In a possible implementation manner, the defect classification method of the liquid crystal panel can be executed by a terminal device or a server or other processing devices. The terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, or a wearable device. In some possible implementations, the defect classification method of the liquid crystal panel may be implemented by a processor calling a computer readable instruction stored in a memory. As shown in fig. 1, the method for classifying defects of a liquid crystal panel includes steps S11 to S14.

In step S11, an image to be classified of the liquid crystal panel is acquired.

In step S12, the image to be classified is defect-classified, and a first defect classification result corresponding to the image to be classified is obtained.

In step S13, in response to that the first defect classification result at least indicates that the image to be classified belongs to a class to be re-examined, based on the image to be classified, obtaining a grayscale value of a first region and a grayscale value of a second region of the image to be classified, where the first region includes a defect region, and the second region does not include the defect region.

In step S14, a second defect classification result of the image to be classified is determined according to the grayscale value of the first region and the grayscale value of the second region.

In the embodiment of the disclosure, the image to be classified of the liquid crystal panel may represent an image of the liquid crystal panel, which needs to be subjected to defect classification. The image to be classified may be an image in which defect detection has been performed and a defect-containing region has been detected, or may be an image in which defect detection has not been performed.

The image to be classified may comprise one or more than two channels. For example, the image to be classified may include 6 channels, wherein different channels of the image to be classified may correspond to different display states of the liquid crystal panel. For example, different channels of the image to be classified may correspond to different color states of the liquid crystal panel. Of course, the number of channels of the image to be classified may be more or less, and those skilled in the art can flexibly select the number according to the actual defect classification requirement. Each channel of the image to be classified may be a grayscale image. By adopting a plurality of channels, richer visual information of the liquid crystal panel is facilitated to be obtained, thereby facilitating more accurate defect classification.

In a possible implementation manner, the acquiring an image to be classified of a liquid crystal panel includes: detecting the defects of the image to be processed of the liquid crystal panel; in response to the defect area being detected from the image to be processed, cutting out an image with a preset size from the image to be processed as the image to be classified by taking the defect area as a geometric center. In this implementation, the image to be processed may be an image photographed on the liquid crystal panel. For example, the size of the image to be processed may be 10000 pixels × 10000 pixels, and 1 pixel may correspond to 0.1 mm of the liquid crystal panel. Of course, those skilled in the art can flexibly select the size of the image to be processed according to the actual defect classification requirement, which is not limited herein. The image to be processed may comprise one or more than two channels. As an example of this implementation, the image to be processed may include 6 channels. For example, the liquid crystal panel may be controlled to display 6 colors, and in the 6 color states, the liquid crystal panel may be respectively subjected to image acquisition to obtain 6 channels of images to be processed.

As an example of this implementation, the image to be processed may be input into a trained third neural network, and the defect detection may be performed on the image to be processed through the third neural network, so as to obtain the position information of the defect area in the image to be processed. The third neural network may be pre-trained by using a training image set, and any one of the training images in the training image set may have label data of position information of a defect region in the training image.

In this implementation manner, by performing defect detection on the image to be processed, whether a defect region exists in the image to be processed can be determined, and the position information of each defect region can be obtained when the defect region exists in the image to be processed. For any defect area detected from the image to be processed, an image with a preset size can be cut out from the image to be processed by taking the defect area as a geometric center to serve as an image to be classified. If the image to be processed has a plurality of defect areas, a plurality of images to be classified can be obtained. For example, the preset size may be 50 pixels × 50 pixels. Of course, those skilled in the art can flexibly select the size of the preset dimension according to the actual defect classification requirement, and the size is not limited herein.

In another possible implementation manner, the acquiring an image to be classified of a liquid crystal panel includes: dividing the image to be processed of the liquid crystal panel into a plurality of image blocks with preset sizes, and taking each image block as an image to be classified respectively.

In the embodiment of the present disclosure, the defect categories of the liquid crystal panel may include two or more. For example, the defect categories of the liquid crystal panel may include a bright point category, a dark point category, a bright-dark juxtaposition category, and an unobtrusive bright point category. The bright point may represent a point that represents R (Red), G (Green), and B (Blue) when the liquid crystal panel is a black screen. The dark dots may represent dots displayed as the non-simple R, G, B in the case where the liquid crystal panel is a white screen. The bright-dark parallel category may indicate that there are brightness and dark spots in the liquid crystal panel at a distance less than or equal to a preset distance, for example, the preset distance may be 4 pixels, 5 pixels, or the like. An unobtrusive bright spot may represent a bright spot having a brightness lower than the bright spot and an area larger than the bright spot. The non-distinct bright spot category may also be referred to as a leak bright spot category. Of course, those skilled in the art may set more or less defect categories according to the actual defect classification requirement, and the setting is not limited herein.

In this embodiment of the present disclosure, the first defect classification result may represent a defect classification result obtained by performing pre-classification on the image to be classified, and the second defect classification result may represent a defect classification result obtained by performing review of the defect classification on the image to be classified. In one possible implementation, the first defect classification result and/or the second defect classification result may include confidences that the image to be classified belongs to more than two defect classes. For example, if the defect classes include 4 types, the first defect classification result and/or the second defect classification result may include confidences that the image to be classified belongs to each of the 4 defect classes. In another possible implementation, the first defect classification result and/or the second defect classification result may include confidences that the image to be classified belongs to two or more defect classes and defect-free classes. For example, if the defect classes include 4 types, the first defect classification result and/or the second defect classification result may include confidences that the image to be classified belongs to each of the 4 defect classes and the non-defect class. In one possible implementation, if the defect classes include 4 types, the first defect classification result and the second defect classification result may be used to indicate to which of the 4 defect classes the image to be classified belongs. In another possible implementation, if the defect classes include 4 types, the first defect classification result and the second defect classification result may be used to indicate which of the 4 defect classes and non-defect classes the image to be classified belongs to.

In a possible implementation manner, the performing defect classification on the image to be classified to obtain a first defect classification result corresponding to the image to be classified includes: and carrying out defect classification on the image to be classified through the trained first neural network to obtain a first defect classification result corresponding to the image to be classified. In this implementation, the first Neural network may be a Deep Neural Network (DNN). The first defect classification result may represent a defect classification result of an image to be classified obtained through the first neural network. In one example, the standardized image to be classified may be obtained by performing a standardization operation on the image to be classified, and the first defect classification result corresponding to the image to be classified may be obtained by performing defect classification on the standardized image to be classified through the trained first neural network. In another example, the image to be classified may be normalized to obtain a normalized image to be classified, and the normalized image to be classified may be defect-classified by the trained first neural network to obtain a first defect classification result corresponding to the image to be classified. In the implementation mode, the trained first neural network is adopted to classify the defects of the image to be classified, so that the accuracy and the speed of classifying the defects of the image to be classified can be improved.

As an example of the implementation manner, the image to be classified includes a plurality of channels, and the image to be classified including the plurality of channels may be subjected to defect classification through the first neural network, so as to obtain a first defect classification result corresponding to the image to be classified.

As an example of this implementation, before the training of the trained first neural network to classify the defect of the image to be classified, the method further includes: training a second neural network by adopting a training image set, wherein training images in the training image set comprise defect type marking data; in response to the completion of the training of the second neural network, pruning the second neural network to obtain the first neural network; and training the first neural network by adopting the training image set. In this example, the second neural network can be made to learn the ability to identify defect classes in the image by training the second neural network with a set of training images having defect class labeling data. The second neural network is pruned to obtain the first neural network by responding to the completion of the training of the second neural network, and the first neural network is finely adjusted by adopting the training image set, so that the precision and the speed of defect classification of the image to be classified can be further improved.

In one example, the training image set may include 4 classes of training images, respectively training images of a bright spot class, training images of a dark spot class, training images of a bright-dark side-by-side class, and training images of a non-distinct bright spot class. Wherein the number of training images per category may be 100 or several hundred.

In another example, the training image set may include 5 classes of training images, respectively a training image of a bright spot class, a training image of a dark spot class, a training image of a bright-dark side-by-side class, a training image of a non-distinct bright spot class, and a training image of a non-defective class. Wherein the number of training images per category may be 100 or several hundred.

In one example, any of the training images may include 6 channels. For any training image, if the image of any channel has a bright spot, the defect type marking data corresponding to the training image can be of a bright spot type; if the image of any channel has dark points, the defect type marking data corresponding to the training image can be the dark point type; if the images of any channel are parallel in brightness and darkness, the defect type marking data corresponding to the training images can be parallel in brightness and darkness; if the image of any channel has an unobvious bright spot, the defect class label data corresponding to the training image may be an unobvious bright spot class.

In one example, the second neural network employs a lightweight neural network. By adopting the lightweight neural network, the better training effect is favorably realized under the condition that the number of training images in the training image set is less, namely, the better small sample learning effect is favorably realized. Since the probability of the liquid crystal panel having a defect is low, it is difficult to obtain a defective sample (i.e., a training image having a defect), and therefore, the probability of the occurrence of overfitting can be reduced by the lightweight neural network.

In one example, the second neural network employs a network structure of ResNet-18, and the first neural network employs a network structure with the first pooling layer of ResNet-18 removed. Because the number of pixels occupied by the defect area is usually small in the defect image of the liquid crystal panel, the removal of the first pooling layer of ResNet-18 is beneficial to keeping the information of the defect pixel point and the information around the defect pixel point, so that the information loss can be reduced. Wherein the defect image represents an image containing a defect. Experiments show that by removing the first pooling layer of ResNet-18 and then fine-tuning, the defect classification accuracy can be improved by about 2% compared with the ResNet-18 before the first pooling layer is removed.

In other examples, the second neural network may also adopt a network structure such as VGG11, MobileNet, and the like, which is not limited herein.

Of course, the second neural network may also employ a non-lightweight neural network where a larger set of defect samples can be obtained.

As another example of this implementation, before the training of the trained first neural network to classify the defect of the image to be classified, the method further includes: and training the first neural network by adopting a training image set, wherein the training images in the training image set comprise defect type marking data. In this example, the first neural network may be ResNet-18, VGG11, MobileNet, or the like. In this example, the trained neural network may not be pruned.

In another possible implementation manner, the image to be classified may be subjected to defect classification through a pre-designed defect classification function, so as to obtain a first defect classification result corresponding to the image to be classified.

In a possible implementation manner, the obtaining, in response to the first defect classification result indicating that the image to be classified at least belongs to a class to be rechecked, a gray value of a first region and a gray value of a second region of the image to be classified based on the image to be classified includes: and in response to the fact that the first defect classification result indicates that the image to be classified belongs to the class to be rechecked and the confidence coefficient of the image to be classified belonging to the class to be rechecked is smaller than or equal to a first preset threshold value, obtaining the gray value of a first region and the gray value of a second region of the image to be classified based on the image to be classified.

As one example of this implementation, the first defect classification result may include a confidence that the image to be classified belongs to more than two defect classes. For example, if the defect classes include 4 types, the first defect classification result may include confidence that the image to be classified belongs to each of the 4 defect classes. For example, the confidence that the image to be classified belongs to more than two defect classes may be obtained by the first neural network.

As another example of this implementation, the first defect classification result may include a confidence that the image to be classified belongs to two or more defect classes and a non-defect class. For example, if the defect classes include 4 types, the first defect classification result may include confidence that the image to be classified belongs to each of the 4 defect classes and the non-defect class. For example, the confidence that the image to be classified belongs to more than two defect classes and defect-free classes can be obtained by the first neural network.

For example, the first preset threshold may be 90% or 95%, and so on. For example, the category to be re-checked includes a bright spot category, and the first preset threshold is 95%. For another example, the to-be-rechecked category includes a dark-spot category, and the first preset threshold is 90%. In the implementation mode, the image to be classified is rechecked for the defect category by responding to the first defect classification result indicating that the image to be classified belongs to the category to be rechecked and the confidence coefficient of the image to be classified, which indicates that the image to be classified belongs to the category to be rechecked, is less than or equal to the first preset threshold, so that the efficiency of rechecking the defect category for the image to be classified can be improved. In this implementation manner, if the first defect classification result indicates that the image to be classified belongs to the class to be rechecked, and the confidence level that the first defect classification result indicates that the image to be classified belongs to the class to be rechecked is greater than the first preset threshold, the rechecking of the defect class on the image to be classified may be performed, and the image to be classified is directly determined to belong to the class to be rechecked.

In another possible implementation manner, the confidence that the image to be classified belongs to the class to be rechecked in the first defect classification result may not be considered, and the image to be classified is rechecked for the defect class in response to the indication that the first defect classification result indicates that the image to be classified belongs to the class to be rechecked.

In the embodiment of the present disclosure, the first region and the second region may represent regions cut out from an image processed from an image to be classified, or may represent regions cut out from an image to be classified. The size of the first area and the size of the second area are both smaller than the size of the image to be classified. The first region and the second region may be the same size or different sizes. For example, the size of the image to be classified is 50 pixels × 50 pixels, and the sizes of the first region and the second region are both 15 pixels × 15 pixels. The size of the defect area is typically small and may be, for example, 2 to 4 pixel units. The gray value of the area may represent a value capable of representing the degree of brightness of the area. For example, the grayscale value of a region may be an average of the grayscale values of the individual pixels in the region. For another example, the gray-scale value of the region may be a median or a mode of the gray-scale values of the respective pixels in the region, which is not limited herein.

In a possible implementation manner, obtaining the grayscale value of the first region and the grayscale value of the second region of the image to be classified includes: carrying out smoothing treatment on the image to be classified to obtain a smooth image corresponding to the image to be classified; and obtaining the gray value of the first area and the gray value of the second area of the image to be classified from the smooth image. For example, the image to be classified may be smoothed by convolution or filtering. For example, gaussian convolution may be performed on the image to be classified to obtain a smooth image corresponding to the image to be classified. For another example, gaussian convolution and normalization may be performed on the image to be classified to obtain a smooth image corresponding to the image to be classified. In the implementation mode, the smooth image corresponding to the image of the defect area in the image to be classified can be enlarged by smoothing the image to be classified, so that the effect of classifying weak defects is improved. The defective area may indicate an area including a defective pixel. Weak defects may represent defects that are easily misclassified into other categories.

As an example of this implementation, the smoothing processing on the image to be classified to obtain a smoothed image corresponding to the image to be classified includes: and smoothing the image to be classified by adopting a 2 x 2 or 3 x 3 Gaussian convolution core to obtain a smooth image corresponding to the image to be classified. In this example, by using a smaller gaussian convolution kernel, information can thus be retained about defective pixels in the image to be classified.

In a possible implementation manner, the obtaining a gray value of a first region and a gray value of a second region of the image to be classified based on the image to be classified includes: obtaining gray values of a plurality of first areas and gray values of a plurality of second areas of the image to be classified based on the image to be classified; determining a second defect classification result of the image to be classified according to the gray value of the first region and the gray value of the second region, wherein the determining comprises the following steps: and determining a second defect classification result of the image to be classified according to the gray values of the plurality of first areas and the gray values of the plurality of second areas. For example, a plurality of first regions including a defective region and a plurality of second regions not including a defective region may be cut out from a smoothed image corresponding to an image to be classified. Each first area comprises a defect area, and the positions of different first areas are different; each second area does not contain a defect area, and the positions of different second areas are different. In the implementation mode, the second defect classification result of the image to be classified is determined according to the gray values of the plurality of first areas and the gray values of the plurality of second areas, so that the accuracy of defect category review of the image to be classified is improved.

Fig. 2 is a schematic diagram illustrating a plurality of first regions are cut from a smooth image corresponding to an image to be classified in the defect classification method for a liquid crystal panel provided by the embodiment of the present disclosure. In the example shown in fig. 2, the smooth image 21 corresponding to the image to be classified includes a defect region 22, where the size of the smooth image 21 is 50 pixels × 50 pixels, and the size of the defect region 22 is 3 pixels × 3 pixels. A plurality of first regions 23 may be cut out from the smoothed image 21, wherein each first region 23 includes a defect region 22, and each first region 23 has a size of 15 pixels × 15 pixels.

Fig. 3 is a schematic diagram illustrating a plurality of second regions are cut from a smooth image corresponding to an image to be classified in the defect classification method for a liquid crystal panel provided by the embodiment of the present disclosure. In the example shown in fig. 3, the smooth image 21 corresponding to the image to be classified includes a defect region 22, where the size of the smooth image 21 is 50 pixels × 50 pixels, and the size of the defect region 22 is 3 pixels × 3 pixels. A plurality of second regions 24 may be cut from the smoothed image 21, wherein each second region 24 does not contain a defective region 22, and each second region 24 has a size of 15 pixels × 15 pixels.

As an example of this implementation, the determining a second defect classification result of the image to be classified according to the gray-scale values of the plurality of first regions and the gray-scale values of the plurality of second regions includes: determining a first gray value and/or a second gray value from the gray values of the plurality of first regions, wherein the first gray value represents the maximum gray value of the gray values of the plurality of first regions, and the second gray value represents the minimum gray value of the gray values of the plurality of first regions; determining a third gray value and/or a fourth gray value from the gray values of the plurality of second areas, wherein the third gray value represents the maximum gray value of the gray values of the plurality of second areas, and the fourth gray value represents the minimum gray value of the gray values of the plurality of second areas; and determining a second defect classification result of the image to be classified according to at least one of the first gray value and the second gray value and at least one of the third gray value and the fourth gray value. In this example, only the first gradation value may be determined from among the gradation values of the plurality of first regions. Alternatively, only the second gray value may be determined from among the gray values of the plurality of first regions. Alternatively, the first gray scale value and the second gray scale value may be respectively determined from among the gray scale values of the plurality of first regions. In this example, only the third gradation value may be determined from among the gradation values of the plurality of second regions. Alternatively, only the fourth gradation value may be determined from among the gradation values of the plurality of second regions. Alternatively, the third gray value and the fourth gray value may be determined from among the gray values of the plurality of second regions. In this example, according to different application scenes of defect classification, the second defect classification result of the image to be classified may be determined according to at least one of the first gray scale value and the second gray scale value (i.e., the maximum gray scale value and/or the minimum gray scale value of the plurality of first regions including the defect region), and at least one of the third gray scale value and the fourth gray scale value (i.e., the maximum gray scale value and/or the minimum gray scale value of the plurality of second regions not including the defect region), so that the accuracy of defect classification and review of the image to be classified can be improved.

For example, in the example shown in fig. 2, the gray scale values of the respective first regions 23 may be determined, respectively, and the maximum gray scale value among the gray scale values of the respective first regions 23 may be taken as the first gray scale value, and the minimum gray scale value among the gray scale values of the respective first regions 23 may be taken as the second gray scale value. In the example shown in fig. 3, the gray scale values of the respective second regions 24 may be determined, respectively, and the maximum gray scale value among the gray scale values of the respective second regions 24 may be taken as the third gray scale value, and the minimum gray scale value among the gray scale values of the respective second regions 24 may be taken as the fourth gray scale value.

In one example, the category to be re-examined includes a highlight category; determining a second defect classification result of the image to be classified according to at least one of the first gray value and the second gray value and at least one of the third gray value and the fourth gray value, including: under the condition that the first defect classification result indicates that the image to be classified belongs to the bright point category, determining that a second defect classification result of the image to be classified belongs to an unobvious bright point category in response to the second gray value, the third gray value and the fourth gray value meeting a first preset condition. In one example, in response to the first defect classification result indicating that the image to be classified belongs to the bright point category and the confidence that the image to be classified belongs to the bright point category is less than or equal to 95%, whether the image to be classified belongs to the non-obvious bright point category or not may be determined according to the second gray value, the third gray value, the fourth gray value and the first preset condition. According to this example, the category correction of the non-significant bright point defect that is erroneously classified into the bright point category can be efficiently and accurately performed.

In one example, the first preset condition includes: the difference value between the fourth gray value and the second gray value is greater than a second preset threshold value, and the second gray value is smaller than a third preset threshold value; or the difference between the fourth gray value and the second gray value is greater than the second preset threshold, and the difference between the third gray value and the second gray value is greater than the fourth preset threshold. For example, the second preset threshold value belongs to the interval [70,90], the third preset threshold value belongs to the interval [20,40], and the fourth preset threshold value belongs to the interval [135,155 ]. For another example, the second predetermined threshold belongs to the interval [75,85], the third predetermined threshold belongs to the interval [25,35], and the fourth predetermined threshold belongs to the interval [140,150 ]. For example, the second preset threshold is 80, the third preset threshold is 30, and the fourth preset threshold is 145. According to this example, the accuracy of classifying the non-significant bright spot defect can be further improved.

In one example, the image to be classified includes images of a plurality of channels; the determining that the image to be classified belongs to the category of non-obvious bright points in response to the second gray value, the third gray value and the fourth gray value meeting a first preset condition includes: determining that a second defect classification result of the image to be classified belongs to an unobvious bright spot category in response to the second gray scale value, the third gray scale value and the fourth gray scale value of the image of any one of the plurality of channels satisfying a first preset condition. For example, the image to be classified includes 6 channels, and if the second gray value, the third gray value, and the fourth gray value of the image of any one of the 6 channels satisfy the first preset condition, the category of the image to be classified may be corrected to be the category of the non-obvious bright point, so that the classification effect on the weak defect (the non-obvious bright point) may be improved.

In another example, the class to be re-checked includes a dark point class; determining a second defect classification result of the image to be classified according to at least one of the first gray value and the second gray value and at least one of the third gray value and the fourth gray value, including: and under the condition that the first defect classification result indicates that the image to be classified belongs to the dark spot category, determining that a second defect classification result of the image to be classified belongs to an unobvious bright spot category in response to that the first gray value and the third gray value meet a second preset condition. In one example, in response to the first defect classification result indicating that the image to be classified belongs to the dark spot category and the confidence level that the image to be classified belongs to the dark spot category is less than or equal to 90%, whether the image to be classified belongs to the non-obvious bright spot category or not may be determined according to the first gray value, the third gray value and the second preset condition. According to this example, it is possible to efficiently and accurately correct the category of an unobtrusive bright point defect that is mistakenly classified into a dark point category.

In one example, the second preset condition includes: the difference value between the third gray value and the first gray value is greater than a fifth preset threshold, and the first gray value is greater than a sixth preset threshold. For example, the fifth preset threshold belongs to the interval [70,90], and the sixth preset threshold belongs to the interval [160,180 ]. As another example, the fifth predetermined threshold belongs to the interval [75,85], and the sixth predetermined threshold belongs to the interval [165,175 ]. For example, the fifth preset threshold is 80, and the sixth preset threshold is 170. According to this example, the accuracy of classifying the non-significant bright spot defect can be further improved.

In one example, the image to be classified includes images of a plurality of channels; the determining that the second defect classification result of the image to be classified belongs to the category of non-obvious bright spots in response to the first gray value and the third gray value meeting a second preset condition includes: determining that a second defect classification result of the image to be classified belongs to an unobvious bright point category in response to the first gray value and the third gray value of the image of any one of the plurality of channels satisfying a second preset condition. For example, the image to be classified includes 6 channels, and if the first gray value and the third gray value of the image of any one of the 6 channels satisfy the second preset condition, the category of the image to be classified may be corrected to be the category of the non-obvious bright point, so that the classification effect on the weak defect (the non-obvious bright point) may be improved.

As another example of this implementation, the determining a second defect classification result of the image to be classified according to the gray-scale values of the plurality of first regions and the gray-scale values of the plurality of second regions includes: and determining a second defect classification result of the image to be classified according to the average value of the gray values of the plurality of first areas and the average value of the gray values of the plurality of second areas.

In another possible implementation manner, the obtaining the gray-level value of the first region and the gray-level value of the second region of the image to be classified based on the image to be classified includes: obtaining a gray value of a first area and gray values of a plurality of second areas based on the image to be classified; determining a second defect classification result of the image to be classified according to the gray value of the first region and the gray value of the second region, wherein the determining comprises the following steps: and determining a second defect classification result of the image to be classified according to the gray value of the first area and the gray values of the plurality of second areas. In this implementation, the area of the first region may be greater than or equal to the area of the defective region.

The embodiment of the disclosure can be applied to the technical fields of computer vision, image processing, intelligent industrial quality inspection and the like, automatically classifies the defects of the liquid crystal panel, reduces manual intervention, and improves the production efficiency and the product quality control.

The following describes a defect classification method for a liquid crystal panel according to an embodiment of the present disclosure with a specific application scenario. In the application scene, the liquid crystal panel can be controlled to display 6 colors, and image acquisition is respectively carried out on the liquid crystal panel under the 6 color states, so that 6 channels of images to be processed are obtained. The defect detection can be carried out on the image to be processed to obtain the defect area in the image to be processed. For any defect area in the image to be processed, an image with 50 pixels × 50 pixels can be cut out from the image to be processed as an image to be classified with the defect area as a geometric center, wherein the image to be classified also comprises 6 channels.

The image to be classified can be input into the first neural network, and the image to be classified is subjected to defect classification through the first neural network, so that a first defect classification result corresponding to the image to be classified is obtained. Wherein the images of 6 channels of the image to be classified as a whole can be input into the first neural network. The first neural network may take the form of a network structure with the first pooling layer of ResNet-18 removed.

The first defect classification result corresponding to the image to be classified may include confidence levels that the image to be classified belongs to a bright point category, a dark point category, a bright-dark parallel category and an unobvious bright point category. If the confidence coefficient that the image to be classified belongs to the bright point category is the highest in the first defect classification result, the first defect classification result indicates that the image to be classified belongs to the bright point category; if the confidence coefficient that the image to be classified belongs to the dark point category is the highest in the first defect classification result, the first defect classification result indicates that the image to be classified belongs to the dark point category; if the confidence coefficient that the image to be classified belongs to the bright-dark parallel category is the highest in the first defect classification result, the first defect classification result indicates that the image to be classified belongs to the bright-dark parallel category; if the confidence coefficient that the image to be classified belongs to the category of the non-obvious bright points is the highest in the first defect classification result, the first defect classification result indicates that the image to be classified belongs to the category of the non-obvious bright points.

If the first defect classification result indicates that the image to be classified belongs to the bright spot category and the confidence coefficient of the image to be classified belonging to the bright spot category is greater than 95%, the defect category of the image to be classified can be judged to be the bright spot category, and the defect category of the image to be classified does not need to be rechecked; if the first defect classification result indicates that the image to be classified belongs to the dark point category and the confidence coefficient of the image to be classified belonging to the dark point category is greater than 90%, the defect category of the image to be classified can be judged to be the dark point category, and further rechecking of the defect category of the image to be classified is not needed; if the first defect classification result indicates that the image to be classified belongs to the bright-dark parallel category, the defect category of the image to be classified can be judged to be the bright-dark parallel category, and further rechecking of the defect category of the image to be classified is not needed; if the first defect classification result indicates that the image to be classified belongs to the category of the non-obvious bright point, the defect category of the image to be classified can be judged to be the category of the non-obvious bright point, and further rechecking of the defect category of the image to be classified is not needed.

If the first defect classification result indicates that the image to be classified belongs to the bright spot category and the confidence coefficient of the image to be classified belonging to the bright spot category is less than or equal to 95%, whether the images of the 6 channels of the image to be classified meet 'b' or not can be respectively judged_min-c_min> 80 and c_min< 30 "or" b_min-c_min> 80 and b_max-c_min> 145' where b_minRepresents a fourth gray value, c_minRepresents a second gray value, b_maxRepresenting a third gray value. If the image of any channel of the image to be classified satisfies' b_min-c_min> 80 and c_min< 30 "or" b_min-c_min> 80 and b_max-c_minAnd if the defect classification result is more than 145', the second defect classification result of the image to be classified can be determined to be that the image to be classified belongs to the category of non-obvious bright spots. If the images of 6 channels of the image to be classified do not satisfy' b_min-c_min> 80 and c_min< 30 "or" b_min-c_min> 80 and b_max-c_minAnd if the defect classification result of the image to be classified is more than 145', the image to be classified belongs to the bright spot category.

If the first defect classification result indicates that the image to be classified belongs to the dark spot category and the confidence coefficient of the image to be classified belonging to the dark spot category is less than 90%, whether the images of 6 channels of the image to be classified meet 'b' or not can be respectively judged_max-c_max> 80 and c_max> 170' where c_maxRepresenting a first grey value, b_maxRepresenting a third gray value. If the image of any channel of the image to be classified satisfies' b_max-c_max> 80 and c_maxAnd more than 170', determining that the image to be classified belongs to the category of non-obvious bright spots as a second defect classification result of the image to be classified. If the images of 6 channels of the image to be classified do not satisfy' b_max-c_max> 80 and c_maxAnd more than 170', determining that the image to be classified belongs to the dark spot category as the second defect classification result of the image to be classified.

The inventors found in the course of carrying out the present invention that, among 4 categories of a bright point category, a dark point category, a bright-dark parallel category, and an unobtrusive bright point category, the unobtrusive bright point category is more likely to be confused with other categories. When certain features of the non-distinct bright spot categories are weak, they are easily mistaken for a bright spot category or a dark spot category. Wherein, the image containing the non-obvious bright spots is mistakenly divided into the bright spots, usually because the dark spot area of the single-channel image with the dark spot area in the image is smaller or not dark enough as a whole; images containing non-distinct bright spots are misclassified as dark spots, typically because the bright spot areas of a single channel image in which bright spot areas are present in the image are smaller or less bright overall. Therefore, in the above application scenario, the secondary review correction is mainly performed in the case where the defects belonging to the non-distinct bright point category are misclassified. The classification accuracy of the 'short-plate' non-obvious bright point category can be improved by 15% on the test set by performing post-processing under the condition that the 'first defect classification result indicates that the image to be classified belongs to the bright point category and the confidence coefficient of the image to be classified belonging to the bright point category is less than or equal to 95%' or the 'first defect classification result indicates that the image to be classified belongs to the dark point category and the confidence coefficient of the image to be classified belonging to the dark point category is less than 90%'.

It is understood that the above-mentioned method embodiments of the present disclosure can be combined with each other to form a combined embodiment without departing from the logic of the principle, which is limited by the space, and the detailed description of the present disclosure is omitted. Those skilled in the art will appreciate that in the above methods of the specific embodiments, the specific order of execution of the steps should be determined by their function and possibly their inherent logic.

In addition, the present disclosure also provides a defect classification apparatus for a liquid crystal panel, an electronic device, a computer-readable storage medium, and a program, which can be used to implement any one of the defect classification methods for a liquid crystal panel provided by the present disclosure, and corresponding technical solutions and technical effects can be referred to in corresponding descriptions of the method sections and are not described again.

Fig. 4 shows a block diagram of a defect classification apparatus of a liquid crystal panel provided by an embodiment of the present disclosure. As shown in fig. 4, the defect classification apparatus of the liquid crystal panel includes:

an obtaining module 41, configured to obtain an image to be classified of a liquid crystal panel;

the first defect classification module 42 is configured to perform defect classification on the image to be classified to obtain a first defect classification result corresponding to the image to be classified;

an obtaining module 43, configured to, in response to that the first defect classification result at least indicates that the image to be classified belongs to a class to be rechecked, obtain, based on the image to be classified, a grayscale value of a first region and a grayscale value of a second region of the image to be classified, where the first region includes a defect region, and the second region does not include the defect region;

and a second defect classification module 44, configured to determine a second defect classification result of the image to be classified according to the gray value of the first region and the gray value of the second region.

In a possible implementation manner, the obtaining module 43 is configured to:

In one possible implementation form of the method,

the obtaining module 43 is configured to: obtaining gray values of a plurality of first areas and gray values of a plurality of second areas of the image to be classified based on the image to be classified;

the second defect classification module 44 is configured to: and determining a second defect classification result of the image to be classified according to the gray values of the plurality of first areas and the gray values of the plurality of second areas.

In one possible implementation, the second defect classification module 44 is configured to:

the second defect classification module 44 is configured to:

In one possible implementation manner, the first preset condition includes:

alternatively, the first and second electrodes may be,

the second defect classification module 44 is configured to:

In one possible implementation manner, the second preset condition includes:

the second defect classification module 44 is configured to:

In one possible implementation, the first defect classification module 42 is configured to:

In one possible implementation, the apparatus further includes:

In a possible implementation manner, the obtaining module 41 is configured to:

detecting the defects of the image to be processed of the liquid crystal panel;

In some embodiments, functions or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementations and technical effects thereof may refer to the description of the above method embodiments, which are not described herein again for brevity.

Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.

Embodiments of the present disclosure also provide a computer program, which includes computer readable code, and when the computer readable code runs in an electronic device, a processor in the electronic device executes the above method.

The disclosed embodiments also provide a computer program product comprising computer readable code or a non-volatile computer readable storage medium carrying computer readable code, which when run in an electronic device, a processor in the electronic device performs the above method.

An embodiment of the present disclosure further provides an electronic device, including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the memory-stored executable instructions to perform the above-described method.

The electronic device may be provided as a terminal, server, or other form of device.

Fig. 5 illustrates a block diagram of an electronic device 800 provided by an embodiment of the disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal.

Referring to fig. 5, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as a wireless network (Wi-Fi), a second generation mobile communication technology (2G), a third generation mobile communication technology (3G), a fourth generation mobile communication technology (4G)/long term evolution of universal mobile communication technology (LTE), a fifth generation mobile communication technology (5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.

Fig. 6 shows a block diagram of an electronic device 1900 provided by an embodiment of the disclosure. For example, the electronic device 1900 may be provided as a server. Referring to fig. 6, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system, such as the Microsoft Server operating system (Windows Server), stored in the memory 1932^TM) Apple Inc. of the present application based on the graphic user interface operating System (Mac OS X)^TM) Multi-user, multi-process computer operating system (Unix)^TM) Free and open native code Unix-like operating System (Linux)^TM) Open native code Unix-like operating System (FreeBSD)^TM) Or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.

The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for classifying defects of a liquid crystal panel is characterized by comprising the following steps:

acquiring an image to be classified of the liquid crystal panel;

2. The method according to claim 1, wherein the obtaining a gray value of a first region and a gray value of a second region of the image to be classified based on the image to be classified in response to the first defect classification result indicating at least that the image to be classified belongs to a class to be re-examined comprises:

3. The method according to claim 1 or 2, wherein the obtaining of the gray value of the first region and the gray value of the second region of the image to be classified based on the image to be classified comprises:

4. The method according to claim 3, wherein the smoothing of the image to be classified to obtain a smoothed image corresponding to the image to be classified comprises:

5. The method according to any one of claims 1 to 4,

6. The method according to claim 5, wherein determining a second defect classification result of the image to be classified according to the gray values of the plurality of first regions and the gray values of the plurality of second regions comprises:

7. The method of claim 6, wherein the category to be re-examined comprises a bright spot category;

8. The method according to claim 7, wherein the first preset condition comprises:

alternatively, the first and second electrodes may be,

9. The method according to claim 7 or 8, characterized in that the image to be classified comprises images of a plurality of channels;

10. The method of claim 6, wherein the class to be re-checked comprises a dark spot class;

11. The method according to claim 10, wherein the second preset condition comprises:

12. The method according to claim 10 or 11, wherein the image to be classified comprises images of a plurality of channels;

13. The method according to any one of claims 1 to 12, wherein the performing defect classification on the image to be classified to obtain a first defect classification result corresponding to the image to be classified includes:

14. The method of claim 13, wherein prior to the classifying the image to be classified by the trained first neural network for defects, the method further comprises:

and training the first neural network by adopting the training image set.

15. The method of claim 14, wherein the second neural network employs a lightweight neural network.

16. The method of claim 14 or 15, wherein the second neural network adopts a network structure of ResNet-18, and the first neural network adopts a network structure with the first pooling layer of ResNet-18 removed.

17. The method according to any one of claims 1 to 16, wherein the acquiring the image to be classified of the liquid crystal panel comprises:

detecting the defects of the image to be processed of the liquid crystal panel;

18. A defect classification device for a liquid crystal panel, comprising:

19. An electronic device, comprising:

one or more processors;

a memory for storing executable instructions;

wherein the one or more processors are configured to invoke the memory-stored executable instructions to perform the method of any one of claims 1 to 17.

20. A computer readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1 to 17.