CN115937065A

CN115937065A - Foreign matter detection method, device and equipment of display module and storage medium

Info

Publication number: CN115937065A
Application number: CN202210014145.1A
Authority: CN
Inventors: 黄建军; 冯翊; 杨俊鑫
Original assignee: Shenzhen University
Current assignee: Shenzhen University
Priority date: 2022-04-20
Filing date: 2022-04-20
Publication date: 2023-04-07

Abstract

The invention discloses a foreign matter detection method, a device, equipment and a storage medium of a display module, wherein after the shot image of the display module is subjected to binarization processing, each pixel is marked with a mark value to obtain a binarized image, then the root mark value of each pixel in the binarized image is identified based on a parallel connected domain marking algorithm carried out by parallel searching, the image is subjected to foreign matter identification detection based on the root mark value to obtain a detection result, and the foreign matter is subjected to interpolation evasion or foreign matter alarm according to the actual production requirement based on the detection result. The method uses the parallel connected domain marking algorithm based on parallel searching, makes full use of the parallel computing capacity of the GPU, adopts a parallel method for three main flows of foreign matter detection, greatly shortens the running time of a foreign matter detection subsystem, can meet the requirements of actual production, and is suitable for industrial production environments with higher requirements on instantaneity and stability.

Description

Foreign matter detection method, device and equipment of display module and storage medium

Technical Field

The invention relates to the technical field of module detection, in particular to a foreign matter detection method, a foreign matter detection device, foreign matter detection equipment and a storage medium of a display module.

Background

In the actual production process of the liquid crystal display panel, foreign matters such as paper scraps, hair, foam paper and the like are inevitably introduced and adhere to a liquid crystal Module (LCD Module), so that a subsequent LCD display defect correction system judges the foreign matters as display defects to compensate, the correction effect is poor, and the product quality is reduced. Therefore, a manufacturer of the liquid crystal display can carry out a foreign matter detection process before a display defect correction process, give an alarm to the liquid crystal module with larger foreign matters or more foreign matters, and carry out image interpolation on the liquid crystal module with smaller foreign matters and smaller foreign matters for eliminating the interference of the foreign matters on the subsequent processes.

In order to solve the above problems, the currently adopted method is to eliminate the influence of the foreign object by a serial way of spot detection, and the implementation way is to extract the foreground and implement the foreground by ways of correcting, marking and the like. In recent years, as the pixel resolution of industrial cameras has become higher, the efficiency and accuracy of detecting foreign objects have become extremely low if the above-described serial image processing method is used for processing.

Disclosure of Invention

The invention mainly aims to provide a foreign matter detection method, a foreign matter detection device, foreign matter detection equipment and a storage medium of a display module, and aims to solve the technical problem that the detection efficiency and accuracy of the existing foreign matter detection scheme of the display module are low.

The invention provides a foreign matter detection method of a display module, which comprises the following steps:

calculating the gray level of each pixel on an image obtained by shooting the display module, and carrying out binarization processing on the image based on the gray level of each pixel and a preset adaptive threshold to obtain a binarized image, wherein the adaptive threshold is a gray threshold value obtained based on the gray level average value of the image;

determining a root mark value of each pixel according to an initial mark value of the pixel in the binary image by using a parallel connected domain marking algorithm carried out by parallel searching, and determining a foreign matter position based on the root mark value;

based on the foreign matter position, counting the number and the area of the foreign matters in the image through a hash table, calculating the actual number of the foreign matters in the image and the area of each foreign matter, generating a detection result of the foreign matters in the display module, and processing the foreign matters in the image in an image interpolation mode according to the detection result and the current production requirement.

Optionally, the calculating and shooting the gray level of each pixel on the image obtained by the display module, and performing binarization processing on the image based on the gray level of each pixel and a preset adaptive threshold to obtain a binarized image includes:

carrying out area division on the image obtained by shooting the display module, and calculating the gray average value of each area based on the result of the area division;

setting a corresponding adaptive threshold according to the gray average value of the region;

calculating the gray level of the pixels in each area, and comparing the gray level of each pixel with the adaptive threshold of the area where the pixel is located to obtain a comparison result;

and taking the pixels with the gray levels of the pixels smaller than the self-adaptive threshold in the comparison result as a foreground, taking the other pixels as a background, and setting the initial mark value of the foreground pixels in the foreground as the index value of the foreground pixels to obtain a binary image.

Optionally, the parallel connected domain labeling algorithm performed by using a parallel search set determines a root label value of each pixel according to an initial label value of a pixel in the binarized image, and determines a foreign object position based on the root label value, including:

configuring at least two parallel threads, identifying whether each foreground pixel in the binary image is communicated with other foreground pixels, and initializing a mark value in the binary image based on an identification result to obtain an initialized mark map group, wherein the initialized mark map group comprises an initialized binary map and at least one foreground pixel communicated dendrogram;

carrying out root searching on each foreground pixel of the initialized binary image to obtain a root node of each foreground pixel;

modifying the mark values of all foreground pixels with the same root node into the mark values of the root node to obtain a query binary image;

taking a root node obtained by root finding as an initial node, converting each foreground pixel connected tree graph into a root tree graph with the depth of 1, and setting a root mark value of a foreground pixel in the query binary graph based on the root tree graph to obtain a combined binary graph;

and determining the position of the foreign matter according to the root mark value of each foreground pixel in the combined binary image.

Optionally, before determining the position of the foreign object according to the root label value of each foreground pixel in the merged binarized map, the method further includes:

carrying out secondary root searching on the merged binary image to obtain a root searching result;

if the same root node exists in the root searching result, combining foreground pixels with the same root node in the combined binary image, and combining all the root tree-shaped images based on the combined binary image;

and if the root searching result shows that the same root node does not exist, determining the position of the foreign matter according to the root mark value of each foreground pixel in the merged binary image.

Optionally, the identifying whether each foreground pixel in the binarized image is connected with other foreground pixels, and performing initialization processing on the label value in the binarized image based on the identification result to obtain an initialization label map group includes:

identifying whether each foreground pixel in the binarized image is communicated with other foreground pixels;

if other foreground pixels are communicated, keeping the mark value of the current foreground pixel, and modifying the mark value of the foreground pixel communicated with the current foreground pixel into the mark value of the current foreground pixel to obtain an initialized binary image;

and constructing at least one foreground pixel connected tree-shaped graph based on the mark value of each foreground pixel in the initialized binary graph.

Optionally, the setting, based on the root tree diagram, a root label value of a foreground pixel in the query binary diagram to obtain a merged binary diagram includes:

judging whether the initial nodes in each root tree-like graph belong to adjacent foreground pixels or not;

if so, merging the root dendrogram with the larger mark value of the starting node into the root dendrogram with the smaller mark value of the starting node, and setting the root mark value of the foreground pixel in the query binarized graph based on the mark value of each foreground pixel in the merged root dendrogram to obtain a merged binarized graph.

Optionally, the counting, based on the foreign object position, the number and the area of the foreign objects in the image through a hash table, calculating the actual number of the foreign objects in the image and the area of each foreign object, generating a detection result of the foreign objects in the display module, and processing the foreign objects in the image through an image interpolation mode according to the detection result and the current production demand, includes:

according to the root mark value corresponding to the foreign body position, calculating the number and the area of the same mark value, and constructing a foreign body hash table;

determining a connected region based on the foreign matter position of the pixel, and counting the number of the connected regions to obtain the actual number of the foreign matters;

calculating the number of pixels in each connected region, and calculating the area of the corresponding foreign matter based on the number;

generating a detection result of the foreign matters in the display module based on the actual number and the area of the foreign matters; and based on the detection result and the current production requirement, avoiding or giving an alarm in an image interpolation mode.

The invention provides a foreign matter detection device of a display module, which comprises:

the binarization processing module is used for calculating the gray level of each pixel on an image obtained by shooting the display module, and performing binarization processing on the image based on the gray level of each pixel and a preset adaptive threshold to obtain a binarization image, wherein the adaptive threshold is a gray threshold value obtained based on the gray level average value of the image;

the marking module is used for determining a root marking value of each pixel according to an initial marking value of the pixel in the binary image by using a parallel connected domain marking algorithm carried out by a parallel search set, and determining a foreign matter position based on the root marking value;

and the detection module is used for counting the number and the area of the foreign matters in the image through a hash table based on the position of the foreign matters, calculating the actual number of the foreign matters in the image and the area of each foreign matter, generating a detection result of the foreign matters in the display module, and processing the foreign matters in the image in an image interpolation mode according to the detection result and the current production requirement.

Optionally, the binarization processing module includes:

the dividing unit is used for carrying out area division on the image obtained by shooting the display module and calculating the gray average value of each area based on the result of the area division;

the setting unit is used for setting a corresponding self-adaptive threshold according to the gray average value of the area;

the comparison unit is used for calculating the gray level of the pixels in each area and comparing the gray level of each pixel with the adaptive threshold of the area where the pixel is located to obtain a comparison result;

and the binarization unit is used for taking the pixels with the gray levels of the pixels smaller than the self-adaptive threshold in the comparison result as the foreground, taking the other pixels as the background, and setting the initial mark value of the foreground pixels in the foreground as the index value of the foreground pixels to obtain a binarization image.

Optionally, the marking module includes:

the initialization unit is used for configuring at least two parallel threads, identifying whether each foreground pixel in the binary image is communicated with other foreground pixels or not, and initializing a mark value in the binary image based on an identification result to obtain an initialization mark map group, wherein the initialization mark map group comprises an initialization binary map and at least one foreground pixel communicated dendrogram;

the first query unit is used for carrying out root searching on each foreground pixel of the initialized binary image to obtain a root node of each foreground pixel; modifying the mark values of all foreground pixels with the same root node into the mark values of the root node to obtain a query binary image;

a merging unit, configured to use a root node obtained by root finding as an initial node, convert each foreground pixel connected dendrogram into a root dendrogram with a depth of 1, and set a root label value of a foreground pixel in the query binarized graph based on the root dendrogram, so as to obtain a merged binarized graph;

and the marking unit is used for determining the position of the foreign matter according to the root mark value of each foreground pixel in the combined binary image.

Optionally, the marking module further includes: a second query unit, specifically configured to:

if the same root node exists in the root finding result, combining foreground pixels with the same root node in the combined binary image, and combining all the root tree-shaped images based on the combined binary image;

Optionally, the initialization unit is specifically configured to:

Optionally, the merging unit is specifically configured to:

and if so, merging the root tree graph with the larger mark value of the starting node into the root tree graph with the smaller mark value of the starting node, and setting the root mark value of the foreground pixel in the query binary graph based on the mark value of each foreground pixel in the merged root tree graph to obtain a merged binary graph.

Optionally, the detection module includes:

the table construction unit is used for calculating the number and the area of the same mark value according to the root mark value corresponding to the foreign body position and constructing a foreign body hash table;

the quantity calculation unit is used for determining the connected regions based on the foreign matter positions of the pixels and counting the quantity of the connected regions to obtain the actual quantity of the foreign matters;

the area calculation unit is used for calculating the number of pixels in each connected region and calculating the area of the corresponding foreign matter based on the number;

the detection unit is used for generating a detection result of the foreign matters in the display module based on the actual quantity and the area of the foreign matters; and avoiding or giving an alarm in an image interpolation mode based on the detection result and the current production requirement.

A third aspect of the present invention provides an electronic device, comprising: the display module comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the steps of the method for detecting the foreign object of the display module according to the first aspect when executing the computer program.

A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps in the method for detecting a foreign object in a display module set provided in the first aspect.

Has the advantages that:

according to the technical scheme, after the shot image of the display module is subjected to binarization processing, each pixel is marked with a mark value to obtain a binarized image, then the root mark value of each pixel in the binarized image is identified based on a parallel connected domain marking algorithm carried out by parallel searching, the image is subjected to foreign matter identification detection based on the root mark value to obtain a detection result, and the foreign matter is subjected to image interpolation or evasion alarm based on the detection result and the production requirement. The method uses a parallel connected domain marking algorithm based on parallel search set, makes full use of the parallel computing capacity of the GPU, adopts a parallel method for three main flows of foreign matter detection, greatly shortens the running time of a foreign matter detection subsystem, can meet the requirements of actual production, and is suitable for industrial production environments with higher requirements on instantaneity and stability.

Drawings

Fig. 1 is a frame diagram of a method for detecting a foreign object in a display module according to the present invention;

FIG. 2 is a schematic view of a first embodiment of a method for detecting foreign objects in a display module according to the present invention;

FIG. 3 is a schematic view of a second embodiment of a method for detecting foreign objects in a display module according to the present invention;

FIG. 4 is a pixel search neighborhood map provided by the present invention;

FIG. 5 is a flow chart of a parallel connected region labeling provided by the present invention;

FIG. 6 is a schematic diagram of a binarized image before initialization according to the present invention;

FIG. 7 is a relationship diagram between an initialized binary image and a pixel connected tree diagram after initialization according to the present invention;

FIG. 8 is a relational diagram of a query binarization graph and a root tree graph provided by the present invention;

FIG. 9 is a relational diagram of a merged binarized graph and a root tree graph according to the present invention;

FIG. 10 is a schematic view of an embodiment of a device for detecting foreign objects in a display module according to the present invention;

fig. 11 is a schematic view of another embodiment of the device for detecting foreign matters in a display module according to the present invention;

fig. 12 is a schematic diagram of an embodiment of an electronic device provided in the present invention.

Detailed Description

The invention provides a parallel foreign matter detection method aiming at the problem of foreign matter detection of an LCD (liquid crystal display). Firstly, foreign matters are parallelly segmented through a regional gray level difference value; and the positions of the foreign matters are obtained by marking the parallel connected domains, and finally the number of the foreign matters is counted in parallel by means of a hash table, so that the time required by foreign matter detection and foreign matter counting in LCD production is greatly reduced.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For convenience of understanding, the following describes specific processes of an embodiment of the present invention, and please refer to fig. 1 and 2, a first embodiment of a method for detecting a foreign object in a display module according to an embodiment of the present invention includes three main processes, where an input of the method is to capture an image uniformly displayed on an LCD, and an output of the method is information about the number of foreign objects adhered to the LCD and the size of the foreign objects, and the method includes: foreign object segmentation, connected region labeling, and region statistics, as shown in fig. 1. The foreign matter detection method includes the steps of:

201. calculating the gray level of each pixel on the image obtained by shooting the display module, and carrying out binarization processing on the image based on the gray level of each pixel and a preset self-adaptive threshold to obtain a binarized image;

the self-adaptive threshold is a gray threshold value set based on the gray average value of the image, and specifically, foreign matters shot in the image are segmented from the background of the liquid crystal module with uneven brightness. Before the brightness of the LCD is not corrected, the brightness of the panel is not uniform, and after the panel is shielded by foreign matters, the gray level of an area which is shot by a camera and shielded by the foreign matters is relatively low, the gradient of the area at the boundary of the foreign matters is relatively large, and because the brightness of the LCD without brightness is not uniform, but the local brightness change of the LCD is relatively smooth; and the method has a better segmentation effect according to the brightness of the foreign matters and the gray difference value of the local blocks.

In practical application, when a marking value is set for each pixel in an image after binarization processing, specifically, the marking value is set on the basis of an index value of the pixel, and a foreground pixel and a background pixel are distinguished, wherein the foreground pixel refers to a foreign matter, the background pixel refers to a background of a module, and pixels processed in a subsequent step are mainly pixels of the foreign matter.

In this embodiment, the binarization processing is to specifically adjust a display gray difference value of an image, distinguish a foreground of a foreign object from a background of a module in the image based on the adjusted gray difference value, then segment the image with the adjusted gray according to pixels to obtain a binarized image, sort each pixel from left to right to obtain an index value of each pixel, and obtain the binarized image by using the index value as a label value of the pixel.

Specifically, the image obtained by shooting the display module is subjected to area division, and the gray level mean value of each area is calculated based on the result of the area division; setting a corresponding self-adaptive threshold according to the gray average value of the region; calculating the gray level of the pixels in each area, and comparing the gray level of each pixel with the adaptive threshold of the area where the pixel is located to obtain a comparison result; and taking the pixels with the gray levels of the pixels smaller than the self-adaptive threshold in the comparison result as a foreground, taking the other pixels as a background, and setting the initial mark value of the foreground pixels in the foreground as the index value of the foreground pixels to obtain a binary image.

In practical application, firstly, shooting the display module image, then carrying out region division on the display module image to obtain the gray average value of each region, and setting a self-adaptive threshold according to the region gray average value;

taking pixels as a dividing unit, comparing the gray level of each pixel with a threshold value of an area where the pixel is located, taking the pixel with the gray level smaller than the threshold value as a foreground, and taking the rest pixels as a background;

and extracting the index value of each pixel, and setting an initial mark value for the corresponding pixel according to the index value of each pixel to obtain a binary image.

202. Determining a root mark value of each pixel according to an initial mark value of the pixel in the binary image by using a parallel connected domain marking algorithm carried out by parallel searching, and determining a foreign matter position based on the root mark value;

in the step, the pixels marked by the parallel connected component marking algorithm are the pixels of foreign matters, and the pixels are called foreground pixels; the characteristic of the data structure of the search set is that it can quickly search and merge, the root node of the set is characterized in that the index value of the root node is equal to the mark value; and performing parallel query and combination processing on each pixel in the binary image through the algorithm.

In this embodiment, when a parallel connected domain labeling algorithm is used to set a label value for a pixel in a binarized image, the method specifically includes four connected segments, which are initialization, search, merging, and re-search, and an implementation flow thereof is shown in fig. 5. Further, the method also comprises the step of identifying the pixels of the foreign matters in the binary image, namely the foreground pixels, before the initialization is carried out.

In the initialization stage, each foreground pixel is searched for a neighborhood, a root mark value is set based on the searched neighborhood, specifically, a pixel at the upper left corner in the binary image is used as a first mark value, namely, a pixel with the minimum index value in the binary image is used as the initialized first mark value, and whether the pixel (namely, the foreground pixel) has a neighborhood or not is inquired, wherein the neighborhood refers to other foreground pixels communicated with the foreground pixel; if the foreground pixel neighborhood has no other connected foreground pixels, the index of the foreground pixel is used as the marking value of the foreground pixel, and the background pixels are marked as-1 (the index value of the C language starts from 0); then, a set is established for each pixel according to the queried neighborhood, the set element has the pixel itself and the pixel with the minimum pixel index value, and specifically, when the neighborhood is queried for each pixel, the query is performed according to the label graph in fig. 4.

In the query stage, root searching is mainly performed on each pixel on the basis of the initialized binary image, and the marking value on the corresponding pixel is modified into the marking value of the root node obtained by root searching on the basis of the root searching result.

The merging stage is mainly to carry out root finding on pixels which have different mark values but belong to adjacent (namely neighborhood) in the binary image obtained after the query stage is finished, and then merge a larger root node set into a smaller root node set.

And in the re-query stage, the root searching is carried out on the binary image obtained after the merging stage is finished, at the moment, the pixels in the same connected domain use the marking information of the same root node, and the marking task is finished.

In practical application, the step can be realized by: identifying the adjacent relation among the pixels in the binary image, and constructing an equivalence pair sequence based on the adjacent relation;

performing iterative root finding on each equivalent pair in the equivalent pair sequence to obtain a plurality of pixel dendrograms;

judging whether root nodes in the pixel dendrograms are adjacent or not;

if so, merging the corresponding pixel tree graphs, and setting the root mark value of each pixel in the binary image based on the merged pixel tree graphs;

performing secondary root searching on the adjusted binary image to obtain a root searching result;

if the root searching result shows that the same root node exists, merging the adjusted tree graphs;

and if the root searching result shows that the same root node does not exist, determining the position of the foreign object based on the root mark value.

203. Based on the foreign matter position, the number and the area of the foreign matters of the image are counted through the Hash table, the actual number of the foreign matters in the image and the area of each foreign matter are calculated, the detection result of the foreign matters in the display module is generated, and the foreign matters in the image are processed in an image interpolation mode according to the detection result and the current production requirement.

Specifically, the step is to actually count the foreign matters after the marking is completed, and specifically includes calculating the area of a connected domain corresponding to each foreign matter, and identifying whether a connected relationship still exists between the connected domains, if yes, merging the connected domains to form a complete connected domain of the foreign matter, and calculating the total area of the merged connected domains; and finally, calculating the number of the merged connected domains to obtain the actual number of the foreign matters, generating a foreign matter detection result of the display module based on the total area and the actual number, wherein the foreign matter detection result can be specifically output in a detection report form, and the detection result of actual production is used for judging whether the image with the foreign matters is avoided through image interpolation or carries out foreign matter alarm.

In the embodiment of the invention, a parallel connected domain marking algorithm based on parallel searching is utilized, a root marking value is set according to the marking position of each pixel in a binary image obtained by shooting a display module, the position of a foreign matter is determined based on the root marking value, and then the detection result of the display module is generated. Meanwhile, the running time of the foreign matter detection subsystem is greatly shortened, the requirement of actual production can be met, and the method is suitable for industrial production environments with high requirements on instantaneity and stability.

Referring to fig. 3, a second embodiment of the method for detecting a foreign object in a display module according to the present invention is illustrated by taking a 4 × 4 divided image as an example, and the method includes the following steps:

301. calculating the gray level of each pixel on the image obtained by the shooting display module, and carrying out binarization processing on the image based on the gray level of each pixel and a preset adaptive threshold to obtain a binarized image;

in this step, generating the binarized image may be specifically realized by the following method:

dividing the image obtained by shooting the display module into areas to obtain the gray average value of each area, and setting an adaptive threshold according to the area gray average value;

comparing the gray level of each pixel with a threshold value of an area where the pixel is located by taking the pixel as a dividing unit, taking the pixel with the gray level smaller than the threshold value as a foreground, and taking the rest pixels as a background to obtain a binary image;

further, before generating the binary image, extracting the index value of each foreground pixel in the foreground, setting an initial mark value for the corresponding foreground pixel according to the index value of each foreground pixel, and finally obtaining the binary image.

In practical application, binarization processing is to distinguish gray level difference values of a foreground and a background in an image, because brightness of a module without brightness correction is not uniform, especially when a foreign matter exists, the foreign matter and the background of the module have a large brightness difference, and the foreign matter shot in the image needs to be segmented from the background of a liquid crystal module with non-uniform brightness, a better segmentation effect is achieved according to the gray level difference value of the gray level of the foreign matter and the gray level average value of a local block, specifically, a certain proportion of the gray level average value of the local block is used as a threshold value threshold to separate the foreign matter from the background, then foreign matter identification processing is performed on the adjusted image, preferably, pixel segmentation can be performed on the image firstly, pixels of the foreign matter are extracted, and a mark value is set on the extracted pixels.

When setting the index value, specifically, the index value is set according to the index value obtained in the sorting process of the pixels, as shown in fig. 6, the index value of each pixel is different, and the index values sequentially increase from left to right when the index value is set. The binary image of 4 × 4 in the figure is marked with mark values 1-16 respectively, wherein the circles of the white background represent the foreground, i.e. the pixels of the foreign matter, the circles of the black background represent the background, i.e. the pixels of the module background, and the mark value of each pixel is equal to the array index value thereof.

302. Configuring at least two parallel threads, identifying whether each foreground pixel in the binary image is communicated with other foreground pixels, and initializing a mark value in the binary image based on an identification result to obtain an initialized binary image and at least one foreground pixel communicated dendrogram;

in this step, at least two parallel threads are configured, the specific number of threads is equal to the number of pixels, each thread executes identification and initialization processing of one pixel, and the initialization processing can be specifically realized by the following method:

identifying whether each pixel in the binary image is communicated with other pixels;

if other pixels are communicated, keeping the mark value of the current pixel, and modifying the mark value of the pixel communicated with the current pixel into the mark value of the current pixel to obtain an initialized binary image;

and constructing at least one pixel connected dendrogram based on the mark value of each pixel in the initialized binary image.

When identifying whether each pixel in the binary image is connected with other pixels, specifically, whether the gray values of the adjacent pixels of each pixel are the same or not is identified, if so, the two pixels are considered to be connected, otherwise, the two pixels are not connected, the method is used for identifying, whether each pixel is connected with the adjacent pixels or not is judged, if so, the two pixels are determined to be the same foreign matter, the mark values of the other pixels are set based on the mark value of the pixel, for example, whether the pixel 1 is connected with other pixels or not is identified, if so, the mark values of the pixel 1 and the other pixels are compared, the smaller one is selected as the mark values of the pixel 1 and the other pixels, and certainly, the identification is carried out according to the index value from small to large when the connection is identified, so that the index value of the current pixel is directly modified as the mark values of the other pixels after the connection is determined to exist.

And outputting the binary image until the identification of the left and right pixels in the binary image is finished to obtain an initialized binary image, meanwhile, constructing a set with connected pixels based on the identification and connection result, and converting the set into a pixel connected tree-shaped image.

In practical application, in the process of querying the initialized binary graph, the pixel label value takes the value with the minimum index value as the label value according to the search rule of fig. 4, and after querying, the relationship between the pixel label value and the pixel connected tree graph is shown as (a) and (b) in fig. 7.

303. Carrying out root searching on each foreground pixel of the initialized binary image to obtain a root node of each foreground pixel;

in this step, during the root finding process, specifically, the root is found based on the communication between the pixels, for example, the pixel 2 is communicated with the pixel 5, the pixel 5 is communicated with the pixel 10, the pixel 10 is communicated with the pixel 13, and the flag value on the pixel 5 is 2, then the root node of the pixel 5 is the pixel 2, and similarly, the root nodes of the

pixels

10 and 13 are both the pixel 2, so as to obtain the root nodes of the

pixels

5, 10, and 13.

304. Modifying the mark values of all foreground pixels with the same root node into the mark value of the root node to obtain a query binary image;

and modifying the label values of all the connected pixels into the minimum label value in the root node based on the relation between the root node and each pixel obtained by root searching in the steps, and outputting an inquiry binary graph.

305. Taking the root node obtained by root finding as an initial node, and converting each foreground pixel connected tree graph into a root tree graph with the depth of 1;

in this embodiment, the initialized binary map and at least one pixel connected tree map obtained in step 302 are subjected to query processing, the initialized binary map and the at least one pixel connected tree map are merged through the query processing, and pixels of the foreign objects having a connected relationship are labeled with the same label value, so as to obtain a query binary map.

Further, after the root node of each pixel is determined, the connection relationship of the original pixel connected tree graph is adjusted to generate a root tree graph with the depth of 1, for example, if the root nodes of the

pixels

5, 10 and 13 are all the pixel 2, the pixel connected tree graph of 2-5-10-13 in fig. 7 is modified into the pixel connected tree graphs of 2-5, 2-10 and 2-13, as shown in (a) and (b) in fig. 8.

306. Setting and inquiring a root label value of a foreground pixel in the binary image based on the root tree image to obtain a combined binary image;

specifically, judging whether the initial nodes in each root tree graph belong to adjacent pixels;

if so, merging the root tree graph with the larger mark value of the starting node into the root tree graph with the smaller mark value of the starting node, and setting the root mark value of the pixel in the query binary graph based on the mark value of each pixel in the merged root tree graph to obtain a merged binary graph.

In practical application, a merged binarization graph is generated, wherein the part of initialization is introduced is regarded as a non-root node of a root node, and the main method is to modify the label value of the non-root node to make the label value of the non-root node be the label value of the real root node. As the label value of the pixel in the query binarized map is modified based on the root tree diagram in fig. 8, i.e., 2-5, 2-10 and 2-13, the label values on the pixel 10 and the pixel 13 are modified to the label value of the pixel 2 due to the relationship of 2-10 and 2-13, thereby outputting the merged binarized map, as shown in (a) and (b) in fig. 9.

307. Determining the position of a foreign matter according to the root mark value of each foreground pixel in the combined binary image;

in this embodiment, in order to optimize consistency of the mark values of the foreign object, before determining the position of the foreign object, the method further includes:

308. Counting the number and the area of the foreign matters in the image through a hash table based on the positions of the foreign matters, calculating the actual number of the foreign matters in the image and the area of each foreign matter, and generating a detection result of the foreign matters in the display module;

309. and processing the foreign matters in the image in an image interpolation mode according to the detection result and the current production requirement.

In the step, a connected region is determined based on the position of the foreign matter of the pixel, and the number of the connected regions is counted to obtain the actual number of the foreign matter;

generating a detection result of the foreign matters in the display module based on the actual number and the area of the foreign matters;

and based on the detection result and the current production requirement, avoiding or giving an alarm in an image interpolation mode.

In practical application, the detection result is generated by mainly counting the sizes and the numbers of the foreign object foregrounds (namely, connected regions with different mark values) after being distinguished in parallel.

After the connected region is marked, the same label is marked on the same connected region, and the connected region is the minimum pixel index value of the connected region, the number of the foreign matters is relatively small (generally, the number of the foreign matters does not exceed 50) for the whole LCD screen, and if the number is too large, a warning is directly sent out without further detection. The invention adopts a data structure of a hash table to store the quantity and the size of the foreign matters, key values are mark values of a connected region, and correspondingly stored values are the number of pixels corresponding to the mark values.

Because the hash table is a non-thread-safe data structure, that is, when hash collision occurs, thread collision occurs; for example, a thread conflict in a hash collision occurs when two different tag values, simultaneously mapped to the same address and occupy the address. Resolving this problem requires conflict-free access and occupies the address empty; such as the atomic operation provided by unified computing architecture (CUDA) of great corporation, can solve this problem well.

Through the implementation of the method, after the shot image of the display module is subjected to binarization processing, a mark value is marked on each pixel to obtain a binarized image, then the root mark value of each pixel in the binarized image is identified based on a parallel connected domain marking algorithm which is carried out by parallel searching, the image is subjected to foreign matter identification detection based on the root mark value to obtain a detection result, the parallel connected domain marking algorithm realizes parallel foreign matter segmentation, parallel connected domain marking and parallel foreign matter quantity and size statistics, and the time required by foreign matter detection in production is greatly reduced.

With reference to fig. 10, the method for detecting a foreign object in a display module according to an embodiment of the present invention is described above, and a foreign object detection apparatus in a display module according to an embodiment of the present invention is described below, where:

a binarization processing module 410, configured to calculate a gray level of each pixel on an image obtained by shooting the display module, and perform binarization processing on the image based on the gray level of each pixel and a preset adaptive threshold to obtain a binarized image, where the adaptive threshold is a gray level threshold value obtained based on a gray level average value of the image;

a labeling module 420, configured to determine a root label value of each pixel according to an initial label value of a pixel in the binarized image by using a parallel connected domain labeling algorithm performed by a parallel search set, and determine a foreign object position based on the root label value;

the detection module 430 is configured to count the number and area of the foreign matters in the image through a hash table based on the position of the foreign matters, calculate the actual number of the foreign matters in the image and the area of each foreign matter, generate a detection result of the foreign matters in the display module, and process the foreign matters in the image in an image interpolation manner according to the detection result and the current production demand.

According to the device provided by the embodiment, the root mark value is set according to the mark position of each pixel in the binary image obtained by shooting the display module by using the parallel connected domain marking algorithm based on the parallel search set, the foreign matter position is determined based on the root mark value, the detection result of the display module is generated, the parallel connected domain marking algorithm realizes parallel foreign matter segmentation, parallel connected domain marking and parallel foreign matter quantity and size statistics, and the time required by foreign matter detection in production is greatly reduced.

Further, please refer to fig. 11, wherein fig. 11 is a detailed schematic diagram of each module of the foreign object detection apparatus of the display module.

In another embodiment of the present invention, the binarization processing module 410 includes:

the dividing unit 411 is configured to perform area division on an image obtained by shooting the display module, and calculate a grayscale mean value of each area based on a result of the area division;

a setting unit 412, configured to set a corresponding adaptive threshold according to the gray level mean of the region;

a comparison unit 413, configured to calculate gray levels of pixels in each region, and compare the gray level of each pixel with a self-adaptive threshold of the region where the pixel is located, to obtain a comparison result;

a binarization unit 414, configured to use a pixel in the comparison result whose gray level is smaller than the adaptive threshold as a foreground, and use the remaining pixels as a background, and set an initial flag value of a foreground pixel in the foreground as an index value of the foreground pixel, so as to obtain a binarized image.

In another embodiment of this embodiment, the marking module 420 includes:

an initialization unit 421, configured to configure at least two parallel threads, identify whether each foreground pixel in the binarized image has other foreground pixels connected, and perform initialization processing on a tag value in the binarized image based on an identification result to obtain an initialization tag map group, where the initialization tag map group includes an initialization binarized map and at least one foreground pixel connected dendrogram;

a first querying unit 422, configured to perform root finding on each foreground pixel of the initialized binary image, to obtain a root node of each foreground pixel; modifying the mark values of all foreground pixels with the same root node into the mark values of the root node to obtain a query binary image;

a merging unit 423, configured to use a root node obtained by root finding as an initial node, convert each foreground pixel connected dendrogram into a root dendrogram with a depth of 1, and set a root label value of a foreground pixel in the query binarized graph based on the root dendrogram, so as to obtain a merged binarized graph;

and a marking unit 424, configured to determine a foreign object position according to the root mark value of each foreground pixel in the merged binary image.

In another embodiment of this embodiment, the marking module 420 further includes: a second query unit 425, configured to:

performing secondary root searching on the combined binary image to obtain a root searching result;

In another embodiment of this embodiment, the initialization unit 421 is specifically configured to:

identifying whether each foreground pixel in the binary image is communicated with other foreground pixels or not;

and constructing at least one foreground pixel connected dendrogram based on the mark value of each foreground pixel in the initialized binary image.

In another embodiment of this embodiment, the merging unit 423 is specifically configured to:

judging whether the starting nodes in each root dendrogram belong to adjacent foreground pixels or not;

In another embodiment of this embodiment, the detecting module 430 includes:

the table construction unit 431 is configured to calculate the number and the area of the same mark value according to the root mark value corresponding to the foreign object position, and construct a foreign object hash table;

a quantity calculation unit 432, configured to determine a connected region based on the foreign object position of the pixel, and count the quantity of the connected regions to obtain an actual quantity of the foreign objects;

an area calculation unit 433 configured to calculate the number of pixels in each connected region, and calculate the area of the corresponding foreign object based on the number;

a detecting unit 434, configured to generate a detection result of the foreign object in the display module based on the actual number and the area of the foreign object; and avoiding or giving an alarm in an image interpolation mode based on the detection result and the current production requirement.

Through the implementation of the device, after the shot image of the display module is subjected to binarization processing, each pixel is marked with a mark value to obtain a binarized image, then the root mark value of each pixel in the binarized image is identified based on a parallel connected domain marking algorithm carried out by parallel searching, and the image is subjected to foreign matter identification detection based on the root mark value to obtain a detection result.

Fig. 10 and fig. 11 describe the foreign object detection apparatus of the display module in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the electronic device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

Fig. 12 is a schematic structural diagram of an electronic device 800, which may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 810 (e.g., one or more processors) and a memory 820, and one or more storage media 830 (e.g., one or more mass storage devices) storing an application 833 or data 832. Memory 820 and storage medium 830 may be, among other things, transitory or persistent storage. The program stored in the storage medium 830 may include one or more modules (not shown), each of which may include a sequence of instructions operating on the electronic device 800. Further, the processor 810 may be configured to communicate with the storage medium 830 and execute a series of instruction operations in the storage medium 830 on the electronic device 800. In practical applications, the application 833 may be divided into the functions of the binarization processing module 410, the labeling module 420, and the detection module 430 (modules in a virtual device).

Electronic device 800 may also include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input-output interfaces 860, and/or one or more operating systems 831, such as: windows Server, macOSX, unix, linux, freeBSD, etc. Those skilled in the art will appreciate that the electronic device structure shown in fig. 8 may also include more or fewer components than shown, or combine certain components, or a different arrangement of components.

An embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements each step in the method for detecting a foreign object of a display module provided in the foregoing embodiment when executing the computer program.

The embodiment of the present invention further provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, where instructions or a computer program are stored in the computer-readable storage medium, and when the instructions or the computer program are executed, the computer executes the steps of the method for detecting the foreign object of the display module provided in the foregoing embodiment.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses, and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A foreign matter detection method of a display module is characterized by comprising the following steps:

2. The method for detecting the foreign object in the display module according to claim 1, wherein the calculating and shooting the gray scale of each pixel on the image obtained by the display module, and performing binarization processing on the image based on the gray scale of each pixel and a preset adaptive threshold to obtain a binarized image comprises:

3. The method for detecting the foreign matter of the display module according to claim 2, wherein the parallel connected component labeling algorithm using the parallel search set determines a root label value of each pixel according to an initial label value of the pixel in the binarized image, and determines the foreign matter position based on the root label value, and comprises:

taking a root node obtained by root finding as an initial node, converting each foreground pixel communication dendrogram into a root dendrogram with the depth of 1, and setting a root label value of foreground pixels in the query binarized graph based on the root dendrogram to obtain a combined binarized graph;

4. The method for detecting the alien material of the display module according to claim 3, wherein before determining the alien material position according to the root label value of each foreground pixel in the merged binary image, the method further comprises:

5. The method for detecting the foreign object on the display module according to claim 4, wherein the identifying whether each foreground pixel in the binarized image has other foreground pixels connected with each other and performing initialization processing on the label value in the binarized image based on the identification result to obtain an initialization label map set comprises:

6. The method for detecting the alien material of a display module according to claim 4, wherein the setting the root label value of the foreground pixel in the query binary map based on the root tree map to obtain the merged binary map comprises:

7. The method for detecting the alien material on the display module according to any one of claims 1 to 6, wherein the method comprises the steps of counting the number and the area of the alien material in the image by a hash table based on the position of the alien material, calculating the actual number of the alien material in the image and the area of each alien material, generating a detection result of the alien material in the display module, and processing the alien material in the image by image interpolation according to the detection result and the current production demand, and comprises the following steps:

8. The utility model provides a display module assembly's foreign matter detection device which characterized in that, foreign matter detection device includes:

the marking module is used for determining the root marking value of each pixel according to the initial marking value of the pixel in the binary image by utilizing a parallel connected domain marking algorithm carried out by parallel searching, and determining the position of the foreign matter based on the root marking value;

9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for detecting the foreign object of the display module according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the steps of the method for detecting a foreign object in a display module according to any one of claims 1 to 7.