CN117710303A - Display screen defect AI (advanced technology attachment) re-judging method, device and medium based on image merging - Google Patents

Abstract

The invention provides a display screen defect AI (advanced technology attachment) restoration method, device and medium based on image combination, wherein the method comprises the following steps: acquiring an AOI detection result of a product to be detected, and cutting and naming an image with a defect in the detection result; combining the cut undetermined defect picture with the corresponding polishing picture; labeling the combined graph dataset and performing AI model training; and performing repeated judgment on the combined images through the deep neural network model to obtain a judgment result. The invention can combine the lighting display screen detection logic to carry out the re-judgment with higher accuracy, can well solve the over-detection problems of frequent replacement and complex product surface state, is beneficial to stable detection, improves the through rate, reduces the re-judgment personnel, reduces the cost, effectively realizes the faster and more stable display screen quality inspection, and has high accuracy.

Description

Display screen defect AI (advanced technology attachment) re-judging method, device and medium based on image merging

Technical Field

The invention relates to the technical field of display screen detection, in particular to a display screen defect AI (advanced technology attachment) restoration method, device and medium based on image combination.

Background

The field of display screen lighting detection has gradually transitioned from a main small-sized screen to a medium-and large-sized display screen. Because the orders of a single model of the medium-size and large-size display screens are less than those of a small-size display screen and most of the display screens are not provided with glass cover plates, the frequency of the change production is faster, and the surface state of the product is more complex. This conversion requires faster and more stable machine vision for screen quality inspection, which is required by traditional algorithmic detection, requiring very specialized personnel for maintenance. Therefore, how to realize faster and more stable quality inspection of the display screen is a problem that needs to be solved in the machine vision industry.

Aiming at the problems, the detection results of the conventional algorithm are subjected to repeated judgment by the AI, and the scheme can solve the problems of frequent and rapid model change and over-detection brought by complex surface states. However, for the lighting detection of the display screen, the accuracy cannot be ensured by simply judging whether the display screen is defective or not by means of the image features of the picture, because the distinction between the partial surface object and the defective features is very low.

Disclosure of Invention

To achieve the above and other advantages and in accordance with the purpose of the present invention, a first object of the present invention is to provide a display screen defect AI restoration method based on image merging, comprising the steps of:

acquiring an AO I detection result of a product to be detected, and cutting and naming an image with a defect detection result;

combining the cut undetermined defect picture with the corresponding polishing picture;

labeling the combined graph dataset and performing AI model training;

and performing repeated judgment on the combined images through the deep neural network model to obtain a judgment result.

Further, the AOI detection of the product to be detected comprises the following steps:

responding to a request for scanning the two-dimensional code of the product to be detected, and enabling the product to be detected to enter AO I detection.

Further, the step of obtaining the AOI detection result of the product to be detected, and cutting and naming the image with the defect detection result comprises the following steps:

obtaining coordinates and pictures of an image with a defect as a detection result, and cutting preset size block data around the undetermined defect on a corresponding scale map, wherein the block data comprise picture map and lighting map data corresponding to the detected undetermined defect;

and naming the cut pictures by using the two-dimensional code, the defect name, the defect coordinate, the defect picture or the polishing picture of the product to be detected, and storing the pictures in folders corresponding to the product to be detected.

Further, the merging the cut undetermined defect picture image with the corresponding lighting image comprises the following steps:

determining an object to be merged according to the naming of the cut image;

combining the picture image of the defect to be determined and the lighting image of the corresponding coordinate into a group of images;

and combining the two corresponding images according to the defect picture images of two channels of the R channel, the G channel and the B channel and the bright image of one channel.

Further, the determining the object to be merged based on the naming when cropping the image comprises the following steps:

matching the two-dimensional code, the name of the undetermined defect, the coordinates of the undetermined defect and the images of the undetermined defect with the same two-dimensional code, the name of the undetermined defect, the coordinates of the undetermined defect and the polishing image data of the undetermined defect;

and if the two-dimensional code, the name of the undetermined defect and the coordinates of the undetermined defect are consistent, determining the image of the undetermined defect and the polishing image as a group of combined images.

Further, the labeling of the merged graph dataset and the AI model training comprises the following steps:

screening and distinguishing the merging graph data sets according to the true and false defects;

uploading the screened and distinguished true and false defect data set to a training platform;

training and testing the deep neural network model by the training platform through the defect data set;

and uploading the trained and tested deep neural network model to an AI server.

Further, the performing the re-judgment on the combined image through the deep neural network model includes the following steps:

during online detection, uploading the combined defect image to be determined to an AI server;

and the AI server automatically judges the result of the combined undetermined defect image through training the tested deep neural network model.

Further, the method also comprises the following steps:

feeding back the result to the AOI device;

and the AOI equipment controls products with good products and bad falsification to flow into the next link according to the feedback judging result, and only transmits the genuine defective products to the buffer.

A second object of the present invention is to provide a computer-readable storage medium having stored thereon program instructions that, when executed, implement a display screen defect AI-restoration method based on image merging.

The third object of the present invention is to provide a display screen defect AI-rendition device based on image merging, which is used for implementing the above method, and comprises an AOI unit, an image merging unit, an AI model training unit, and an AI rendition unit; wherein,

the AOI unit is used for calculating the picture image of the display screen shot by the industrial camera through an image processing algorithm to obtain a detection result, and cutting and naming the image with the defect detection result;

the image merging unit is used for merging the small images after cutting according to the defect picture images of two channels of the R channel, the G channel and the B channel and the lighting image of one channel, and transmitting the merged images to the AI complex judgment unit;

the AI model training unit is used for carrying out deep neural network model training through marked true and false defect data sets under the offline condition, optimizing a deep neural network model and updating the deep neural network model to the AI re-judging unit;

the AI re-judging unit is used for receiving the combined images, judging the images based on a preset deep neural network model and feeding back the judging result to the AOI unit during online detection.

Compared with the prior art, the invention has the beneficial effects that:

the invention provides a display screen defect AI (advanced technology) re-judging method, device and medium based on image combination, which can combine with lighting display screen detection logic to carry out re-judgment with higher accuracy, can well solve the problem of frequent change and over-detection brought by complex surface states of products, is beneficial to stable detection, improves the through rate, reduces re-judging personnel, reduces the cost, effectively realizes faster and more stable display screen quality inspection, and has high accuracy.

The foregoing description is only an overview of the present invention, and is intended to provide a better understanding of the present invention, as it is embodied in the following description, with reference to the preferred embodiments of the present invention and the accompanying drawings. Specific embodiments of the present invention are given in detail by the following examples and the accompanying drawings.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:

FIG. 1 is a block diagram of an AI rendition device based on image merging for display screen defects in accordance with embodiment 1;

FIG. 2 is a flowchart of a display screen defect AI re-judging method based on image merging in embodiment 2;

FIG. 3 is a flow chart of AOI detection of the product under test of example 2;

FIG. 4 is a flow chart of example 2 for obtaining AOI detection results of a product to be detected, and cutting and naming images with defects in the detection results;

FIG. 5 is a flow chart of combining the cut undetermined defect picture with the corresponding polishing picture in embodiment 2;

FIG. 6 is a flowchart for determining objects to be merged based on naming when cropping images according to embodiment 2;

FIG. 7 is a flowchart for labeling a merged graph dataset and performing AI model training in accordance with example 2;

FIG. 8 is a flowchart of the re-judgment of the combined image by the deep neural network model in embodiment 2;

fig. 9 is a flowchart of control of the AOI device according to embodiment 2 according to the determination result;

fig. 10 is a schematic diagram of a storage medium of embodiment 3.

Detailed Description

The present invention will be further described with reference to the accompanying drawings and detailed description, wherein it is to be understood that, on the premise of no conflict, the following embodiments or technical features may be arbitrarily combined to form new embodiments.

Example 1

The display screen defect AI re-judging device based on image combination comprises an AOI (Automated Opt ica l I nspect ion, automatic optical detection) unit, an image combination unit, an AI model training unit and an AI re-judging unit as shown in figure 1; wherein,

the AOI unit is used for calculating the picture image of the display screen shot by the industrial camera through an image processing algorithm to obtain a detection result, and cutting and naming the image with the defect (NG) as the detection result;

in this embodiment, a two-dimensional code of a product to be detected is scanned, the product to be detected enters AOI detection, coordinates and pictures detected as NG result are obtained, 256x256 size block data around a defect to be determined is cut on a corresponding scale map, and the block data includes: and correspondingly detecting the picture image and the polishing image data of the undetermined defect. And naming the cut pictures by the two-dimensional code, the defect name, the defect coordinate, the defect picture or the polishing picture of the product, and storing the cut pictures in folders corresponding to the detected products.

The image merging unit is used for merging the corresponding cut small images (the defect picture image and the polishing image) according to the defect picture image of two channels of the R channel, the G channel and the B channel, wherein one channel is the polishing image, for example, the R channel is the defect picture image, the G channel is the polishing image, the B channel is the defect picture image, and the merged images are transmitted to the AI (analog) complex judging unit;

combining the two corresponding images into a group, namely combining the picture image of the undetermined defect and the polishing image of the corresponding coordinate into a group of images, and determining an object to be combined according to the naming of the cut images, namely matching the same two-dimensional code, the name of the undetermined defect, the coordinate of the undetermined defect and the picture image of the undetermined defect with the polishing image data of the undetermined defect. If the two-dimensional code, the name of the undetermined defect and the coordinates of the undetermined defect are consistent, determining the undetermined defect picture image and the polishing image as a group of combined objects, and combining two corresponding images according to the R channel as the defect picture image, the G channel as the polishing image and the B channel as the defect picture image.

The AI model training unit is used for carrying out deep neural network model training through marked true and false defect data sets under the offline condition, optimizing the deep neural network model and updating the deep neural network model to the AI re-judging unit;

the defect data sets are screened and distinguished according to the true and false defects, the screened and distinguished true and false defect data sets are uploaded to a training platform, the training platform trains and tests the deep neural network model through the defect image data sets, and the deep neural network model which passes the training and testing is uploaded to an AI server again.

And the AI re-judging unit is used for receiving the combined images, judging the images based on a preset deep neural network model and feeding back the judging result to the AOI unit during online detection.

In this embodiment, the online detection uploads the combined pending defect image to the AI server, the AI server automatically determines a result based on the deep neural network model passing the training test, and feeds back the result to the AOI device, and the AOI device automatically flows good products and false products into the next link according to the fed back result, and only transfers the true defective products to a Buffer (Buffer) for processing by on-site production personnel.

The lighting detection logic of the display screen generally uses all suspected defects visible on the lighting picture to remove dust, scratches, bubbles and other interference visible on the surface of the lighting picture, so as to obtain the true defects of the display screen. The embodiment combines the images and then carries out AI re-judgment, and exactly fits the lighting detection logic of the display screen, thereby providing effective logic guarantee for the accuracy of AI re-judgment.

The display screen defect AI re-judging device based on image combination can combine the lighting display screen detection logic to carry out re-judging with higher accuracy, can well solve the problem of frequent mold changing and over-detection brought by complex surface states of products, is beneficial to stable detection, improves the through rate, reduces re-judging personnel and reduces cost.

Example 2

The detailed description of the device may refer to the corresponding description in the above device embodiment, and will not be repeated here. As shown in fig. 2, the method comprises the steps of:

s1, acquiring an AOI detection result of a product to be detected, and cutting and naming an image with a defect detection result;

before the AOI detection result of the product to be detected is obtained, the AOI detection is required to be carried out on the product to be detected.

In this embodiment, as shown in fig. 3, the AOI test of the product to be tested includes the following steps:

s0, responding to a request for scanning the two-dimensional code of the product to be detected, and enabling the product to be detected to enter AOI detection. Namely, scanning the two-dimensional code of the product to be detected, and enabling the product to be detected to enter AOI detection.

In this embodiment, the AOI unit may use a conventional image processing algorithm to calculate a display screen image captured by an industrial camera, obtain a detection result, and crop and name an image whose detection result is NG. As shown in fig. 4, obtaining an AOI detection result of a product to be detected, and clipping and naming an image whose detection result is defective, includes the following steps:

s11, acquiring coordinates and pictures of an image with a defect as a detection result, and cutting out block data with preset size (for example, the preset size is 256x 256) around the defect on a corresponding scale map, wherein the block data comprises picture map and lighting map data corresponding to the detected defect;

s12, naming the cut pictures by the two-dimensional code, the defect name, the defect coordinate, the defect picture or the polishing picture of the product to be detected, and storing the pictures in folders corresponding to the product to be detected.

S2, merging the cut undetermined defect picture with the corresponding polishing picture; as shown in fig. 5, the method specifically comprises the following steps:

s21, determining an object to be combined according to the naming of the cut image; as shown in fig. 6, the method specifically comprises the following steps:

s211, matching the two-dimensional code, the undetermined defect name, the undetermined defect coordinate and the undetermined defect picture with the same two-dimensional code, the undetermined defect name, the undetermined defect coordinate and the polishing image data of the undetermined defect;

s212, judging whether the two-dimensional code, the undetermined defect name and the undetermined defect coordinate are consistent;

s213, if the two-dimensional code, the name of the undetermined defect and the coordinates of the undetermined defect are consistent, determining the image of the undetermined defect and the polishing image as a group of combined images;

and S214, if the two-dimensional code, the name of the undetermined defect and the coordinates of the undetermined defect are inconsistent, determining the image of the undetermined defect and the polishing image as a group of images which can not be combined.

S22, combining the picture image of the defect to be determined and the polishing picture image of the corresponding coordinates into a group of images, namely combining the two corresponding pictures into a group;

s23, two channels of the R channel, the G channel and the B channel are taken as defect picture images, one channel is taken as a lighting image, for example, the R channel is taken as the defect picture image, the G channel is taken as the lighting image, the B channel is taken as the defect picture image, and the corresponding two images are combined.

S3, labeling the combined graph data set and performing AI model training; as shown in fig. 7, the method specifically comprises the following steps:

s31, screening and distinguishing the merging graph data set according to true and false defects;

s32, uploading the true and false defect data sets distinguished by screening to a training platform;

s33, training and testing the deep neural network model through the defect data set by the training platform;

and S34, uploading the trained and tested deep neural network model to an AI server.

S4, performing repeated judgment on the combined images through the deep neural network model to obtain judgment results.

As shown in fig. 8, the method specifically comprises the following steps:

s41, uploading the combined defect image to an AI server during online detection;

s42, the AI server automatically judges the result of the combined undetermined defect image through training the tested deep neural network model.

After the AI is repeated, as shown in fig. 9, the embodiment further includes the following steps:

s43, feeding back the result to the AOI equipment;

s44, the AOI equipment controls products with good products and bad falsifiability to flow into the next link according to the feedback judging result, and only conveys the genuine defective products to a Buffer (Buffer) for processing by on-site production personnel.

The lighting detection logic of the display screen generally uses all suspected defects visible on the lighting picture to remove dust, scratches, bubbles and other interference visible on the surface of the lighting picture, so as to obtain the true defects of the display screen. The embodiment combines the images and then carries out AI re-judgment, and exactly fits the lighting detection logic of the display screen, thereby providing effective logic guarantee for the accuracy of AI re-judgment.

The display screen defect AI re-judging method based on image combination can combine the lighting display screen detection logic to perform re-judgment with higher accuracy, can well solve the problem of frequent mold changing and over-detection brought by complex surface states of products, is beneficial to stable detection, improves the through rate, reduces re-judgment personnel and reduces cost.

Example 3

A computer readable storage medium having stored thereon program instructions that when executed implement a display screen defect AI-restoration method based on image merging, as shown in fig. 10. For detailed description of the method, reference may be made to corresponding descriptions in the above method embodiments, and details are not repeated here.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments.

The foregoing is illustrative of the embodiments of the present disclosure and is not to be construed as limiting the scope of the one or more embodiments of the present disclosure. Various modifications and alterations to one or more embodiments of this description will be apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, or the like, which are within the spirit and principles of one or more embodiments of the present disclosure, are intended to be included within the scope of the claims of one or more embodiments of the present disclosure.

Claims (10)

1. The display screen defect AI (advanced technology attachment) restoration method based on image combination is characterized by comprising the following steps of:

acquiring an AOI detection result of a product to be detected, and cutting and naming an image with a defect in the detection result;

combining the cut undetermined defect picture with the corresponding polishing picture;

labeling the combined graph dataset and performing AI model training;

and performing repeated judgment on the combined images through the deep neural network model to obtain a judgment result.

2. The method for rechecking display screen defects AI based on image merging as set forth in claim 1, wherein: the AOI detection of the product to be detected comprises the following steps:

responding to a request for scanning the two-dimensional code of the product to be detected, and enabling the product to be detected to enter AOI detection.

3. The method for rechecking display screen defects AI based on image merging as set forth in claim 2, wherein: the method for obtaining the AOI detection result of the product to be detected, cutting and naming the image with the defect detection result comprises the following steps:

obtaining coordinates and pictures of an image with a defect as a detection result, and cutting preset size block data around the undetermined defect on a corresponding scale map, wherein the block data comprise picture map and lighting map data corresponding to the detected undetermined defect;

and naming the cut pictures by using the two-dimensional code, the defect name, the defect coordinate, the defect picture or the polishing picture of the product to be detected, and storing the pictures in folders corresponding to the product to be detected.

4. The method for rendition of display screen defect AI based on image merging as set forth in claim 3, wherein: combining the cut undetermined defect picture image with the corresponding polishing image comprises the following steps:

determining an object to be merged according to the naming of the cut image;

combining the picture image of the defect to be determined and the lighting image of the corresponding coordinate into a group of images;

and combining the two corresponding images according to the defect picture images of two channels of the R channel, the G channel and the B channel and the bright image of one channel.

5. The method for recovering from display screen defects AI based on image merging as claimed in claim 4, wherein: the method for determining the object to be combined based on naming when cutting out the image comprises the following steps:

matching the two-dimensional code, the name of the undetermined defect, the coordinates of the undetermined defect and the images of the undetermined defect with the same two-dimensional code, the name of the undetermined defect, the coordinates of the undetermined defect and the polishing image data of the undetermined defect;

and if the two-dimensional code, the name of the undetermined defect and the coordinates of the undetermined defect are consistent, determining the image of the undetermined defect and the polishing image as a group of combined images.

6. The method for rechecking display screen defects AI based on image merging as set forth in claim 1 or 4, wherein: the labeling of the composition graph dataset and the AI model training comprises the following steps:

screening and distinguishing the merging graph data sets according to the true and false defects;

uploading the screened and distinguished true and false defect data set to a training platform;

training and testing the deep neural network model by the training platform through the defect data set;

and uploading the trained and tested deep neural network model to an AI server.

7. The method for recovering from display screen defects AI based on image merging as claimed in claim 6, wherein: the method for performing the repeated judgment on the combined image through the deep neural network model comprises the following steps of:

during online detection, uploading the combined defect image to be determined to an AI server;

and the AI server automatically judges the result of the combined undetermined defect image through training the tested deep neural network model.

8. The method for recovering from display screen defects AI based on image merging as claimed in claim 7, further comprising the steps of:

feeding back the result to the AOI device;

and the AOI equipment controls products with good products and bad falsification to flow into the next link according to the feedback judging result, and only transmits the genuine defective products to the buffer.

9. A computer readable storage medium, having stored thereon program instructions which, when executed, implement the method of any of claims 1-8.

10. A display screen defect AI-rendition device based on image merging, for implementing the method according to any one of claims 1 to 8, characterized in that: the system comprises an AOI unit, an image merging unit, an AI model training unit and an AI re-judging unit; wherein,

the AOI unit is used for calculating the picture image of the display screen shot by the industrial camera through an image processing algorithm to obtain a detection result, and cutting and naming the image with the defect detection result;

the image merging unit is used for merging the small images after cutting according to the defect picture images of two channels of the R channel, the G channel and the B channel and the lighting image of one channel, and transmitting the merged images to the AI complex judgment unit;

the AI model training unit is used for carrying out deep neural network model training through marked true and false defect data sets under the offline condition, optimizing a deep neural network model and updating the deep neural network model to the AI re-judging unit;

the AI re-judging unit is used for receiving the combined images, judging the images based on a preset deep neural network model and feeding back the judging result to the AOI unit during online detection.