CN113298864A - Equipment and method for detecting panel line segment flaws - Google Patents
Equipment and method for detecting panel line segment flaws Download PDFInfo
- Publication number
- CN113298864A CN113298864A CN202110603369.1A CN202110603369A CN113298864A CN 113298864 A CN113298864 A CN 113298864A CN 202110603369 A CN202110603369 A CN 202110603369A CN 113298864 A CN113298864 A CN 113298864A
- Authority
- CN
- China
- Prior art keywords
- line segment
- image
- panel
- sample
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 61
- 230000007547 defect Effects 0.000 claims abstract description 57
- 238000013528 artificial neural network Methods 0.000 claims abstract description 39
- 238000005286 illumination Methods 0.000 claims description 11
- 238000012360 testing method Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000005192 partition Methods 0.000 claims description 6
- 238000007789 sealing Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 2
- 238000007689 inspection Methods 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Processing (AREA)
Abstract
The invention provides a device and a method for detecting panel line segment flaws. The equipment comprises an image acquisition device, a sample storage device and a processing device, wherein the processing device receives an initial image obtained by shooting an area where a panel to be detected is placed from the image acquisition device, acquires an image of the panel to be detected corresponding to the panel to be detected from the initial image, acquires sample data from the sample storage device so as to set an image of a sample line section position specified by sample data on the image of the panel to be detected as an image of the line section to be detected, and executes a neural network classification program to check the image of the line section to be detected so as to judge whether the line section to be detected has line section defects. Therefore, the panel quality problem caused by different artificial inspection standards can be reduced.
Description
Technical Field
The present invention relates to a panel inspection technology, and more particularly, to an apparatus and method for inspecting defects of panel line segments.
Background
In modern society where large-sized panels are prevalent, how to ensure that the panels are of a quality is not concerned has been a very important part of the panel production process. In the prior art, many manufacturers use manual inspection of the panel to confirm the quality of the panel. However, the manual inspection of the panel is time and labor consuming, and because each person subjectively determines different factors, there is a high possibility that the inspection standards are different, and the panel quality may fluctuate.
Disclosure of Invention
In view of the above, an object of the present invention is to provide an apparatus and method for detecting panel line segment defects, which can perform an operation of automatically locating the position of a panel line segment, and further send the image of the located panel line segment to a neural network classification program to determine whether there is a defect in the panel line segment by a pre-trained neural network classification program, thereby reducing the panel quality problem caused by different inspection standards.
From one perspective, the present disclosure provides an apparatus for detecting line segment defects of a panel, which is suitable for determining whether line segment defects exist on a panel to be detected. The apparatus includes an image capture device, a sample storage device, and a processing device. The image capturing device shoots the area where the panel to be detected is placed so as to obtain a corresponding initial image. The sample storage device stores sample data including sample line segment locations. The processing device is electrically connected to the image capturing device and the sample storage device, wherein the processing device receives an initial image from the image capturing device and acquires an image of the panel to be detected corresponding to the panel to be detected from the initial image; in addition, the processing device also obtains sample data from the sample storage device so as to set the image of the panel image to be measured at the position corresponding to the sample line segment as the image of the line segment to be measured; and the processing device also executes a neural network classification program to check the image of the segment to be detected so as to judge whether the segment to be detected has segment defects.
In an embodiment, the processing device divides an image block of the panel image to be tested, which includes a line segment to be tested, into a plurality of image sub-blocks, and uses each image sub-block as input data of the neural network classification program, so that the neural network classification program determines whether a line segment defect exists in a portion of the line segment to be tested included in the input image sub-block.
In one embodiment, when the neural network classification program determines that a line segment defect exists in a portion of a line segment to be detected included in the input image sub-block, the processing device reports a type of the line segment defect and a position of the image sub-block including the line segment defect.
In an embodiment, the apparatus for detecting a line segment defect of a panel further includes an illumination device and a plurality of closed partitions, the illumination device and the image capturing device are disposed on opposite sides of the panel to be detected and project light toward the panel to be detected, and the closed partitions are combined to form an opaque region surrounding the image capturing device, the panel to be detected and the illumination device.
In one embodiment, the processing device further finds at least one existing line segment from each of the plurality of ideal sample panel images, divides the existing line segments into a plurality of position groups according to the positions of the existing line segments, and stores the averaged position of the at least one existing line segment in each position group in the sample storage device as a part of the positions of the sample line segments.
Viewed from another perspective, the present invention further provides a method for detecting line segment defects of a panel, which is suitable for determining whether line segment defects exist on a panel to be detected. The method comprises the steps of shooting a panel to be detected to obtain an initial image, and capturing an image of the panel to be detected corresponding to the panel to be detected from the initial image; then, by first obtaining sample data stored in advance, setting an image on the panel image to be detected, which corresponds to the position of the sample line segment, as an image of the line segment to be detected according to the position of the sample line segment included in the sample data; finally, the image of the segment to be detected is checked by utilizing a neural network classification program to judge whether the segment to be detected has segment defects.
In an embodiment, the step of determining whether there is a line segment defect in the line segment to be detected by inspecting the image of the line segment to be detected using a neural network classification program includes: dividing an image block containing a line segment to be detected in an image of a panel to be detected into a plurality of image sub-blocks; and using each image sub-block as input data of a neural network classification program so that the neural network classification program judges whether the part of the line segment to be detected, which is contained in the input image sub-block, has line segment defects.
In one embodiment, when the neural network program determines that a line segment defect exists in a portion of a line segment to be detected included in the input image sub-block, the type of the line segment defect and the position of the image sub-block including the line segment defect are further reported.
In one embodiment, before the panel under test is photographed to obtain the initial image, the following steps are further performed: finding at least one existing line segment from each of a plurality of ideal sample panel images; dividing the existing line segments into a plurality of position groups according to the positions of the found existing line segments, wherein each position group comprises at least one existing line segment; averaging the positions of the existing line segments in each position group to obtain a corresponding average result; and storing each averaged result as a portion of the sample segment position.
In one embodiment, the finding at least one existing line segment from each of the plurality of ideal sample panel images includes: performing the following steps on each ideal sample panel image, wherein each ideal sample panel image comprises a plurality of pixels, and the plane of each ideal sample panel image is formed by expanding a first axial direction and a second axial direction: converting the image value of each pixel in the ideal sample panel image into a first value or a second value; in the ideal sample panel image, calculating a first calculation quantity which takes the image value of the pixel with the first axis coordinate value as a first value for each coordinate value of the first axis, and setting a first line segment to be integrated in the first axis coordinate value when the first calculation quantity exceeds a first preset value, wherein the first line segment to be integrated is a part of the existing line segment; and in the ideal sample panel image, calculating a second calculation quantity which takes the image value of the pixel with the second axis coordinate value as the first value for each coordinate value of the second axis, setting a second line segment to be integrated in the second axis coordinate value when the second calculation quantity exceeds a second preset value, and enabling the second line segment to be integrated to be a part of the existing line segment.
According to the above, the apparatus and method for detecting panel line segment defects provided in the present disclosure can automatically position the position of the panel line segment to be detected on the panel by the image of the panel, and further send the positioned image of the panel line segment to the neural network classification program to determine whether the panel line segment has defects by the neural network classification program trained in advance. Therefore, the technology provided by the invention not only can reduce the manpower required for judging the defects, but also can reduce the problem of uneven panel quality caused by different manual inspection standards.
Drawings
Fig. 1 is an external view of an apparatus for detecting defects of panel line segments according to an embodiment of the invention.
FIG. 2 is a block diagram of an apparatus for detecting defects in panel line segments according to an embodiment of the present invention.
FIG. 3 is a flowchart illustrating a method for detecting defects in a panel line segment according to an embodiment of the invention.
FIG. 4A is a schematic diagram of an ideal sample panel image used in accordance with one embodiment of the present invention.
Fig. 4B is a bar graph showing the relationship between the X-axis coordinate and the number of black pixels in fig. 4A.
Fig. 4C is a bar graph showing the relationship between the Y-axis coordinate and the number of black pixels in fig. 4A.
FIG. 5 is a flow chart of establishing sample data according to an embodiment of the invention.
FIG. 6 is a diagram illustrating an image block being divided into a plurality of image sub-blocks according to an embodiment of the present invention.
FIG. 7 is a schematic diagram illustrating an appearance of an apparatus for detecting defects in panel line segments according to an embodiment of the present invention.
Wherein the symbols in the drawings are briefly described as follows:
10. 70: a device for detecting panel line segment flaws; 40: an ideal sample panel image; 60: the panel image to be detected; 100: an image capturing device; 110: a processing device; 115: a neural network classification procedure; 120: a sample storage device; 125: sample data; 130: a panel to be tested; 140: an illumination device; 155: a shooting area; 400. 402, 404, 406, 410, 412, 414: an existing line segment; 600. 602, 604, 606, 610, 612, 614: a line segment to be detected; 602-1, 602-2, 602-N, 612-1, 612-2, 612-M: an image sub-block; 700: sealing the partition plate; n: the number of the particles; s300 to S308: an implementation step of an embodiment of the present invention; s500 to S510: the implementation step of one embodiment of the invention when establishing sample data; x, Y: a shaft.
Detailed Description
Referring to fig. 1 and fig. 2, fig. 1 is an external view of an apparatus for detecting panel line segment defects according to an embodiment of the present invention, and fig. 2 is a circuit block diagram of the apparatus for detecting panel line segment defects shown in fig. 1. In the embodiment, the apparatus 10 includes an image capturing device 100, a processing device 110 and a sample storage device 120, wherein the processing device 110 is electrically connected to the image capturing device 100 and the sample storage device 120 respectively, so that the processing device 110 can perform data transmission with the image capturing device 100 and the sample storage device 120 respectively.
In order to make the operation of the apparatus 10 more clearly understood by those skilled in the art, the following description is made in conjunction with fig. 1, fig. 2 and fig. 3, wherein fig. 3 is a flowchart of a method for detecting a panel line segment defect according to an embodiment of the present invention.
As shown in fig. 1, the shooting area 155 is the range that can be covered by the image capturing apparatus 100 during shooting, and the panel 130 to be tested is placed in the shooting area 155. Before actually detecting the panel line segment defect, the processing device 110 or other suitable control device may be controlled by a predetermined control program to operate the image capturing device 100 to capture the image of the capturing area 155, so as to obtain an image (hereinafter referred to as an initial image) of the entire capturing area 155 (step S300). It should be noted that although the size of the capturing region 155 is larger than the size of the panel 130, in the embodiment, the size of the capturing region 155 may be smaller than the size of the panel 130, and the entire image of the panel 130 may be obtained by moving the position of the panel 130 or the image capturing apparatus 100 and performing multiple capturing.
The initial image captured by the image capturing device 100 is transmitted to the processing device 110 for further processing. After acquiring the initial image from the image acquisition device 100, the processing device 110 then acquires a portion corresponding to the panel under test 130 from the initial image to obtain a panel under test image (step S302). Taking the embodiment shown in fig. 1 as an example, since the size of the photographing region 155 is larger than that of the panel 130 to be tested, there exists an image other than the panel 130 to be tested in the initial image. In such a situation, the processing device 110 may perform an image processing related procedure (e.g., a neural network procedure U-Net or other procedures) to remove an image portion that is not the panel under test 130 or extract an image portion related to the panel under test 130 to finally generate a panel under test image. In another embodiment, if the size of the capture area 155 is smaller than the size of the panel 130 to be tested, the processing device 110 may also combine a plurality of initial images by performing a related procedure of image processing to generate a panel image to be tested required for subsequent processing.
In addition to the pre-processing procedure, the processing device 110 obtains the sample data 125 stored in advance in the sample storage device 120 (step S304). It can be seen that the sample data 125 should be established before the line segment defect is actually detected. In order to create the sample data 125, the processing device 110 must first collect information about the location of the line segment to be detected. Referring to fig. 4A and 5 together, fig. 4A is a schematic diagram of an ideal sample panel image used according to an embodiment of the invention, and fig. 5 is a flowchart of establishing sample data according to an embodiment of the invention. An ideal sample panel image corresponding to the ideal sample panel can be obtained by controlling the image capturing apparatus 100 to capture an ideal sample panel and performing the above steps S300 and S302 (step S500). In the present embodiment, the ideal sample panel image 40 shown in fig. 4A includes a plurality of pixels, and the plane thereof is formed by extending from the X axis (hereinafter, also referred to as the first axis direction) and the Y axis (hereinafter, also referred to as the second axis direction). By the image processing technique, the processing device 110 can find the existing line segments 400, 402, 404 and 406 extending along the Y axis and the existing line segments 410, 412 and 414 extending along the X axis from the ideal sample panel image 40 (step S502). The position information of the existing line segments 400-414 found in step S502 can be directly used for subsequent processing, or, as shown in fig. 5, after an ideal sample panel image is subjected to image processing and the position information of the corresponding existing line segment is found, it can be further determined whether or not further ideal sample panel images need to be obtained (step S504). If other ideal sample panel images need to be obtained, the processing device 110 will repeat steps S500-S502 again to obtain the position information of another set of existing line segments. On the other hand, if it is not necessary to obtain another ideal sample panel image, the processing device 110 proceeds to step S506 with the position information of all existing line segments and continues the subsequent processing.
It should be noted that although the existing line segment can be found in step S502 by manual marking, the present invention proposes to improve the process of finding the existing line segment automatically. In order to find the existing line segments without relying on human labor, in one embodiment, the processing device 110 may first binarize the image values of the pixels in the ideal sample panel image, i.e., adjust the ideal sample panel image that originally exhibits gray scale or color during shooting to an image with only black and white colors. Next, the processing device 110 starts to calculate the number of pixels representing the color (e.g., black) of the existing line segment appearing on the same X-axis coordinate (hereinafter referred to as a first calculated number), and the number of pixels representing the color (e.g., black) of the existing line segment appearing on the same Y-axis coordinate (hereinafter referred to as a second calculated number). For example, referring to fig. 4B, the horizontal axis represents the coordinate value of the X axis, and the vertical axis represents the number n of black pixels, so that the heights of the four straight lines from left to right in fig. 4B represent the number of black pixels of the existing line segment 400, the number of black pixels of the existing line segment 402, the number of black pixels of the existing line segment 404, and the number of black pixels of the existing line segment 406, respectively; referring to fig. 4C, the horizontal axis represents the coordinate value of the Y axis, and the vertical axis represents the number n of black pixels, so that the heights of the three straight lines from left to right in fig. 4C represent the number of black pixels of the existing line segment 414, the number of black pixels of the existing line segment 412, and the number of black pixels of the existing line segment 410, respectively.
In the embodiment shown in FIG. 4A, for each X-axis coordinate except the positions of the existing line segments 400-406, the black pixels existing due to the existing line segments 410-414 may be calculated, and the black pixels generated due to the camera distortion or the image processing misjudgment may also be calculated, so that the portions of the existing line segments not extending along the Y-axis must be excluded after obtaining the first calculated number corresponding to each X-axis coordinate in FIG. 4B. Thus, a preset value (hereinafter referred to as a first preset value) may be set in advance and only the coordinates where the first calculated number exceeds the first preset value may be set as the positions where existing line segments exist. Similarly, for each Y-axis coordinate except the positions of the existing line segments 410-414, the black pixels existing due to the existing line segments 400-406 may be calculated, and the black pixels generated due to the camera distortion or the image processing misjudgment may also be calculated, so that the portions of the existing line segments not extending along the X-axis must be excluded after the second calculated number corresponding to each Y-axis coordinate in FIG. 4C is obtained. Then, it is also possible to previously set a preset value (hereinafter referred to as a second preset value) and set only the coordinates where the second calculated number exceeds the second preset value as the positions where existing line segments exist.
Through the method, the process of searching the existing line segments can be automated, and the labor consumption is reduced.
Please refer back to fig. 5. After the operations of steps S500 to S504, the processing device 110 first groups existing line segments obtained from the same ideal sample panel image in step S506. When grouping, the processing device 110 classifies different existing line segments into different Position groups (Position groups). For example, as mentioned above, the existing line segments 400-414 can be obtained from the ideal sample panel image 40, and thus the processing device 110 classifies the different existing line segments 400-414 into different position groups, for example, the existing line segment 400 is classified into a first position group, the existing line segment 402 is classified into a second position group, the existing line segment 404 is classified into a third position group, the existing line segment 406 is classified into a fourth position group, the existing line segment 410 is classified into a fifth position group, the existing line segment 412 is classified into a sixth position group, and the existing line segment 414 is classified into a seventh position group. The positions included in each position group may include positions within a predetermined range from the existing line segments, in addition to the positions of the existing line segments. For example, the first position group may include a section extending from the leftmost end of the ideal sample panel image 40 all the way to the right to the section covered at the center line between the existing line segment 400 and the existing line segment 402; the second group of locations may include a range extending from the centerline between the existing line segment 400 and the existing line segment 402 all the way to the right to the centerline between the existing line segment 402 and the existing line segment 404; a third group of locations may include a region extending from the centerline between the existing line segment 402 and the existing line segment 404 all the way to the right to the region encompassed by the centerline between the existing line segment 404 and the existing line segment 406; the fourth group of positions may include a range extending from the centerline between the existing line segment 404 and the existing line segment 406 all the way to the right of the rightmost end of the ideal sample panel image 40. Similarly, the fifth position group may include a region extending from the top of the ideal sample panel image 40 all the way down to the center line between the existing line segment 410 and the existing line segment 412; the sixth group of positions may include an interval extending straight down from the midline between the existing line segment 410 and the existing line segment 412 to the extent covered at the midline between the existing line segment 412 and the existing line segment 414; the seventh group of locations may include the interval extending from the centerline between the existing line segment 412 and the existing line segment 414 all the way down to the lowermost end of the ideal sample panel image 40.
It should be noted that the size of the segment covered by each position group may be defined in other ways, for example, a segment including 10 pixels near the position of each existing line segment, and the like, which can be adjusted by those skilled in the art as required.
Next, when there is only one ideal sample panel image for which the operation of step S506 is required, the processing device 110 may carry the information on each position group (the position of the existing line segment in each position group) obtained at this time into step S508 and perform the subsequent processing. In contrast, if there are many ideal sample panel images that require the operation of step S506, the processing device 110 needs to perform the above-mentioned classification operation once for each of the different ideal sample panel images. As known to those skilled in the art, since different ideal sample panel images obtained by photographing each of a plurality of ideal sample panels are to be processed, it is expected that corresponding existing line segments in these ideal sample panel images should fall at approximate positions. For example, an existing line segment 400 found in the first ideal sample panel image would theoretically appear in the same location as an existing line segment 400 found in the second ideal sample panel image; even in view of possible errors in the image processing, the existing line segments 400 found in the first ideal sample panel image should not be too far apart from the existing line segments 400 found in the second ideal sample panel image. Therefore, after performing the aforementioned Position Group classification on each ideal sample panel image, the processing device 110 may classify Position groups that are close to each other and obtained from different ideal sample panel images into the same Position Group (Position Group Set).
In various embodiments, the processing device 110 may employ any suitable classification criteria to classify the different location groups into the same location group, for example, to compare the coverage of the first location group of the first ideal sample panel image with the coverage of the first to fourth location groups of the second ideal sample panel image that also include the existing line segment extending along the Y-axis, and to classify the location group of the second ideal sample panel image that overlaps most with the coverage of the first location group of the first ideal sample panel image into the same location group as the first location group of the first ideal sample panel image. Similar operations are also applied to each position group of each ideal sample panel image in step S506 to thereby divide the existing line segments into a plurality of position groups.
After the processing of step S506, each position group includes at least one existing line segment. Next, the processing device 110 averages the positions of the existing line segments in each position group in step S508 to obtain the corresponding average result (if there is only one existing line segment, the average result is naturally the position of the existing line segment), and stores the obtained average result in the sample data 125 so that each average result becomes a sample line segment position or a part of the sample data 125 (step S510). It is obvious that when the average result has only one (i.e. only one position group), the sample segment position (or the sample data 125) has only one content; when the average result has N, the sample segment position (or the sample data 125) has N contents.
Referring back to fig. 3, after the sample data 125 obtained through the operation flow of fig. 5 is obtained, the processing device 110 sets the position specified by each content corresponding to the sample segment position in the panel image to be measured and the image block within the specific distance adjacent to the position specified by the content to be measured to be an image of a corresponding segment to be measured according to the sample data 125 (step S306). After obtaining the images of the segments to be detected, the processing device 110 may input the image of each segment to be detected into the neural network classifier 115 (for example, but not limited to, the neural network program MobileNetV2) to determine whether there is a specific segment defect in the image of each segment to be detected by the neural network classifier 115 (step S308).
The neural network classifier 115 should be trained before the line segment flaw is determined. Once the neural network classifier 115 is able to complete the determination training for some types of line segment defects (e.g., line width variation, holes, line breaks, etc.), after the image of the line segment to be detected is input into the neural network classifier 115, the neural network classifier 115 can determine whether the line segment to be detected has these specific types of line segment defects according to a fixed standard.
It should be noted that the aforementioned image of the line segment to be measured is not limited to be completely input into the neural network classification program 115. In various embodiments, the processing device 110 may also divide the image block including the line segment to be detected into a plurality of image sub-blocks, and use each image sub-block as the input data of the neural network classification program 115 so that the neural network classification program 115 can analyze the one image sub-block, thereby determining whether there is a specific line segment defect in the portion of the line segment to be detected included in the one image sub-block. FIG. 6 is a diagram illustrating an image block including a line segment to be measured divided into a plurality of image sub-blocks according to an embodiment of the present invention. As shown in fig. 6, the panel image 60 to be measured includes a plurality of line segments 600, 602, 604, 606, 610, 612, and 614, and the processing device 110 can divide the image block including the line segment to be measured into a plurality of image sub-blocks, which are only exemplified by the line segments 602 and 612. As shown in fig. 6, the processing device 110 can divide the image block including the line segment to be measured 602 into a plurality of image sub-blocks 602-1, 602-2,. and 602-N, and divide the image block including the line segment to be measured 612 into a plurality of image sub-blocks 612-1, 612-2,. and 612-M (each block is shown by a dotted line in the figure). When the processing device 110 determines that there is a specific line defect in the portion of the line segments to be measured included in an image sub-block by using the neural network classification program 115, the type of the line defect and the position of the image sub-block including the line defect may be reported for subsequent handling. For example, if the processing device 110 reports that there is a broken line defect at the position of the image sub-block 612-6 (not shown), the line segment at the position of the image sub-block 612-6 in the panel to be tested can be repaired by laser, so as to improve the yield of the panel. It can be understood by those skilled in the art that in the embodiment of using the image sub-blocks as the input data of the neural network classification program 115, when training the neural network classification program 115, it is necessary to train the received image sub-blocks (instead of the image of the entire line segment to be measured) as the input data, and each image sub-block as the input data of the neural network classification program 115 must be scaled to the same image size, which is not described herein again.
As can be seen from the above description, the present invention has the capability of automatically determining line segment defects.
Referring again to fig. 1, the apparatus 10 may further include an illumination device 140. In the present embodiment, the illumination device 140 and the image capturing device 100 are respectively disposed on two opposite sides of the panel 130 to be tested and project light toward the panel 130 to be tested. Further, in order to reduce the image problem caused by the external light, in the embodiment shown in fig. 7, an opaque sealing partition 700 is further used to combine an opaque region capable of enclosing the image capturing apparatus 100, the panel 130 to be measured and the illumination apparatus 140 shown in fig. 1. In this way, since the light used by the image capturing device 100 is only the light provided by the illumination device 140, the problem caused by the interference of the external light and shadow change on the image can be greatly reduced when the device 70 is used to determine the line segment defect.
In summary, the apparatus and method for detecting panel line segment defects provided in the above disclosure can automatically locate the position of the panel line segment to be detected on the panel through the image of the panel, and further send the located image of the panel line segment to the neural network classification program to determine whether there is a defect in the panel line segment through the neural network classification program trained in advance. Therefore, the technology provided by the invention not only can reduce the manpower required for judging the defects, but also can reduce the problem of uneven panel quality caused by different manual inspection standards.
The above description is only for the preferred embodiment of the present invention, and it is not intended to limit the scope of the present invention, and any person skilled in the art can make further modifications and variations without departing from the spirit and scope of the present invention, therefore, the scope of the present invention should be determined by the claims of the present application.
Claims (10)
1. An apparatus for detecting line segment defects of a panel, adapted to determine whether line segment defects exist on a panel to be tested, the apparatus comprising:
the image acquisition device shoots an area where the panel to be detected is placed so as to acquire an initial image;
the sample storage device stores sample data, and the sample data comprises a sample line segment position; and
a processing device electrically connected to the image capturing device and the sample storage device,
wherein the processing device receives the initial image from the image capturing device and obtains an image of the panel to be tested corresponding to the panel to be tested from the initial image, and the processing device further obtains the sample data from the sample storage device to set the image of the panel to be tested corresponding to the position of the sample line segment as the image of the line segment to be tested,
the processing device executes a neural network classification program to check the image of the segment to be detected so as to judge whether the segment to be detected has the segment defect.
2. The apparatus of claim 1, wherein the processing device divides an image block of the panel image to be tested including the line segment to be tested into a plurality of image sub-blocks, and uses each of the plurality of image sub-blocks as input data of the neural network classification process, so that the neural network classification process determines whether the line segment defect exists in a portion of the line segment to be tested included in the input image sub-block.
3. The apparatus of claim 2, wherein the processing device reports the type of the line segment defect and the location of the image sub-block containing the line segment defect when the neural network classification procedure determines that the line segment defect exists in the portion of the image sub-block containing the line segment to be tested.
4. The apparatus of claim 1, further comprising an illumination device and a plurality of sealing partitions, the illumination device and the image capture device being disposed on opposite sides of the panel under test and projecting light toward the panel under test, the plurality of sealing partitions forming, in combination, an opaque region surrounding the image capture device, the panel under test, and the illumination device.
5. The apparatus of claim 1, wherein the processing device further finds at least one existing line segment from each of the plurality of ideal sample panel images, divides the at least one existing line segment into a plurality of position groups according to the position of the at least one existing line segment, and stores the averaged position of the at least one existing line segment in each of the plurality of position groups to the sample storage device as a portion of the position of the sample line segment.
6. A method for detecting line segment flaws of a panel is suitable for determining whether line segment flaws exist on the panel to be detected, and is characterized by comprising the following steps:
shooting the panel to be tested to obtain an initial image;
capturing an image of a panel to be tested corresponding to the panel to be tested from the initial image;
obtaining sample data stored in advance, wherein the sample data comprises a sample line segment position;
setting the image corresponding to the sample line segment position on the panel image to be detected as the image of the line segment to be detected according to the sample line segment position; and
and checking the image of the segment to be detected by utilizing a neural network classification program to judge whether the segment to be detected has the segment defect.
7. The method of claim 6, wherein the inspecting the image of the line segment to be tested to determine whether the line segment to be tested has the line segment defect by using the neural network classification process comprises:
dividing the image block of the panel image to be detected, which contains the line segment to be detected, into a plurality of image sub-blocks; and
and taking each image sub-block as input data of the neural network classification program so that the neural network classification program judges whether the line segment flaw exists in the part of the line segment to be detected contained in the input image sub-block.
8. The method of claim 7, wherein when the neural network classification process determines that the line segment defect exists in the portion of the line segment under test included in the input image sub-block, the method further comprises:
the type of the line segment defect and the position of the image sub-block containing the line segment defect are reported.
9. The method of claim 6, wherein before capturing the panel under test to obtain the initial image, further comprising:
finding at least one existing line segment from each of a plurality of ideal sample panel images;
dividing the at least one existing line segment into a plurality of position groups according to the position of the at least one existing line segment, wherein each of the plurality of position groups comprises the at least one existing line segment;
averaging the positions of the at least one existing line segment in each of the plurality of position groups to obtain a corresponding average result; and
each of the averaged results is stored as part of the sample segment position.
10. The method of claim 9, wherein finding the at least one existing line segment from each of a plurality of ideal sample panel images comprises:
performing the following steps on each of the plurality of ideal sample panel images, wherein each of the plurality of ideal sample panel images comprises a plurality of pixels, and a plane of each of the plurality of ideal sample panel images is formed by extending a first axis and a second axis:
converting the image value of each of the plurality of pixels in the ideal sample panel image into a first value or a second value;
calculating a first calculation quantity of the first value of the image values of the plurality of pixels having the first axis coordinate value for each coordinate value of the first axis in the ideal sample panel image, setting a first segment to be integrated in the first axis coordinate value when the first calculation quantity exceeds a first preset value, and making the first segment to be integrated be a part of the at least one existing segment; and
in the ideal sample panel image, a second calculated number is calculated for each coordinate value of the second axis, wherein the image values of the pixels having the second axis coordinate value are the first value, a second line segment to be integrated is set to exist in the second axis coordinate value when the second calculated number exceeds a second preset value, and the second line segment to be integrated is a part of the at least one existing line segment.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW110113744A TWI841834B (en) | 2021-04-16 | 2021-04-16 | Apparatus and method for checking line defects on panel |
| TW110113744 | 2021-04-16 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113298864A true CN113298864A (en) | 2021-08-24 |
Family
ID=77326513
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110603369.1A Pending CN113298864A (en) | 2021-04-16 | 2021-05-31 | Equipment and method for detecting panel line segment flaws |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113298864A (en) |
| TW (1) | TWI841834B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104122264A (en) * | 2013-04-25 | 2014-10-29 | 鸿富锦精密工业（深圳）有限公司 | Appearance flaw detection system and method |
| CN104297256A (en) * | 2013-07-19 | 2015-01-21 | 鸿富锦精密工业（深圳）有限公司 | Appearance flaw detection system and method |
| TWI642126B (en) * | 2017-06-30 | 2018-11-21 | 中華大學 | Semiconductor wafer analyzing sytem and method thereof |
| TWM606740U (en) * | 2020-08-17 | 2021-01-21 | 宜谷京科技實業有限公司 | Defect detection system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11379967B2 (en) * | 2019-01-18 | 2022-07-05 | Kla Corporation | Methods and systems for inspection of semiconductor structures with automatically generated defect features |
| US20220222806A1 (en) * | 2019-05-22 | 2022-07-14 | Applied Materials Israel Ltd. | Machine learning-based classification of defects in a semiconductor specimen |
| CN110186938A (en) * | 2019-06-28 | 2019-08-30 | 笪萨科技(上海)有限公司 | Two-sided defect analysis equipment and defects detection and analysis system |
| US11042981B2 (en) * | 2019-07-12 | 2021-06-22 | SVXR, Inc. | Methods and systems for printed circuit board design based on automatic corrections |
-
2021
- 2021-04-16 TW TW110113744A patent/TWI841834B/en active
- 2021-05-31 CN CN202110603369.1A patent/CN113298864A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104122264A (en) * | 2013-04-25 | 2014-10-29 | 鸿富锦精密工业（深圳）有限公司 | Appearance flaw detection system and method |
| CN104297256A (en) * | 2013-07-19 | 2015-01-21 | 鸿富锦精密工业（深圳）有限公司 | Appearance flaw detection system and method |
| TWI642126B (en) * | 2017-06-30 | 2018-11-21 | 中華大學 | Semiconductor wafer analyzing sytem and method thereof |
| TWM606740U (en) * | 2020-08-17 | 2021-01-21 | 宜谷京科技實業有限公司 | Defect detection system |
Also Published As
| Publication number | Publication date |
|---|---|
| TWI841834B (en) | 2024-05-11 |
| TW202242790A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4935109B2 (en) | Substrate inspection device, inspection logic setting method and inspection logic setting device | |
| CN105510348A (en) | Flaw detection method and device of printed circuit board and detection equipment | |
| TW202001798A (en) | Optical inspection method, optical inspection device and optical inspection system | |
| KR20210104028A (en) | Compensation for reference misalignment during part inspection | |
| CN112053318A (en) | Two-dimensional PCB defect real-time automatic detection and classification device based on deep learning | |
| CN108445010B (en) | Automatic optical detection method and device | |
| CN116109635B (en) | Method, device, equipment and medium for detecting surface quality of composite suspension insulator | |
| CN115908420A (en) | Method, device and equipment for detecting defects of printed circuit board and storage medium | |
| WO2017071406A1 (en) | Method and system for detecting pin of gold needle element | |
| JP4230880B2 (en) | Defect inspection method | |
| CN115239727A (en) | PCB surface defect detection method | |
| CN113176275B (en) | Method, device and system for retesting display panel | |
| JP2007033126A (en) | Substrate inspection device, parameter adjusting method thereof and parameter adjusting device | |
| CN117589770A (en) | PCB patch board detection method, device, equipment and medium | |
| CN112330590A (en) | Wafer defect verification method | |
| CN116993654B (en) | Camera module defect detection method, device, equipment, storage medium and product | |
| JP4506395B2 (en) | Substrate inspection device, parameter setting method and parameter setting device | |
| CN113298864A (en) | Equipment and method for detecting panel line segment flaws | |
| JP2002310937A (en) | Method and apparatus for inspection of defect | |
| CN115546140A (en) | Display panel detection method and system and electronic device | |
| JP2005252451A (en) | Image quality inspection method and image quality inspection apparatus | |
| CN113933309A (en) | Method for retesting defects in blind hole by AOI machine | |
| JP4507785B2 (en) | Substrate inspection device, parameter setting method and parameter setting device | |
| JP2006284543A (en) | Method and device for inspecting mounted circuit board | |
| JP4419778B2 (en) | Substrate inspection device, parameter setting method and parameter setting device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |