CN116165216B - Liquid crystal display micro scratch flaw 3D detection method, system and computing equipment - Google Patents
Liquid crystal display micro scratch flaw 3D detection method, system and computing equipment Download PDFInfo
- Publication number
- CN116165216B CN116165216B CN202310250820.5A CN202310250820A CN116165216B CN 116165216 B CN116165216 B CN 116165216B CN 202310250820 A CN202310250820 A CN 202310250820A CN 116165216 B CN116165216 B CN 116165216B
- Authority
- CN
- China
- Prior art keywords
- image
- depth
- difference
- liquid crystal
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
-
- G—PHYSICS
- G02—OPTICS
- G02F—OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
- G02F1/00—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics
- G02F1/01—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour
- G02F1/13—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour based on liquid crystals, e.g. single liquid crystal display cells
- G02F1/1306—Details
- G02F1/1309—Repairing; Testing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration by non-spatial domain filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration by the use of local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration by the use of local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N2021/9513—Liquid crystal panels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20224—Image subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30121—CRT, LCD or plasma display
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention discloses a liquid crystal display micro scratch flaw 3D detection method, a liquid crystal display micro scratch flaw 3D detection system and a liquid crystal display micro scratch flaw computing device, which comprise the following steps: product position correction, data acquisition, defect detection, segment difference elimination, image processing, micro scratch defect detection, detection result acquisition and detection result output. According to the invention, the 3D laser sensor is combined with the camera, so that the angle deviation of the liquid crystal display screen in the horizontal direction and the depth direction is calibrated, the influence of the angle deviation of the liquid crystal display screen on defect detection is avoided, and the measurement precision is greatly improved; the step difference value is divided and calculated through the image, so that the step difference caused by limited line width of the laser sensor disappears; the filtering image for removing the background texture can be obtained under the condition that the defect is reserved by processing the image, so that the micro scratch defect and the background can be distinguished in the depth image; the detection precision is extremely high, the whole work flow is smooth, the degree of automation is high, and the work efficiency is high.
Description
Technical Field
The invention relates to the technical field of machine vision, in particular to a liquid crystal display micro scratch flaw 3D detection method, a liquid crystal display micro scratch flaw detection system and a liquid crystal display micro scratch flaw computing device.
Background
At present, the use amount of the tablet personal computer rises year by year, a liquid crystal display screen on the tablet personal computer generally adopts substrate glass, and the substrate glass possibly has the conditions of insufficient dissolution of glass raw materials, mixing in foreign matters and the like in the processing process, so that the tablet personal computer can be influenced by the defects of scratches, stripes, bubbles, rugged, stripes and the like; therefore, it is necessary to detect the sheet glass before assembly and to feed back the detection result in time.
The existing detection device and detection method generally adopt a two-dimensional vision system to photograph the surface of a liquid crystal display screen, and obvious defects are easily found through two-dimensional image processing points. However, when the defect range is smaller, the simple two-dimensional image processing cannot meet the detection precision, the thickness direction of the liquid crystal display screen cannot be scanned, the specific depth of the defect cannot be determined, the conditions of missing detection, false detection and erroneous judgment can exist, and the detection precision is lower.
In this regard, chinese patent CN113393464a provides a three-dimensional detection method for defects of sheet glass, comprising the steps of: s1, collecting early-stage data; s2, obtaining depth data of the plate glass; s3, extracting a depth comparison model; s4, making a comparison rule and comparing the depth data according to the comparison rule; s5, integrating the two-dimensional image data with the three-dimensional image; s6, marking an NG area; s7, recording a detection result; according to the application, the plane and the edge of the flat glass are scanned through the three-dimensional visual detection system to obtain the corresponding point positions and the depth data thereof, part of the corresponding point positions are selected to form the corresponding point position group training comparison model, the comparison model is brought back to the corresponding point position depth data to determine the comparison rule, the range of the defects allowed to exist on the flat glass can be further specified, whether the defects detected by the preamble two-dimensional visual detection system are in the allowed range or not is further determined, the overall detection precision is improved, and the problem that the available flat glass is abandoned due to detection errors is avoided. However, it also has certain drawbacks during use: the influence of angle deviation and depth direction deviation before and during the detection of the liquid crystal display screen on the detection result cannot be eliminated, and the detection result has a certain error; and because the line width of the 3D laser sensor is limited, the section difference exists in the image splicing process, and the plane fitting effect and the detection effect are affected.
Therefore, the 3D detection method, the system and the computing equipment for the micro-scratch defects of the liquid crystal display screen are used for eliminating the influence of the angle deviation of the liquid crystal display screen on the detection result through the combination of the 3D laser sensor and the camera, eliminating the section difference and detecting the micro-scratch defects, realizing high-speed and high-precision 3D detection, and obviously having practical significance.
Disclosure of Invention
The invention aims to provide a 3D detection method, a system and a computing device for micro-scratch flaws of a liquid crystal display screen, which eliminate the influence of level differences and angles through image processing and improve the detection precision.
In order to achieve the above purpose, the invention adopts the following technical scheme: A3D detection method for micro scratch flaws of a liquid crystal display screen comprises the following steps:
s1, correcting the product position: taking a picture of one corner of the liquid crystal display screen by using 1 camera, calculating angle deviation after obtaining an image, so that when a product is placed, the edge of the product is flush with a reference line of a jig;
s2, data acquisition: full-screen scanning is carried out on the liquid crystal display screen according to a specified track by utilizing a 3D laser sensor, point cloud data are converted into a plurality of depth images, and the depth images obtained by multiple scanning are spliced into a complete large image, namely, the spliced depth images are obtained;
s3, defect detection: traversing the spliced depth image obtained in the whole step S2 by using the size of a designated area, initially detecting tiny scratch defects of the liquid crystal display screen, using the size of the area as an ROI (Region Of Interest ), cutting the depth image of the ROI to perform plane fitting, and subtracting a plane fitting image from the spliced depth image to obtain a difference image after correcting the plane of the ROI;
s4, eliminating the step difference: dividing an image in the difference map after the plane correction of the ROI region into a plurality of images of rectangular regions through threshold segmentation, carrying out plane fitting on the images of the rectangular regions to obtain a difference map C, dividing a position where the difference map C has a section difference, calculating a section difference value, adjusting the pixel depth according to the section difference value, and re-splicing and fitting to obtain the difference map C;
s5, image processing: converting the spatial domain into the frequency domain, filtering the image in the frequency domain to remove background textures, and reserving micro scratch flaws;
s6, detecting micro scratch flaws: detecting micro scratch defects in the filtered frequency domain depth image of the background texture according to detection standards, filtering by adopting a depth threshold mode, calculating depth mean value, depth variance and depth discrete degree, filtering irrelevant areas, and reserving areas with depth lower than the mean value;
s7, obtaining a detection result: performing expansion operation on each reserved area, performing linear detection to obtain a micro-scratch flaw detection effect diagram, and outputting a cross-section depth waveform diagram of the depth of the flaw;
s8, outputting a detection result: and (3) judging whether the micro scratch flaw detection effect diagram is OK or NG according to the micro scratch flaw detection effect diagram and the cross section depth waveform diagram which are output in the step (S7), and recording.
Preferably, the designated track scanned in step S2 is a track formed by S-shaped left-right alternate detection, that is, the track is scanned from left to right in the horizontal direction, after the 3D laser sensor moves a line width distance after reaching the edge, the track is scanned from right to left in the horizontal direction, and the above operation is repeated to perform the reciprocating scanning until the whole display screen is scanned.
Preferably, the line width of the 3D laser sensor is 4.3mm, which is far smaller than the size of the product.
In the above, taking a display screen with length of 337mm and width of 226mm as an example, the line width of the 3D laser sensor is only 4.3mm, the total scanning is 57 times, the stitching is 56 times, the point cloud data is converted into a depth image, and the depth images scanned for multiple times are stitched into a complete large image; in the step S2, the size of the spliced depth image comprises 12-14G, and the memory occupied by the operation of the detection method comprises 28-32G.
Preferably, the micro-scratch defects comprise defects having a defect length of 10 μm or more, a width of 10 μm or more, and a depth of 1 μm or more.
Preferably, in the step S3, the size of the designated area is 5mm x 5mm to 10mm x 10mm; the size of the ROI area is the same as the size of the designated area.
In the above, since the line width of the 3D laser sensor is only 4.3mm, after the image is spliced, the depth detection of micro-scratch flaws can be affected by the height difference of 0.01mm between the depth images obtained by two scans.
Preferably, in step S4, the method for acquiring the difference map C includes: dividing an image in the difference map after the plane correction of the ROI area into a plurality of images of rectangular areas, obtaining a depth image A of each rectangular area, performing plane fitting to obtain a plane fitting map B, subtracting the plane fitting map B from the depth image A to obtain a difference map C after the plane correction, dividing a position where the difference map C has a section difference, calculating a section difference value, adjusting pixel depth according to the section difference value, and re-splicing and fitting to obtain the difference map C.
Preferably, in step S4, the method for calculating the level difference value includes: operating on the difference graph C, and dividing and cutting the difference graph C along the section difference to obtain an image A1 and an image A2; respectively acquiring average depth a of 1 pixel area on divided image A1 and image A2 Depth And b Depth Calculate the difference of height diff=a Depth -b Depth 。
Preferably, 1 pixel region acquired on image A1 is adjacent to 1 pixel region acquired on image A2.
Preferably, in step S4, the pixel depth is adjusted according to the segment difference value, and the method for obtaining the difference map c includes adding Diff to the pixel depth of each of the images A2 to obtain a new image A2new, stitching the image A1 and the image A2new into a new depth image a, performing plane fitting to obtain a plane fitting map b, subtracting the plane fitting map b from the depth image a to obtain a plane corrected difference map c, and making the segment difference disappear through the above processing.
In the above, since the obtained mosaic image has a certain angle offset when the multiple pictures are spliced, the difference image is acquired after the plane is matched, so as to eliminate the influence caused by the deviation of the splicing angle.
Preferably, in step S5, the specific method for filtering out the background texture includes: background textures exist in the vertical direction and the horizontal direction of the depth image in the space domain, texture features in the vertical direction and the horizontal direction belong to high-frequency information in the frequency domain, fast Fourier transform (fft_genetic) is carried out on the image to obtain a frequency spectrum power_real of the image, frequency peaks corresponding to the interference background textures are detected in the frequency spectrum, a cross-shaped filter is established in the frequency domain to filter frequencies, and the frequency spectrum is applied to the frequency spectrum.
In the above, the low frequency represents the overall outline of the image, and the high frequency represents the image noise.
In the above, the cross filter is used for filtering out the frequencies and is applied to the frequency spectrum, the foreground information can be separated from the interference background texture under the frequency domain filtering, the filtered image for removing the background texture is obtained, and the micro scratch defect is reserved.
Preferably, the cross filter is a cross of a rectangle in a horizontal direction and a rectangle in a vertical direction, namely, an image center, and the rectangle length in the horizontal vertical direction and the rectangle length in the vertical direction are respectively the image length and the image width, and the rectangle width is 3 pixels.
Preferably, in step S6, the detection criteria for micro-scratch flaw defects is 1 μm deep.
Preferably, in step S6, the calculation formulas for calculating the depth mean, the depth variance, and the depth discrete degree are: value = Mean-a device;
wherein Value represents a threshold Value, and Mean represents an average Value of all corresponding image depth data in the ROI; the expression represents the variance of all corresponding image depth data in the ROI, a is a coefficient, and 0.5 is taken.
Further, in step S7, for each reserved region, determining the angle of the connected region of each region, performing morphological expansion operation for a specific angle of each region, wherein the expansion direction is not fixed and is changed according to the angle of each region; and detecting sub-pixel line segments in the expanded region, setting the width of the sub-pixel outline to be 7, setting the contrast to be 5 to 21, preliminarily detecting the sub-pixel outlines of a large number of line segments through a lines_gauss operator, screening the line segment outlines with the length of more than 5 pixels, the minimum external connection length of more than 5 pixels and the width of less than 10 pixels, and finally connecting the line segment outlines into a line.
The application also claims a liquid crystal display micro scratch flaw 3D detecting system, carrying the method as described above, comprising:
the image acquisition module is used for acquiring a corner photo of the liquid crystal display screen and acquiring a depth image of the 3D laser sensor for carrying out full-screen scanning on the liquid crystal display screen;
correcting the product position: taking a picture of one corner of the liquid crystal display screen by using 1 camera, calculating angle deviation after obtaining an image, so that when a product is placed, the edge of the product is flush with a reference line of a jig;
the image processing model is used for calculating angle deviation according to the corner photo so as to correct the product position and detect the defects of the depth image, eliminate the section difference and process the image;
and the defect detection module is used for detecting and calculating micro scratch flaws and defects of the image processed in the image processing model, outputting a detection result image and judging the detection result image to be NG or OK.
The application also claims a computing device comprising a memory, a processor and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program; the processor sets an open setting for external parameters.
Due to the application of the technical scheme, compared with the prior art, the invention has the following advantages:
1. according to the invention, the 3D laser sensor is combined with the camera, and the angle calibration is carried out on the horizontal direction of the liquid crystal display screen by analyzing the corner image; by acquiring the difference image, the angle deviation of the depth direction of the liquid crystal display screen is calibrated, so that the influence of the angle deviation of the liquid crystal display screen on defect detection is avoided, and the measurement accuracy is greatly improved;
2. the invention obtains the ROI through dividing, splicing, fitting and plane correction on the image for a plurality of times, calculates the average depth at the section difference and the depth difference at the two sides to obtain the section difference value, adjusts the pixel depth according to the section difference value, and re-splices and fits to obtain a difference map, so that the section difference caused by the limited line width of the laser sensor disappears;
3. according to the invention, the detection image is subjected to fast Fourier transform, image filtering is carried out in a frequency domain, frequency peaks corresponding to interference background textures are detected in a frequency spectrum, a cross filter is established to filter the frequencies, the cross filter is applied to the frequency spectrum, and a filtered image for removing the background textures is obtained under the condition that defects are reserved, so that micro scratch defect defects and the background can be distinguished in a depth image;
4. the invention also carries out tiny scratch flaw detection from the depth direction in an image processing mode, the detection precision is extremely high, the detection capability of the depth direction reaches 1 mu m, and the detection capability of the horizontal direction reaches 10 mu m and 10 mu m;
5. the method is simple, the acquired image occupies a larger memory, the algorithm can be rapidly processed and calculated under the condition that the memory occupied by the algorithm is larger, the whole working flow is smooth, the degree of automation is high, and the working efficiency is high.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that some drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a detection method according to a first embodiment of the invention.
Fig. 2 is a scanning trace diagram of a 3D laser sensor according to a first embodiment of the present invention.
Fig. 3 is a schematic diagram of dividing a difference graph C at a level difference according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of an embodiment of the invention before image level difference elimination.
Fig. 5 is a schematic diagram of an image with a level difference eliminated according to the first embodiment of the present invention.
Fig. 6 is a depth image of an image in a spatial domain and a frequency domain according to a first embodiment of the present invention.
FIG. 7 is a schematic diagram showing the effect of micro-scratch defect detection and a waveform diagram showing the cross section of micro-scratch defect detection in the first embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made apparent and fully in view of the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
As shown in fig. 1, the embodiment relates to a 3D detection method for micro scratch flaws of a liquid crystal display screen, which includes the following steps:
s1, correcting the product position: taking a picture of one corner of the liquid crystal display screen by using 1 camera, calculating angle deviation after obtaining an image, so that when a product is placed, the edge of the product is flush with a reference line of a jig;
s2, data acquisition: full-screen scanning is carried out on the liquid crystal display screen according to a specified track by utilizing a 3D laser sensor, point cloud data are converted into a plurality of depth images, and the depth images obtained by multiple scanning are spliced into a complete large image, namely, the spliced depth images are obtained;
s3, defect detection: traversing the spliced depth image obtained in the whole step S2 by using the size of a designated region, initially detecting tiny scratch defects of the liquid crystal display screen, cutting the depth image of the ROI region by using the size of the region as the ROI region, performing plane fitting, and subtracting a plane-fitted image from the spliced depth image to obtain a difference image after plane correction of the ROI region;
s4, eliminating the step difference: dividing an image in the difference map after the plane correction of the ROI region into a plurality of images of rectangular regions through threshold segmentation, carrying out plane fitting on the images of the rectangular regions to obtain a difference map C, dividing a position where the difference map C has a section difference, calculating a section difference value, adjusting the pixel depth according to the section difference value, and re-splicing and fitting to obtain the difference map C;
s5, image processing: converting the spatial domain into the frequency domain, filtering the image in the frequency domain to remove background textures, and reserving micro scratch flaws;
s6, detecting micro scratch flaws: detecting micro scratch defects in the filtered frequency domain depth image of the background texture according to detection standards, filtering by adopting a depth threshold mode, calculating depth mean value, depth variance and depth discrete degree, filtering irrelevant areas, and reserving areas with depth lower than the mean value;
s7, obtaining a detection result: performing expansion operation on each reserved area, performing linear detection to obtain a micro-scratch flaw detection effect diagram, and outputting a cross-section depth waveform diagram of the depth of the flaw;
s8, outputting a detection result: and (3) judging whether the micro scratch flaw detection effect diagram is OK or NG according to the micro scratch flaw detection effect diagram and the cross section depth waveform diagram which are output in the step (S7), and recording.
As shown in fig. 2, the designated track scanned in step S2 is a track formed by S-shaped left-right alternate detection, that is, the track is scanned from left to right in the horizontal direction, after the 3D laser sensor moves a line width distance after reaching the edge, the track is scanned from right to left in the horizontal direction, and the above operation is repeated to perform the reciprocating scanning until the whole display screen is scanned.
Further, the line width of the 3D laser sensor is 4.3mm, which is far smaller than the size of the product.
In the embodiment, a display screen with the length of 337mm and the width of 226mm is taken as an example, and the line width of the 3D laser sensor is only 4.3mm, and the display screen is scanned 57 times and spliced 56 times in total, so that point cloud data are converted into depth images, and the depth images scanned for multiple times are spliced into a complete large image; in the step S2, the size of the spliced depth image comprises 12-14G, and the memory occupied by the operation of the detection method comprises 28-32G.
Further, in step S2, the size of the depth image after the stitching is 13G, and the memory occupied by the operation of the detection method includes 30G.
Further, the micro scratch defects comprise defects with the defect length more than or equal to 10 mu m, the defect width more than or equal to 10 mu m and the defect depth more than 1 mu m.
Further, in the step S3, the size of the designated area is 5mm x 5mm to 10mm x 10mm; the size of the ROI area is the same as the size of the designated area.
As shown in fig. 3-5, since the line width of the 3D laser sensor is only 4.3mm, after the images are spliced, the depth detection of micro-scratch flaws is affected by the height difference of 0.01mm between the depth images obtained by two scans.
Further, in step S4, the method for obtaining the difference map C includes: dividing an image in the difference map after the plane correction of the ROI area into a plurality of images of rectangular areas, obtaining a depth image A of each rectangular area, performing plane fitting to obtain a plane fitting map B, subtracting the plane fitting map B from the depth image A to obtain a difference map C after the plane correction, dividing a position where the difference map C has a section difference, calculating a section difference value, adjusting pixel depth according to the section difference value, and re-splicing and fitting to obtain the difference map C.
Further, in step S4, the method for calculating the level difference value includes: operating on the difference graph C, and dividing and cutting the difference graph C along the section difference to obtain an image A1 and an image A2; respectively acquiring average depth a of 1 pixel area on divided image A1 and image A2 Depth And b Depth Calculate the difference of height diff=a Depth -b Depth 。
Further, 1 pixel region acquired on the image A1 is adjacent to 1 pixel region acquired on the image A2.
Further, in step S4, the pixel depth is adjusted according to the segment difference value, and the method for obtaining the difference map c includes adding Diff to the pixel depth of each of the images A2 to obtain a new image A2new, stitching the image A1 and the image A2new into a new depth image a, performing plane fitting to obtain a plane fitting map b, subtracting the plane fitting map b from the depth image a to obtain a plane corrected difference map c, and enabling the segment difference to disappear through the above processing.
In the above, since the obtained mosaic image has a certain angle offset when the multiple pictures are spliced, the difference image is acquired after the plane is matched, so as to eliminate the influence caused by the deviation of the splicing angle.
Further, in step S5, the specific method for filtering out the background texture includes: background textures exist in the vertical direction and the horizontal direction of the depth image in the space domain, texture features in the vertical direction and the horizontal direction belong to high-frequency information in the frequency domain, fast Fourier transform is carried out on the image to obtain a frequency spectrum power real of the image, a frequency peak corresponding to the interference background textures is detected in the frequency spectrum, a cross-shaped filter is established in the frequency domain to filter out frequencies and is applied to the frequency spectrum, the depth image of the image in the space domain and the frequency domain is shown in fig. 6, wherein a picture on the left side is the depth image of the image in the space domain, and a picture on the right side is the depth image of the image in the frequency domain.
In the above, the low frequency represents the overall outline of the image, and the high frequency represents the image noise.
In the above, the cross filter is used for filtering out the frequencies and is applied to the frequency spectrum, the foreground information can be separated from the interference background texture under the frequency domain filtering, the filtered image for removing the background texture is obtained, and the micro scratch defect is reserved.
Further, the cross filter is a cross of a rectangle in the horizontal direction and a rectangle in the vertical direction, namely an image center, the length of the rectangle in the horizontal vertical direction and the length of the rectangle in the vertical direction are respectively the length and the width of the image, and the width of the rectangle is 3 pixels.
Further, in step S6, the detection standard of the micro scratch defect is 1 μm deep.
Further, in step S6, the calculation formulas for calculating the depth mean, the depth variance, and the depth discrete degree are as follows: value = Mean-a device;
wherein Value represents a threshold Value, and Mean represents an average Value of all corresponding image depth data in the ROI; the expression represents the variance of all corresponding image depth data in the ROI, a is a coefficient, and 0.5 is taken.
Further, in step S7, for each reserved region, determining the angle of the connected region of each region, performing morphological expansion operation for a specific angle of each region, wherein the expansion direction is not fixed and is changed according to the angle of each region; and detecting sub-pixel line segments in the expanded region, setting the width of the sub-pixel outline to be 7, setting the contrast to be 5 to 21, preliminarily detecting the sub-pixel outlines of a large number of line segments through a lines_gauss operator, screening the line segment outlines with the length of more than 5 pixels, the minimum external connection length of more than 5 pixels and the width of less than 10 pixels, and finally connecting the line segment outlines into a line.
As shown in fig. 7, the upper part of the figure is a micro-scratch defect detection effect figure, and the horizontal line segment penetrating the image in the horizontal direction in the figure is a cross-section line segment of the area where the micro-scratch defect is located, so that the cross-section depth waveform figure of the depth where the defect is located is convenient to observe; the lower part is a micro scratch flaw detection cross section waveform diagram, and the depth of the micro scratch flaw in the upper square box area is the depth data in the lower square box area.
Example two
The present embodiment is performed based on the first embodiment, and the same points as the first embodiment are not repeated.
The embodiment relates to a 3D detection system for micro scratch flaws of a liquid crystal display screen, which is provided with the method, and comprises the following steps:
the image acquisition module is used for acquiring a corner photo of the liquid crystal display screen and acquiring a depth image of the 3D laser sensor for carrying out full-screen scanning on the liquid crystal display screen;
correcting the product position: taking a picture of one corner of the liquid crystal display screen by using 1 camera, calculating angle deviation after obtaining an image, so that when a product is placed, the edge of the product is flush with a reference line of a jig;
the image processing model is used for calculating angle deviation according to the corner photo so as to correct the product position and detect the defects of the depth image, eliminate the section difference and process the image;
and the defect detection module is used for detecting and calculating micro scratch flaws and defects of the image processed in the image processing model, outputting a detection result image and judging the detection result image to be NG or OK.
Example III
The present embodiment is performed based on the first embodiment, and the same points as the first embodiment are not repeated.
The present embodiment relates to a computing device comprising a memory, a processor and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program; the processor sets an open setting for external parameters.
Further, the computer device may include one or more processors, such as one or more Central Processing Units (CPUs) or Graphics Processors (GPUs), each of which may implement one or more hardware threads. The computer device may also comprise any memory for storing any kind of information, such as code, settings, data, etc., having thereon a computer program executable on a processor, which when executed by the processor, may perform the instructions of the method described above. The computer device may also include an input/output interface (I/O) for receiving various inputs (via the input device) and for providing various outputs (via the output device). The computer device may also include one or more network interfaces for exchanging data with other devices via one or more communication links. One or more communication buses couple the above-described components together.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (6)
1. The 3D detection method for the micro scratch defects of the liquid crystal display screen is characterized by comprising the following steps of:
s1, correcting the product position: taking a picture of one corner of the liquid crystal display screen by using 1 camera, calculating angle deviation after obtaining an image, so that when a product is placed, the edge of the product is flush with a reference line of a jig;
s2, data acquisition: full-screen scanning is carried out on the liquid crystal display screen according to a specified track by utilizing a 3D laser sensor, point cloud data are converted into a plurality of depth images, and the depth images obtained by multiple scanning are spliced into a complete large image, namely, the spliced depth images are obtained;
s3, defect detection: traversing the spliced depth image obtained in the whole step S2 by using the size of a designated region, initially detecting tiny scratch defects of the liquid crystal display screen, cutting the depth image of the ROI region by using the size of the region as the ROI region, performing plane fitting, and subtracting a plane-fitted image from the spliced depth image to obtain a difference image after plane correction of the ROI region;
s4, eliminating the step difference: dividing an image in the difference map after the plane correction of the ROI region into a plurality of images of rectangular regions through threshold segmentation, carrying out plane fitting on the images of the rectangular regions to obtain a difference map C, dividing a position where the difference map C has a section difference, calculating a section difference value, adjusting the pixel depth according to the section difference value, and re-splicing and fitting to obtain the difference map C;
s5, image processing: converting the spatial domain into the frequency domain, filtering the image in the frequency domain to remove background textures, and reserving micro scratch flaws;
s6, detecting micro scratch flaws: detecting micro scratch defects in the filtered frequency domain depth image of the background texture according to detection standards, filtering by adopting a depth threshold mode, calculating depth mean value, depth variance and depth discrete degree, filtering irrelevant areas, and reserving areas with depth lower than the mean value;
s7, obtaining a detection result: performing expansion operation on each reserved area, performing linear detection to obtain a micro-scratch flaw detection effect diagram, and outputting a cross-section depth waveform diagram of the depth of the flaw;
s8, outputting a detection result: judging whether the micro scratch flaw detection effect diagram is OK or NG according to the micro scratch flaw detection effect diagram and the cross section depth waveform diagram which are output in the step S7, and recording;
in step S4, the method for acquiring the difference map C includes: dividing an image in the difference map after the plane correction of the ROI area into a plurality of images of rectangular areas, obtaining a depth image A of each rectangular area, performing plane fitting to obtain a plane fitting map B, subtracting the plane fitting map B from the depth image A to obtain a difference map C after the plane correction, dividing a position where the difference map C has a section difference, calculating a section difference value, adjusting pixel depth according to the section difference value, and re-splicing and fitting to obtain a difference map C;
in step S4, the method for calculating the level difference value includes: operating on the difference graph C, and dividing and cutting the difference graph C along the section difference to obtain an image A1 and an image A2; respectively acquiring average depth a of 1 pixel area on divided image A1 and image A2 Depth And b Depth Calculate the difference of height diff=a Depth -b Depth ；
The 1 pixel region acquired on the image A1 is adjacent to the 1 pixel region acquired on the image A2;
in step S4, the pixel depth is adjusted according to the segment difference value, and the method for obtaining the difference map c includes adding the height difference Diff to the pixel depth of each of the images A2 to obtain a new image A2new, stitching the image A1 and the image A2new into a new depth image a, performing plane fitting to obtain a plane fitting map b, subtracting the plane fitting map b from the depth image a to obtain a plane corrected difference map c, and eliminating the segment difference through the above processing.
2. The method for 3D detection of micro-scratch defects of a liquid crystal display screen according to claim 1, wherein the micro-scratch defects comprise defects with a defect depth of not less than 10 μm, a defect width of not less than 10 μm and a defect depth of not less than 1 μm.
3. The method for 3D detection of micro-scratches on a liquid crystal display according to claim 1, wherein the size of the designated area in step S3 is 5mm by 5mm to 10mm by 10mm; the size of the ROI area is the same as the size of the designated area.
4. The method for 3D detection of micro-scratch defects on a liquid crystal display according to claim 1, wherein in step S5, the specific method for filtering out background textures comprises: background textures exist in the vertical direction and the horizontal direction of the depth image in the space domain, texture features in the vertical direction and the horizontal direction belong to high-frequency information in the frequency domain, the image is subjected to fast Fourier transform to obtain a frequency spectrum of the image, frequency peaks corresponding to the interference background textures are detected in the frequency spectrum, a cross-shaped filter is established in the frequency domain to filter the frequencies, and the frequency spectrum is applied to the frequency spectrum.
5. A liquid crystal display micro-scratch defect 3D detection system, characterized by being configured to implement the method of any one of claims 1-4, comprising:
the image acquisition module is used for acquiring a corner photo of the liquid crystal display screen and acquiring a depth image of the 3D laser sensor for carrying out full-screen scanning on the liquid crystal display screen;
correcting the product position: taking a picture of one corner of the liquid crystal display screen by using 1 camera, calculating angle deviation after obtaining an image, so that when a product is placed, the edge of the product is flush with a reference line of a jig;
the image processing model is used for calculating angle deviation according to the corner photo so as to correct the product position and detect the defects of the depth image, eliminate the section difference and process the image;
and the defect detection module is used for detecting and calculating micro scratch flaws and defects of the image processed in the image processing model, outputting a detection result image and judging the detection result image to be NG or OK.
6. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method of any of claims 1-4 when the computer program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310250820.5A CN116165216B (en) | 2023-03-16 | 2023-03-16 | Liquid crystal display micro scratch flaw 3D detection method, system and computing equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310250820.5A CN116165216B (en) | 2023-03-16 | 2023-03-16 | Liquid crystal display micro scratch flaw 3D detection method, system and computing equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116165216A CN116165216A (en) | 2023-05-26 |
| CN116165216B true CN116165216B (en) | 2023-08-04 |
Family
ID=86418332
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310250820.5A Active CN116165216B (en) | 2023-03-16 | 2023-03-16 | Liquid crystal display micro scratch flaw 3D detection method, system and computing equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116165216B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116577350A (en) * | 2023-07-13 | 2023-08-11 | 北京航空航天大学杭州创新研究院 | Material surface hair bulb point cloud acquisition device and material surface hair bulb data acquisition method |
| CN117066702B (en) * | 2023-08-25 | 2024-04-19 | 上海频准激光科技有限公司 | Laser marking control system based on laser |
| CN116934746B (en) * | 2023-09-14 | 2023-12-01 | 常州微亿智造科技有限公司 | Scratch defect detection method, system, equipment and medium thereof |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596873A (en) * | 2018-03-14 | 2018-09-28 | 浙江大学山东工业技术研究院 | The recognition methods of refractory brick deep defects based on height histogram divion |
| CN110596116A (en) * | 2019-07-23 | 2019-12-20 | 浙江科技学院 | Vehicle surface flaw detection method and system |
| CN110770794A (en) * | 2018-08-22 | 2020-02-07 | 深圳市大疆创新科技有限公司 | Image depth estimation method and device, readable storage medium and electronic equipment |
| CN111551559A (en) * | 2020-05-13 | 2020-08-18 | 深圳市全洲自动化设备有限公司 | LCD (liquid Crystal display) liquid crystal screen defect detection method based on multi-view vision system |
| CN112651968A (en) * | 2021-01-20 | 2021-04-13 | 广东工业大学 | Wood board deformation and pit detection method based on depth information |
| CN112801930A (en) * | 2019-10-28 | 2021-05-14 | 上海铼钠克数控科技股份有限公司 | Method and equipment for detecting warp and weft defects of fabric in backlight based on machine vision |
| CN113393464A (en) * | 2021-08-18 | 2021-09-14 | 苏州鼎纳自动化技术有限公司 | Three-dimensional detection method for plate glass defects |
| CN115375629A (en) * | 2022-08-04 | 2022-11-22 | 广东工业大学 | Method for detecting line defect and extracting defect information in LCD screen |
| CN115619980A (en) * | 2022-11-23 | 2023-01-17 | 青岛小优智能科技有限公司 | Belted layer overlap flaw detection method based on point cloud |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| SG11201808609YA (en) * | 2016-03-30 | 2018-10-30 | Agency Science Tech & Res | System and method for imaging a surface defect on an object |
-
2023
- 2023-03-16 CN CN202310250820.5A patent/CN116165216B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596873A (en) * | 2018-03-14 | 2018-09-28 | 浙江大学山东工业技术研究院 | The recognition methods of refractory brick deep defects based on height histogram divion |
| CN110770794A (en) * | 2018-08-22 | 2020-02-07 | 深圳市大疆创新科技有限公司 | Image depth estimation method and device, readable storage medium and electronic equipment |
| CN110596116A (en) * | 2019-07-23 | 2019-12-20 | 浙江科技学院 | Vehicle surface flaw detection method and system |
| CN112801930A (en) * | 2019-10-28 | 2021-05-14 | 上海铼钠克数控科技股份有限公司 | Method and equipment for detecting warp and weft defects of fabric in backlight based on machine vision |
| CN111551559A (en) * | 2020-05-13 | 2020-08-18 | 深圳市全洲自动化设备有限公司 | LCD (liquid Crystal display) liquid crystal screen defect detection method based on multi-view vision system |
| CN112651968A (en) * | 2021-01-20 | 2021-04-13 | 广东工业大学 | Wood board deformation and pit detection method based on depth information |
| CN113393464A (en) * | 2021-08-18 | 2021-09-14 | 苏州鼎纳自动化技术有限公司 | Three-dimensional detection method for plate glass defects |
| WO2023019847A1 (en) * | 2021-08-18 | 2023-02-23 | 苏州鼎纳自动化技术有限公司 | Method for three-dimensional detection of defect of plate glass |
| CN115375629A (en) * | 2022-08-04 | 2022-11-22 | 广东工业大学 | Method for detecting line defect and extracting defect information in LCD screen |
| CN115619980A (en) * | 2022-11-23 | 2023-01-17 | 青岛小优智能科技有限公司 | Belted layer overlap flaw detection method based on point cloud |
Non-Patent Citations (1)
| Title |
|---|
| 金属球面缺陷的图像检测方法;乐静;郭俊杰;朱虹;;电子学报(第06期);193-196 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116165216A (en) | 2023-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116165216B (en) | Liquid crystal display micro scratch flaw 3D detection method, system and computing equipment | |
| CN109345528B (en) | Display screen defect detection method and device based on human visual characteristics | |
| JP6620477B2 (en) | Method and program for detecting cracks in concrete | |
| CN107678192B (en) | Mura defect detection method based on machine vision | |
| CN114494045B (en) | Large spur gear geometric parameter measurement system and method based on machine vision | |
| CN113393464B (en) | Three-dimensional detection method for plate glass defects | |
| CN113340909B (en) | Glue line defect detection method based on machine vision | |
| CN107644417B (en) | Method for detecting appearance defect of strain gauge | |
| CN112991287B (en) | Automatic indentation measurement method based on full convolution neural network | |
| CN111652844B (en) | X-ray defect detection method and system based on digital image region growing | |
| CN109828394A (en) | A kind of detection method of display panel and detection board | |
| CN112782179A (en) | Method and system for detecting defects of light-reflecting surface of product | |
| CN112489042A (en) | Metal product printing defect and surface damage detection method based on super-resolution reconstruction | |
| CN115575416A (en) | Flexible circuit board pad detection method and system | |
| US9972095B2 (en) | Image measuring apparatus and non-temporary recording medium on which control program of same apparatus is recorded | |
| CN105787870A (en) | Graphic image splicing fusion system | |
| US10516822B2 (en) | Method and device for merging images of calibration devices | |
| CN113112396B (en) | Method for detecting conductive particles | |
| CN109506569B (en) | Method for monitoring three-dimensional sizes of cubic and columnar crystals in crystallization process based on binocular vision | |
| CN116596987A (en) | Workpiece three-dimensional size high-precision measurement method based on binocular vision | |
| CN112837285B (en) | Edge detection method and device for panel image | |
| JP2019120644A (en) | Surface inspection device and surface inspection method | |
| CN115082424B (en) | 3D detection method of liquid crystal display screen | |
| CN105987667B (en) | The wide measurement method of angular thread cusp and device | |
| CN115496754B (en) | Curvature detection method and device of SSD, readable storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |