CN112824881A - System and method for detecting defects of transparent or semitransparent medium based on light field camera - Google Patents
System and method for detecting defects of transparent or semitransparent medium based on light field camera Download PDFInfo
- Publication number
- CN112824881A CN112824881A CN202010412397.0A CN202010412397A CN112824881A CN 112824881 A CN112824881 A CN 112824881A CN 202010412397 A CN202010412397 A CN 202010412397A CN 112824881 A CN112824881 A CN 112824881A
- Authority
- CN
- China
- Prior art keywords
- light field
- transparent
- field camera
- light
- medium
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/958—Inspecting transparent materials or objects, e.g. windscreens
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0008—Industrial image inspection checking presence/absence
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/8861—Determining coordinates of flaws
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10052—Images from lightfield camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30121—CRT, LCD or plasma display
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Immunology (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Pathology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
A transparent or semitransparent medium defect detecting system comprises a transparent or semitransparent medium to be detected, at least one light source for irradiating light to the transparent or semitransparent medium, and a light field camera for shooting and acquiring an image of the transparent or semitransparent medium so as to detect three-dimensional defects existing in the transparent or semitransparent medium, wherein the detecting step comprises the following steps: shooting a plurality of defocusing soft light plates by using the light field camera after the light rings are matched, calibrating the white image of the light field, and completing the center calibration of the micro lens; calibrating the scale of the light field camera, and setting up a light source with a proper angle; and shooting the defect area of the detected medium by using a light field camera, processing to obtain a multi-view and depth image, removing non-defect points in the depth image based on the shape and color of the defect in the multi-view image, and finally obtaining the three-dimensional information of the defect of the transparent or semitransparent medium.
Description
Technical Field
The invention relates to the technical field of three-dimensional photoelectric detection of transparent or semitransparent media, in particular to a system and a method for detecting three-dimensional defects of the transparent or semitransparent media based on a light field camera.
Background
In recent years, with the rapid growth of technology and industrial level, the mass production of products such as consumer screen electronic products, large screen displays and lenses, etc. has been carried out, and the quality requirement of such transparent or semitransparent media is also increasing. Three-dimensional defect inspection of transparent or translucent media has been one of the most difficult issues for industrial appearance inspection.
The three-dimensional defect detection technology is a core technology in the field of machine vision and measurement. Three-dimensional defect detection refers to identifying object defects and obtaining corresponding three-dimensional information. At present, most of the defect detection of transparent or semitransparent media in the industry is human eye detection and two-dimensional camera detection, and a two-dimensional industrial camera matched with a proper light source can only detect whether dust and defects exist on the media, but cannot distinguish the dust and the defects; dust can be washed away by cleaning, products can be continuously used, and products containing defects need to be scrapped; therefore, the human eyes or the two-dimensional camera need to repeat the detection and cleaning procedures repeatedly, the efficiency is low, and the misjudgment rate is high. Since the media is typically made of multiple layers bonded together, defects may occur in both of these layers. Judging the layer where the defect is located inside is helpful for a factory to determine which procedure has a problem, so that the production process is effectively improved, and dust appears on the upper surface and the lower surface of the medium, which is also a great difference between the three-dimensional space positions of the defect and the dust. The detection of defects and dust in the three-dimensional position of transparent or translucent media is of great benefit to existing production. Automated solutions are urgently needed for three-dimensional defect detection of transparent or translucent media.
The advent of light field cameras has provided a new solution to three-dimensional defect detection. The light field camera is additionally provided with a micro lens array between a sensor and a main lens of a conventional camera, so that the propagation direction of light rays is recorded, a unique light field image coded by the lens array is formed, the light field image is processed and rendered, and then three-dimensional information can be obtained. The light field camera is a passive three-dimensional measurement mode, so that defect detection and three-dimensional measurement can be performed through a transparent or semitransparent medium.
Disclosure of Invention
The embodiment of the invention provides a system and a method for detecting defects of a transparent or semitransparent medium, aiming at realizing comprehensive accurate detection of three-dimensional defects of the transparent or semitransparent medium by utilizing a light field camera.
In one embodiment of the present invention, a transparent or translucent medium defect detection system includes,
the transparent or translucent medium to be detected,
at least one light source for irradiating light to the transparent or translucent medium,
a light field camera for capturing images of the transparent or translucent medium for detecting three-dimensional defects in the transparent or translucent medium, the detecting step comprising:
shooting a plurality of defocusing soft light plates by using the light field camera after the light rings are matched, calibrating the white image of the light field, and completing the center calibration of the micro lens; calibrating the scale of the light field camera, and setting up a light source with a proper angle; and shooting the defect area of the detected medium by using a light field camera, processing to obtain a multi-view and depth image, removing non-defect points in the depth image based on the shape and color of the defect in the multi-view image, and finally obtaining the three-dimensional information of the defect of the transparent or semitransparent medium.
The embodiment of the invention has the beneficial effects that:
1. by utilizing the characteristic of passive measurement of the light field camera, the light field camera can carry out three-dimensional imaging through a transparent or semitransparent medium. Therefore, not only can three-dimensional measurement of upper surface defects be performed, but also three-dimensional inspection of internal and lower surface defects can be performed.
2. The three-dimensional detection method based on the light field camera is efficient, convenient and high in precision and universality.
Drawings
The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:
fig. 1 is a flow chart of a detection process according to one embodiment of the invention.
FIG. 2 is a schematic diagram of a light field camera shooting a measured medium under multi-angle light source irradiation according to an embodiment of the present invention.
Of these, 100-a light field camera,
201-first light source, 202-second light source, 203-third light source, 204-fourth light source, 300-transparent or translucent medium with defects.
FIG. 3 is an example of an image in an embodiment of the present invention, where 3-a is a light field center view angle image of a mobile phone screen glass and 3-b is a corresponding depth image in this example; 3-c are side views of the corresponding three-dimensional point cloud image in this example.
FIG. 4 is an example of an image in an embodiment of the present invention, and 4-a is a light field center view angle image of a polarizing plate; 4-b is the corresponding depth image in this example; 4-c are side views of the corresponding three-dimensional point cloud image of this example.
FIG. 5 is an example of an image in an embodiment of the present invention, 5-a is a light field center viewing angle image of a transparent CPI film of an OLED screen; 5-b is the depth image corresponding to this example; 5-c are side views of the corresponding three-dimensional point cloud image of this example.
Detailed Description
In accordance with one or more embodiments, a transparent or translucent media defect detection system is shown in FIG. 2. The system includes a transparent or translucent medium to be detected; at least one light source for irradiating light to the transparent or translucent medium; a light field camera for capturing images of the transparent or translucent medium for detecting three-dimensional defects in the transparent or translucent medium, the detecting step comprising: shooting a plurality of defocusing soft light plates by using the light field camera after the light rings are matched, calibrating the white image of the light field, and completing the center calibration of the micro lens; calibrating the scale of the light field camera, and setting up a light source with a proper angle; and shooting the defect area of the detected medium by using a light field camera, processing to obtain a multi-view and depth image, removing non-defect points in the depth image based on the shape and color of the defect in the multi-view image, and finally obtaining the three-dimensional information of the defect of the transparent or semitransparent medium.
According to one or more embodiments, as shown in fig. 1, a method for detecting a three-dimensional defect of a transparent or semitransparent medium based on a light field camera comprises the following steps:
and A1, selecting an optical lens with proper focal length and magnification according to the measurement area size and the measurement depth range of the transparent or semitransparent medium. Adjusting the lens aperture to match the aperture of the light field camera, i.e. the micro lens aperture matches the main lens aperture, is embodied in that the light field camera shoots a defocused soft light solid color calibration plate image in which the micro lens array is just or nearly tangent. The light field white image or the light field camera white image refers to a pure white background image shot by the light field camera, and the shape of the micro lens array is particularly obvious on the image. The aperture can thus be adjusted based on the image, ensuring that the microlens image is exactly tangential.
After the adjustment is finished, a plurality of pure-color background plates with uniform light intensity at the defocusing position of the light field camera are shot, namely the defocusing soft light pure-color calibration plate. Averaging and normalizing a plurality of light field white images to obtain a vignetting removing matrix
All the light field original images shot subsequently need to be point-divided by the vignetting removing matrix, and therefore light field white image calibration is completed. The light field original image refers to a light field image which is not processed by the light field multi-view image algorithm.
After the light field packet image calibration step is completed, processing the light field white image by using a filter, removing noise of the light field white image, and performing non-maximum value suppression on the filtered light field image; then, according to the processed image, taking a local maximum value, wherein the maximum value is just the integer-level center of the light field camera micro-lens; and (3) taking the center of the integer level microlens as an initial iteration value, iteratively optimizing the arrangement grid of the microlenses, finally obtaining the arrangement angle and the arrangement distance of the microlenses, and obtaining the center of the subpixel level microlens.
A2, the light field camera scale calibration step needs to assemble a displacement table and a scale calibration plate: firstly, fixing a scale calibration plate in the focal plane area of the light field camera, continuously moving the calibration plate to a fixed spatial distance from the focal plane, and shooting, wherein the spatial position of a point on the calibration plate is known, so that the spatial position of the point on the calibration plate in the whole moving process can be obtained. And the dot calibration points form a diffusion circle on the light field image, the diameter of the diffusion circle is obtained through processing, the parallax value of the diffusion circle and the pixel coordinate of the diffusion circle are further obtained through calculation, and the relation between the three-dimensional coordinate in the space and the pixel coordinate and the parallax value of the light field camera is obtained through fitting according to the light field camera scale calibration model.
A3, irradiating a plurality of angle light sources according to the measurement requirements of the transparent or semitransparent medium to be measured, wherein the plurality of angle light sources irradiate three-dimensional defects on the surface and inside of the transparent or semitransparent medium as shown in FIG. 2, so that the three-dimensional defects can be imaged by a light field camera; the light source is not limited in this process and can be, for example, a ring light source, a backlight, a bar light source, a sphere integral light source, a dome light source, a coaxial light source, or the like.
A4, performing conventional light field rendering and depth estimation based on the light field camera raw light field image containing defect information. Firstly, performing light field multi-view rendering to obtain a light field multi-view image with defect information; and then further calculating to obtain a light field parallax image, and converting the light field parallax image into a light field depth image according to a light field camera scale calibration result, wherein the depth image also contains depth information of the defect.
A5, the light field multi-view image is not different from the conventional two-dimensional camera image in nature, and can be regarded as that a plurality of two-dimensional cameras with different angles shoot the same object, so that the multi-view image can be subjected to defect extraction. The defect identification and extraction can use various methods, such as shape-based template matching, gray-scale-based contour extraction, defect identification based on a convolutional neural network and the like to extract the pixel coordinates of the defects in the image and eliminate non-defect areas on the depth image.
And A6, mapping the depth images of only the residual defect areas into a space coordinate system based on the light field scale calibration result of A2, and finally forming a three-dimensional point cloud of the defect. And finally, obtaining gray information, three-dimensional coordinate information and shape information of the defect.
The invention discloses an embodiment of a transparent or semitransparent medium three-dimensional defect detection method based on a light field camera, and belongs to the technical field of image detection. A light field camera is adopted to match with a lens with a proper aperture and a proper focal length, then a defocusing soft light pure color calibration plate is shot, and light field white image calibration and micro lens center calibration are carried out; shooting a plurality of scale calibration plates with different spatial positions by using the camera to calibrate the scale of the light field camera; the light source with a proper angle is matched to irradiate the transparent or semitransparent medium to be detected, so that the defects can be imaged by the camera; the method comprises the following steps that a light field camera shoots a detected transparent or semitransparent medium defect area, and light field multi-view rendering and depth calculation are carried out; removing non-defect points according to defect shapes and color information in the multi-view images; and finally, calculating to obtain the three-dimensional position and shape information of the defect in the measured medium. The embodiment of the invention can accurately and efficiently acquire the three-dimensional position and shape information of the internal and surface defects of the transparent or semitransparent medium, and effectively solve the problems of the existing equipment inspection and manual inspection.
Hereinafter, 3 different examples are given according to different types of detection media, namely, a three-dimensional defect detection example of a transparent screen of a mobile phone, a three-dimensional defect detection example of a transparent Polyimide (CPI) film of an organic light-Emitting layer in an organic light-Emitting semiconductor (OLED), and a three-dimensional defect detection example of a translucent polarizer.
The first embodiment comprises the following specific processes:
the whole system firstly adopts a two-dimensional camera to carry out whole mobile phone screen shooting, and is matched with a plurality of light sources with different angles to carry out irradiation, wherein the light sources comprise a barrel-shaped light source right above a measured object, side light sources irradiating in 4 side directions above the measured object and a backlight light source right below the measured object; after the defects on the mobile phone screen are clearly irradiated, the two-dimensional camera shoots; determining the relative position of the defect on the mobile phone screen by the horizontal position of the defect in the two-dimensional image in the image; transmitting the information to a conveyor belt; the transmission belt moves the tested sample to a position where the visual field of the light field camera just covers the defect;
according to the size (thickness 0.7mm) of the mobile phone screen glass and the corresponding defects and dust sizes, the embodiment adopts a 4-time magnification lens to shoot; the light field camera is matched with a lens with a proper diaphragm and a proper focal length to shoot a defocusing soft light pure color calibration plate, and light field white image calibration and micro-lens center calibration are carried out; the light field camera shoots a plurality of scale calibration plates with different spatial positions to calibrate the light field camera scale; matching a barrel-shaped light source, wherein the barrel-shaped light source covers the upper part of the screen glass of the mobile phone to be detected, and the barrel-shaped light source emits blue light from three angles, namely from top to bottom to irradiate an object to be detected, so that defects and dust on the transparent screen can be shot by the light field camera to form images; the light field camera carries out light field multi-view rendering and depth calculation to obtain a corresponding two-dimensional central view image (figure 3-a) of the mobile phone screen defect and a corresponding depth image (figure 3-b); removing non-defect points according to defect shapes and color information in the multi-view images; and finally, calculating to obtain the three-dimensional point cloud information (position information) and the shape information of the defects in the measured medium.
The second embodiment comprises the following specific processes:
the whole system firstly adopts a two-dimensional camera to carry out integral polaroid shooting and matches with a plurality of light sources with different angles to carry out irradiation, wherein the light sources comprise a barrel-shaped light source right above a measured object, side light sources irradiating in 4 side directions above the measured object and a backlight light source right below the measured object; after the defects on the polaroid are clearly irradiated, shooting by a two-dimensional camera; determining the relative position of the defect on the polaroid by aiming at the horizontal position of the defect in the two-dimensional image in the image; transmitting the information to a conveyor belt; the transmission belt moves the tested sample to a position where the visual field of the light field camera just covers the defect;
according to the size (thickness 1.1mm) of the polaroid and the size of corresponding defects and dust, the embodiment adopts a lens with 4 times of magnification to shoot; the light field camera is matched with a lens with a proper diaphragm and a proper focal length to shoot a defocusing soft light pure color calibration plate, and light field white image calibration and micro-lens center calibration are carried out; the light field camera shoots a plurality of scale calibration plates with different spatial positions to calibrate the light field camera scale; the two side light sources are arranged above the semi-transparent polaroid to be detected in an inclined angle direction of 45 degrees and are symmetrically arranged based on a main optical axis of the camera, so that the defects and dust of the semi-transparent polaroid can be shot and imaged by the light field camera; the light field camera carries out light field multi-view rendering and depth calculation to obtain a corresponding two-dimensional central view image (figure 4-a) of the mobile phone screen defect and a corresponding depth image (figure 4-b); removing non-defect points according to defect shapes and color information in the multi-view images; and finally, calculating to obtain the three-dimensional point cloud information (position information) and the shape information of the defects in the measured medium.
The third embodiment comprises the following specific processes:
the whole system firstly adopts a two-dimensional camera to shoot the whole transparent CPI film, and is matched with a plurality of light sources with different angles to carry out irradiation, wherein the light sources comprise a barrel-shaped light source right above a measured object, side light sources irradiating in 4 side directions above the measured object and a backlight light source right below the measured object; after the defects on the transparent CPI film are clearly irradiated, shooting by a two-dimensional camera; determining the relative position of the defect on the transparent CPI film by aiming at the horizontal position of the defect in the two-dimensional image in the image; transmitting the information to a conveyor belt; the transmission belt moves the tested sample to a position where the visual field of the light field camera just covers the defect;
according to the size (thickness is 0.12mm) of the transparent CPI film and the corresponding size of the defects and dust, the embodiment adopts a lens matched with 6 times of magnification to shoot; the light field camera is matched with a lens with a proper diaphragm and a proper focal length to shoot a defocusing soft light pure color calibration plate, and light field white image calibration and micro-lens center calibration are carried out; the light field camera shoots a plurality of scale calibration plates with different spatial positions to calibrate the light field camera scale; the side light source is arranged below the transparent CPI film to be detected, and the irradiation direction of the side light source is superposed with the main optical axis of the camera, so that the defects and dust of the transparent CPI film can be shot and imaged by the light field camera; the light field camera carries out light field multi-view rendering and depth calculation to obtain a corresponding two-dimensional central view image (figure 5-a) of the mobile phone screen defect and a corresponding depth image (figure 5-b); removing non-defect points according to defect shapes and color information in the multi-view images; and finally, calculating to obtain the three-dimensional point cloud information (position information) and the shape information of the defects in the measured medium.
In accordance with one or more embodiments, in the inspection process, in addition to inspecting the transparent or translucent medium for defects, it is also necessary to locate the three-dimensional positions of the upper and lower surfaces of the medium. Therefore, the three-dimensional position detection method for the upper surface and the lower surface of the transparent or semitransparent medium based on the light field camera can be used for detecting the three-dimensional position of the upper surface and the lower surface of the transparent or semitransparent medium, wherein the medium can be a single layer, or one of multiple layers of superposed and attached media, such as a mobile phone screen glass cover plate. The detection method comprises the following steps:
and A1, selecting an optical lens with proper focal length and magnification according to the measurement area size and the measurement depth range of the transparent or semitransparent medium. Adjusting the lens aperture to match the aperture of the light field camera, i.e. the micro lens aperture matches the main lens aperture, is embodied in that the light field camera shoots a defocused soft light solid color calibration plate image in which the micro lens array is just or nearly tangent. The light field white image or the light field camera white image refers to a pure white background image shot by the light field camera, and the shape of the micro lens array is particularly obvious on the image. The aperture can thus be adjusted based on the image, ensuring that the microlens image is exactly tangential.
After the adjustment is finished, a plurality of pure-color background plates with uniform light intensity at the defocusing position of the light field camera are shot, namely the defocusing soft light pure-color calibration plate. Averaging and normalizing a plurality of light field white images to obtain a vignetting removing matrix
All light field original images of subsequent shotsThe dots need to be divided by the de-vignetting matrix to complete the light field white image calibration. The light field original image refers to a light field image which is not processed by the light field multi-view image algorithm.
After the light field packet image calibration step is completed, processing the light field white image by using a filter, removing noise of the light field white image, and performing non-maximum value suppression on the filtered light field image; then, according to the processed image, taking a local maximum value, wherein the maximum value is just the integer-level center of the light field camera micro-lens; and (3) taking the center of the integer level microlens as an initial iteration value, iteratively optimizing the arrangement grid of the microlenses, finally obtaining the arrangement angle and the arrangement distance of the microlenses, and obtaining the center of the subpixel level microlens.
A2, the light field camera scale calibration step needs to assemble a displacement table and a scale calibration plate: firstly, fixing a scale calibration plate in the focal plane area of the light field camera, continuously moving the calibration plate to a fixed spatial distance from the focal plane, and shooting, wherein the spatial position of a point on the calibration plate is known, so that the spatial position of the point on the calibration plate in the whole moving process can be obtained. And the dot calibration points form a diffusion circle on the light field image, the diameter of the diffusion circle is obtained through processing, the parallax value of the diffusion circle and the pixel coordinate of the diffusion circle are further obtained through calculation, and the relation between the three-dimensional coordinate in the space and the pixel coordinate and the parallax value of the light field camera is obtained through fitting according to the light field camera scale calibration model.
A3, irradiating the light source at multiple angles according to the measurement requirement of the transparent or semitransparent medium to be measured, wherein the light source can be emitted from the upper side or the lower side as shown in FIGS. 2-3; a light source injected from the upper side is firstly reflected on the upper surface of the medium to form a first texture point, then partial light is refracted to enter the medium, and a reflection phenomenon is generated on the lower surface to generate a second texture point; the light source emitted from the lower side is firstly reflected on the lower surface of the medium to form a first texture point, then part of light rays are refracted to enter the medium, and a reflection phenomenon is generated on the upper surface to generate a second texture point. The light source in this process is not limited, and for example, a coherent/incoherent point laser light source, a point light source, a line light source, an area array light source, and the like can be used.
A4, performing conventional light field rendering and depth estimation based on the light field camera raw light field image containing texture information. Firstly, performing light field multi-view rendering to obtain a texture light field multi-view image of meaning medium information; and then further calculating to obtain a light field parallax image, and converting the light field parallax image into a light field depth image according to a light field camera scale calibration result, wherein the depth image also comprises depth information of upper and lower surface textures of the medium.
A5, based on the depth information of the texture in A4, the three-dimensional position information of the upper and lower surfaces of the medium can be further deduced.
It should be noted that while the foregoing has described the spirit and principles of the invention with reference to several specific embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in these aspects cannot be combined. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (10)
1. A transparent or translucent media defect detection system, comprising,
the transparent or translucent medium to be detected,
at least one light source for irradiating light to the transparent or translucent medium,
a light field camera for capturing images of the transparent or translucent medium for detecting three-dimensional defects in the transparent or translucent medium, the detecting step comprising:
shooting a plurality of defocusing soft light plates by using the light field camera after the light rings are matched, calibrating the white image of the light field, and completing the center calibration of the micro lens; calibrating the scale of the light field camera, and setting up a light source with a proper angle; and shooting the defect area of the detected medium by using a light field camera, processing to obtain a multi-view and depth image, removing non-defect points in the depth image based on the shape and color of the defect in the multi-view image, and finally obtaining the three-dimensional information of the defect of the transparent or semitransparent medium.
2. The system of claim 1, wherein the method for detecting the presence of the three-dimensional defect in the transparent or translucent medium comprises the steps of:
a1, adjusting the focal length and the aperture of the light field camera, shooting a plurality of defocusing soft light pure color calibration plates, acquiring a white image of the light field camera,
calculating according to the white image of the light field camera to obtain a vignetting removing matrix and a light field camera micro-lens sub-pixel level central coordinate matrix;
a2, shooting a plurality of dot calibration plates with known spatial three-dimensional positions by using the light field camera, establishing a light field mathematical model from three-dimensional coordinates to parallax, and completing the scale calibration of the light field camera;
a3, irradiating the light source to the measured transparent or semi-transparent medium to enable the possible defects of the transparent or semi-transparent medium to be shot and imaged by the light field camera;
a4, shooting the region where the transparent or semitransparent medium defect is located by using the light field camera, and performing light field multi-view rendering and depth calculation to obtain a light field multi-view image and a light field depth image;
a5, removing non-defect points in the light field depth image according to the shape and color information of the defects in the light field multi-view image;
and A6, calculating the three-dimensional position and shape information of the internal and surface defects of the transparent or semitransparent medium.
3. The transparent or translucent medium defect detection system of claim 2, wherein step A1 further comprises,
adjusting a main lens and an aperture of the light field camera to ensure that the micro-lens array of the original light field white image of the light field camera is just or approximately tangent;
making the light field camera shoot a plurality of defocusing soft light pure color calibration plate images, wherein the calibration plate is positioned on a pure color background plate with uniform light intensity at the defocusing position of the light field camera, wherein,
the vignetting removing matrix is a matrix obtained by averaging and normalizing a plurality of original light field white images W (u, v)
The microlens subpixel level central coordinate matrix of the light field camera is used for white image of light field
And after the local maximum value of each micro lens is processed, performing iterative optimization to obtain the center coordinate of the sub-pixel micro lens.
4. The system of claim 3, wherein the three-dimensional coordinates of each dot on the dot calibration plate in step A2 are known, and the degree of diffusion and the corresponding parallax value of the dots on the calibration plate are obtained after the calibration plate is photographed by a light field camera, and then the relationship between the parallax value and the three-dimensional coordinates is obtained by fitting calibration.
5. The system of claim 4, wherein the light source is angled in step A3 so that the light source can illuminate the surface and internal defects of the transparent or translucent medium clearly and ensure that the light field camera images the defects of the transparent or translucent medium.
6. The system of claim 5, wherein the light field camera captures the defect image in step A4, and then performs light field multi-view rendering to obtain a light field multi-view image and a light field parallax image, and the parallax image is converted into a depth image according to the parallax obtained by calibration in step A2 and the conversion relationship between the three-dimensional coordinates.
7. The transparent or translucent medium defect detection system of claim 6, wherein for the shape and color information of the defect in the light field multi-view image in step a5, classification and identification are performed according to the shape common to the defect and the color information under the illumination, and the position of the defect in the light field multi-view image is further located, and the non-defect region is removed in the depth image.
8. The system of claim 7, wherein step A6 is performed by three-dimensional mapping the depth image based on the scale calibration result to obtain three-dimensional point cloud information of the defect.
9. A method for detecting defects of a transparent or semitransparent medium, which comprises the steps of setting a light source to irradiate the transparent or semitransparent medium, shooting the transparent or semitransparent medium by using a light field camera, and detecting the defects in the transparent or semitransparent medium, wherein the method comprises the following steps:
a1, adjusting the focal length and the aperture of the light field camera, shooting a plurality of defocusing soft light pure color calibration plates, acquiring a white image of the light field camera,
calculating according to the white image of the light field camera to obtain a vignetting removing matrix and a light field camera micro-lens sub-pixel level central coordinate matrix;
a2, shooting a plurality of dot calibration plates with known spatial three-dimensional positions by using the light field camera, establishing a light field mathematical model from three-dimensional coordinates to parallax, and completing the scale calibration of the light field camera;
a3, irradiating the light source to the measured transparent or semi-transparent medium to enable the possible defects of the transparent or semi-transparent medium to be shot and imaged by the light field camera;
a4, shooting the region where the transparent or semitransparent medium defect is located by using the light field camera, and performing light field multi-view rendering and depth calculation to obtain a light field multi-view image and a light field depth image;
a5, removing non-defect points in the light field depth image according to the shape and color information of the defects in the light field multi-view image;
and A6, calculating the three-dimensional position and shape information of the internal and surface defects of the transparent or semitransparent medium.
10. A light field camera for detecting defects of a transparent or translucent medium illuminated by a light source, the detection method of the light field camera comprising the steps of:
a1, adjusting the focal length and the aperture of the light field camera, shooting a plurality of defocusing soft light pure color calibration plates, acquiring a white image of the light field camera,
calculating according to the white image of the light field camera to obtain a vignetting removing matrix and a light field camera micro-lens sub-pixel level central coordinate matrix;
a2, shooting a plurality of dot calibration plates with known spatial three-dimensional positions by using the light field camera, establishing a light field mathematical model from three-dimensional coordinates to parallax, and completing the scale calibration of the light field camera;
a3, irradiating the light source to the measured transparent or semi-transparent medium to enable the possible defects of the transparent or semi-transparent medium to be shot and imaged by the light field camera;
a4, shooting the region where the transparent or semitransparent medium defect is located by using the light field camera, and performing light field multi-view rendering and depth calculation to obtain a light field multi-view image and a light field depth image;
a5, removing non-defect points in the light field depth image according to the shape and color information of the defects in the light field multi-view image;
and A6, calculating the three-dimensional position and shape information of the internal and surface defects of the transparent or semitransparent medium.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010350740 | 2020-04-28 | ||
| CN2020103507403 | 2020-04-28 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112824881A true CN112824881A (en) | 2021-05-21 |
Family
ID=75907545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010412397.0A Withdrawn CN112824881A (en) | 2020-04-28 | 2020-05-15 | System and method for detecting defects of transparent or semitransparent medium based on light field camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112824881A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113406111A (en) * | 2021-05-31 | 2021-09-17 | 清华大学深圳国际研究生院 | Defect detection method and device based on structural light field video stream |
| CN113473031A (en) * | 2021-09-06 | 2021-10-01 | 华能南京金陵发电有限公司 | Dust leakage monitoring device and method |
| CN113567464A (en) * | 2021-06-16 | 2021-10-29 | 美晟通科技(苏州)有限公司 | Transparent medium stain position detection method and device |
| CN113920090A (en) * | 2021-10-13 | 2022-01-11 | 无锡雪浪数制科技有限公司 | Preform appearance defect automatic detection method based on deep learning |
| CN113984790A (en) * | 2021-09-28 | 2022-01-28 | 歌尔光学科技有限公司 | Lens quality detection method and device |
| CN116148277A (en) * | 2023-04-19 | 2023-05-23 | 武汉精一微仪器有限公司 | Three-dimensional detection method, device and equipment for defects of transparent body and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2580551A1 (en) * | 2004-09-17 | 2006-03-23 | De.Vice Scientific Incorporated | Optical inspection of flat media using direct image technology |
| CN106918306A (en) * | 2017-04-22 | 2017-07-04 | 许晟明 | Industrial products three-dimensional appearance real-time detecting system based on light field one camera |
| CN107957425A (en) * | 2017-12-08 | 2018-04-24 | 湖南科创信息技术股份有限公司 | Transparent material defect detecting system and method |
| CN109146794A (en) * | 2018-09-29 | 2019-01-04 | 哈尔滨工业大学 | A kind of light field image rotation error bearing calibration |
| CN109166154A (en) * | 2018-08-29 | 2019-01-08 | 上海交通大学 | Light-field camera calibration method for light field three dimensional particles image reconstruction |
-
2020
- 2020-05-15 CN CN202010412397.0A patent/CN112824881A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2580551A1 (en) * | 2004-09-17 | 2006-03-23 | De.Vice Scientific Incorporated | Optical inspection of flat media using direct image technology |
| CN106918306A (en) * | 2017-04-22 | 2017-07-04 | 许晟明 | Industrial products three-dimensional appearance real-time detecting system based on light field one camera |
| CN107957425A (en) * | 2017-12-08 | 2018-04-24 | 湖南科创信息技术股份有限公司 | Transparent material defect detecting system and method |
| CN109166154A (en) * | 2018-08-29 | 2019-01-08 | 上海交通大学 | Light-field camera calibration method for light field three dimensional particles image reconstruction |
| CN109146794A (en) * | 2018-09-29 | 2019-01-04 | 哈尔滨工业大学 | A kind of light field image rotation error bearing calibration |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113406111A (en) * | 2021-05-31 | 2021-09-17 | 清华大学深圳国际研究生院 | Defect detection method and device based on structural light field video stream |
| WO2022252698A1 (en) * | 2021-05-31 | 2022-12-08 | 清华大学深圳国际研究生院 | Defect detection method and apparatus based on structured light field video stream |
| CN113567464A (en) * | 2021-06-16 | 2021-10-29 | 美晟通科技(苏州)有限公司 | Transparent medium stain position detection method and device |
| CN113473031A (en) * | 2021-09-06 | 2021-10-01 | 华能南京金陵发电有限公司 | Dust leakage monitoring device and method |
| CN113984790A (en) * | 2021-09-28 | 2022-01-28 | 歌尔光学科技有限公司 | Lens quality detection method and device |
| CN113920090A (en) * | 2021-10-13 | 2022-01-11 | 无锡雪浪数制科技有限公司 | Preform appearance defect automatic detection method based on deep learning |
| CN113920090B (en) * | 2021-10-13 | 2022-08-30 | 无锡雪浪数制科技有限公司 | Prefabricated rod appearance defect automatic detection method based on deep learning |
| CN116148277A (en) * | 2023-04-19 | 2023-05-23 | 武汉精一微仪器有限公司 | Three-dimensional detection method, device and equipment for defects of transparent body and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112824881A (en) | System and method for detecting defects of transparent or semitransparent medium based on light field camera | |
| CN112748071A (en) | Transparent or translucent medium defect detection system and method | |
| CN113205593B (en) | High-light-reflection surface structure light field three-dimensional reconstruction method based on point cloud self-adaptive restoration | |
| CN104361315B (en) | 3D fingerprint identification device based on monocular multi-view solid machine vision | |
| CN107869954B (en) | Binocular vision volume weight measurement system and implementation method thereof | |
| CN110657785A (en) | Efficient scene depth information acquisition method and system | |
| CN109883391B (en) | Monocular distance measurement method based on digital imaging of microlens array | |
| CN1735789A (en) | Ranging apparatus | |
| CN106289099B (en) | A kind of single camera vision system and the three-dimensional dimension method for fast measuring based on the system | |
| CN106500629B (en) | Microscopic three-dimensional measuring device and system | |
| CN114693583A (en) | Defect layering detection method and system based on light field camera and detection production line | |
| WO2022126870A1 (en) | Three-dimensional imaging method and method based on light field camera and three-dimensional imaging measuring production line | |
| CN110554054B (en) | Three-dimensional blade breakage detection system of mechanically-clamped indexable turning tool | |
| CN108474720B (en) | Apparatus and method for inspecting tyres | |
| TW201337211A (en) | Shape reflector and surface contour mapping | |
| CN112001917A (en) | Machine vision-based geometric tolerance detection method for circular perforated part | |
| CN112747670A (en) | BGA packaging solder ball detection system and method | |
| CN114280075B (en) | Online visual detection system and detection method for surface defects of pipe parts | |
| CN105158275B (en) | Apple stem/calyx detection method based on near-infrared coding dot matrix structure light | |
| CN112767302A (en) | Camera module defect detection system and method | |
| CN212301365U (en) | Light source device and defect detection system | |
| CN116577349B (en) | System and method for detecting defects on smooth surface of semiconductor | |
| CN112747822A (en) | Three-dimensional imaging system and method | |
| CN112747692A (en) | Three-dimensional measurement method and device for precise small hole | |
| CN113923445B (en) | Light field camera calibration method and system under shift imaging condition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 201100 room 1206, building 1, No. 951, Jianchuan Road, Minhang District, Shanghai Applicant after: Yimu (Shanghai) Technology Co.,Ltd. Address before: 201109 room 1103, building 1, 951 Jianchuan Road, Minhang District, Shanghai Applicant before: Yimu (Shanghai) Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210521 |
|
| WW01 | Invention patent application withdrawn after publication |