CN116106318A - Object surface defect detection method and device and three-dimensional scanner - Google Patents
Object surface defect detection method and device and three-dimensional scanner Download PDFInfo
- Publication number
- CN116106318A CN116106318A CN202310104898.6A CN202310104898A CN116106318A CN 116106318 A CN116106318 A CN 116106318A CN 202310104898 A CN202310104898 A CN 202310104898A CN 116106318 A CN116106318 A CN 116106318A
- Authority
- CN
- China
- Prior art keywords
- matrix
- light source
- dimensional scanner
- image
- target object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/01—Arrangements or apparatus for facilitating the optical investigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8806—Specially adapted optical and illumination features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N2201/00—Features of devices classified in G01N21/00
- G01N2201/06—Illumination; Optics
- G01N2201/062—LED's
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Health & Medical Sciences (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Pathology (AREA)
- Analytical Chemistry (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses an object surface defect detection scheme, which belongs to the technical field of three-dimensional scanners, and comprises the following steps: scanning the surface of a target object by using a three-dimensional scanner, wherein a light source on the three-dimensional scanner is subjected to angle-division rapid sequential exposure according to a calibrated light source direction calibration sequence in the scanning process, and simultaneously, the three-dimensional scanner performs image acquisition on the surface of the target object to obtain a first image; acquiring a normal vector matrix and a reflectivity distribution matrix of the surface of a target object based on a light source direction matrix obtained when the light source direction is calibrated by a photometric stereo three-dimensional reconstruction technology; generating a gradient distribution matrix of the surface of the target object based on the normal vector matrix; and detecting concave-convex flaws and scratches existing on the surface of the target object based on the gradient distribution matrix and the reflectivity distribution matrix. By the object surface defect detection scheme provided by the invention, the defects are detected more comprehensively and positioned more accurately.
Description
Technical Field
The present invention relates to the field of three-dimensional scanners, and in particular, to a method and an apparatus for detecting a surface defect of an object, and a three-dimensional scanner.
Background
A three-dimensional scanner is a scientific instrument used to detect and analyze shape and appearance data of objects or environments in the real world. The collected data is often used to perform three-dimensional reconstruction calculations to create a digital model of the real object in the virtual world. The purpose of a three-dimensional scanner is to create a point cloud of the geometric surface of an object, which points can be used to interpolate the surface shape of the object, denser point clouds can create more accurate models, and the process of creating models based on point clouds is called three-dimensional reconstruction. If the three-dimensional scanner is capable of acquiring surface color, a texture map, so-called texture printing, may be further attached to the reconstructed surface.
The existing three-dimensional scanner is difficult to detect and identify fine concave-convex flaws and scratches on the surface of an object due to the limitation of resolution, and finally the accuracy of the created three-dimensional model is affected.
Disclosure of Invention
The embodiment of the invention aims to provide a method and a device for detecting surface defects of an object and a three-dimensional scanner, which can solve the problem that the existing three-dimensional scanner is difficult to detect and identify fine concave-convex flaws and scratches on the surface of the object.
In order to solve the technical problems, the invention provides the following technical scheme:
the embodiment of the invention provides a method for detecting surface defects of an object, wherein the method is applied to a three-dimensional scanner, and the method comprises the following steps:
scanning the surface of a target object by using a three-dimensional scanner, wherein a light source on the three-dimensional scanner is subjected to angle-division rapid sequential exposure according to a calibrated light source direction calibration sequence in the scanning process, and simultaneously, the three-dimensional scanner performs image acquisition on the surface of the target object to obtain a first image;
acquiring a normal vector matrix and a reflectivity distribution matrix of the surface of a target object based on a light source direction matrix obtained when the light source direction is calibrated by a photometric stereo three-dimensional reconstruction technology;
generating a gradient distribution matrix of the surface of the target object based on the normal vector matrix;
and detecting concave-convex flaws and scratches existing on the surface of the target object based on the gradient distribution matrix and the reflectivity distribution matrix.
Optionally, before the step of scanning the surface of the target object using the three-dimensional scanner, the method further comprises:
and calibrating the direction of the light source on the three-dimensional scanner through a space curved object with known reflectivity to obtain a light source direction matrix when the light source is subjected to angle-division rapid and sequential exposure.
Optionally, the step of calibrating the light source direction of the three-dimensional scanner through the space curved object with known reflectivity to obtain a light source direction matrix when the light source is exposed in rapid sequence in different angles comprises the following steps:
scanning a space curved object with known reflectivity by using the three-dimensional scanner, enabling a light source of the three-dimensional scanner to be exposed at an angle in sequence, and acquiring an image of the space curved object by using the three-dimensional scanner to obtain a second image;
obtaining the surface point cloud data of the space curved surface object based on the second image;
determining a normal vector matrix of the surface of the space curved object according to the point cloud data;
and obtaining a light source direction matrix when the light sources on the three-dimensional scanner are subjected to angle-division rapid and sequential exposure based on the gray matrix and the normal vector matrix of the second image.
Optionally, the step of obtaining the spatial curved surface object surface point cloud data based on the second image includes:
determining a full bright image and an encoded image in the second image;
and processing the full-bright image and the coded image through multi-eye imaging operation to obtain the surface point cloud data of the space curved object.
Optionally, based on the gray matrix and the normal vector matrix of the second image, the step of obtaining the light source direction matrix when the light sources are exposed in rapid and sequential directions in different angles on the three-dimensional scanner includes:
solving a curved surface normal vector at each pixel based on a space curved surface equation of a preset reference object aiming at any pixel under the illumination of a light source;
calculating to obtain a unit light source incidence vector at the pixel based on the normal vector of the curved surface, the reflected illumination, the reflectivity at the pixel, the normal vector at the illumination and a preset equation set;
and obtaining a light source direction matrix when the light sources are subjected to angle-division rapid and sequential exposure on the three-dimensional scanner through the incidence vectors of the unit light sources at each pixel under the irradiation of the light sources.
Optionally, the step of generating a gradient distribution matrix of the target object surface based on the normal vector matrix includes:
determining a distribution of gradients of the image in a X, Y direction based on the normal vector matrix;
obtaining gradient distribution of the image according to the distribution of the gradient of the image in the X, Y direction;
and determining a gradient distribution matrix of the surface of the target object based on the image gradient distribution.
Optionally, the step of detecting the concave-convex flaws and scratches existing on the surface of the target object based on the gradient distribution matrix and the reflectivity distribution matrix includes:
fusing the gradient distribution matrix, the reflectivity distribution matrix and the image gray matrix acquired by the three-dimensional scanner camera to form multichannel target surface information to be detected;
and detecting defects existing on the surface of the target object based on the information of the surface of the target to be detected of the multiple channels.
The embodiment of the invention also provides an object surface defect detection device which is applied to a three-dimensional scanner, and the device comprises:
the control module is used for scanning the surface of the target object by using the three-dimensional scanner, the light source on the three-dimensional scanner is rapidly and sequentially exposed in an angle-dividing manner according to the calibrated light source direction calibration sequence in the scanning process, and the three-dimensional scanner acquires images of the surface of the target object to obtain a first image;
the acquisition module is used for acquiring a normal vector matrix and a reflectivity distribution matrix on the surface of the target object based on a light source direction matrix obtained when the light source direction is calibrated through a photometric stereo three-dimensional reconstruction technology;
the generation module is used for generating a gradient distribution matrix of the surface of the target object based on the normal vector matrix;
and the detection module is used for detecting concave-convex flaws and scratches on the surface of the target object based on the gradient distribution matrix and the reflectivity distribution matrix.
Optionally, the apparatus further comprises:
the calibration module is used for calibrating the light source direction on the three-dimensional scanner through a space curved object with known reflectivity before the control module scans the surface of the target object by using the three-dimensional scanner, so as to obtain a light source direction matrix when the light sources are exposed in a rapid and sequential manner in different angles.
Optionally, the calibration module includes:
the first sub-module is used for scanning a space curved object with known reflectivity by using the three-dimensional scanner, the light source of the three-dimensional scanner is subjected to angle-division rapid and sequential exposure, and the three-dimensional scanner performs image acquisition on the space curved object to obtain a second image;
a second sub-module, configured to obtain the spatial curved surface object surface point cloud data based on the second image;
the third sub-module is used for determining a normal vector matrix of the surface of the space curved object according to the point cloud data;
and the fourth sub-module is used for obtaining a light source direction matrix when the light sources on the three-dimensional scanner are subjected to angle-division rapid and sequential exposure based on the gray matrix and the normal vector matrix of the second image.
Optionally, the second submodule is specifically configured to:
determining a full bright image and an encoded image in the second image; and processing the full-bright image and the coded image through multi-eye imaging operation to obtain the surface point cloud data of the space curved object.
Optionally, the fourth submodule is specifically configured to:
solving a curved surface normal vector at each pixel based on a space curved surface equation of a preset reference object aiming at any pixel under the illumination of a light source;
calculating to obtain a unit light source incidence vector at the pixel based on the normal vector of the curved surface, the reflected illumination, the reflectivity at the pixel, the normal vector at the illumination and a preset equation set;
and obtaining a light source direction matrix when the light sources are subjected to angle-division rapid and sequential exposure on the three-dimensional scanner through the incidence vectors of the unit light sources at each pixel under the irradiation of the light sources.
Optionally, the generating module includes:
a fifth sub-module for determining a distribution of gradients of the image in a X, Y direction based on the normal vector matrix;
a sixth sub-module, configured to obtain gradient distribution of the image according to distribution of gradients of the image in a X, Y direction;
and a seventh sub-module, configured to determine a gradient distribution matrix of the target object surface based on the image gradient distribution.
Optionally, the detection module includes:
an eighth sub-module, configured to fuse the gradient distribution matrix, the reflectivity distribution matrix, and the image gray matrix acquired by the three-dimensional scanner camera to form multi-channel surface information of the target to be detected;
and a ninth sub-module, configured to detect a defect existing on the surface of the target object based on the information of the target surface to be detected of the multiple channels.
The embodiment of the invention provides electronic equipment, which comprises a processor, a memory and a program or instructions stored on the memory and capable of running on the processor, wherein the program or instructions realize the steps of any one of the object surface defect detection methods when being executed by the processor.
An embodiment of the present invention provides a readable storage medium having stored thereon a program or instructions which, when executed by a processor, implement the steps of any one of the above-described object surface defect detection methods.
According to the object surface defect detection scheme provided by the embodiment of the invention, the defects such as concave-convex flaws, scratches and the like on the surface of the target object are detected by combining the characteristic information such as normal vector distribution, reflectivity distribution, gradient distribution and the like on the surface of the target object, so that the defects are detected more comprehensively and positioned more accurately.
Drawings
FIG. 1 is a flow chart showing the steps of a method for detecting defects on an object surface according to an embodiment of the present application;
FIG. 2 is a schematic diagram showing the structure of a three-dimensional scanner;
FIG. 3 is a schematic diagram showing the layout of an LED light source and a camera in a three-dimensional scanner;
fig. 4 is a block diagram showing a structure of an object surface defect detecting apparatus according to an embodiment of the present application.
Detailed Description
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
The photometric stereo detection method is a method for estimating the surface geometry by using a plurality of light source directions, can reconstruct the normal vector of the surface of an object and the reflectivity of different surface points of the object, and has the best effect in reconstructing a continuous and smooth target surface. When the light intensity stereo detection method is used for reconstructing the target surface, the illumination directions of the LED light sources with different angles need to be marked in advance, the illumination directions are reversely pushed by shooting the highlight metal ball, the process is complicated, and a certain experience is required for an operator.
The existing three-dimensional scanner is difficult to detect and identify fine concave-convex flaws and scratches due to the limitation of resolution, and meanwhile, the detection of the fine concave-convex flaws and scratches can be negatively influenced by the emitted laser line during scanning; the photometric stereo detection method can accurately reconstruct the normal vector and reflectivity of the surface of the object, so that flaw detection is realized, but the light source direction calibration process is complex. In the application, the two are combined through creative labor, the three-dimensional scanner is used for calibrating the illumination direction of photometric stereo detection, and then the three-dimensional scanner is used for detecting micro concave-convex flaws and scratches.
The following describes in detail the object surface defect detection scheme provided in the embodiment of the present application through specific embodiments and application scenarios thereof with reference to the accompanying drawings.
As shown in fig. 1, the method for detecting the surface defects of the object according to the embodiment of the application includes the following steps:
step 101: and scanning the surface of the target object by using a three-dimensional scanner, wherein a light source on the three-dimensional scanner is subjected to angle-division rapid and sequential exposure according to the calibrated light source direction calibration sequence in the scanning process, and simultaneously, the three-dimensional scanner performs image acquisition on the surface of the target object to obtain a first image.
An exemplary three-dimensional scanner is schematically illustrated in fig. 2, the three-dimensional scanner comprising: a light source such as an LED (Light Emitting Diode ) light source for scanning an object to be scanned, a camera for emitting light, and a laser for capturing an image. The layout of the LED light sources and the camera in the three-dimensional scanner is schematically shown in fig. 3, and the LED light sources are uniformly distributed around the camera. It should be noted that fig. 3 is only an exemplary illustration, and the specific number and size of the LED light sources in the practical implementation are not limited to those in fig. 3, and may be flexibly set by those skilled in the art. The LED light sources may also be unevenly and symmetrically distributed.
In the embodiment of the application, before using a three-dimensional scanner to detect defects on the surface of a target object, the light source direction on the three-dimensional scanner is calibrated through a space curved object with known reflectivity, so as to obtain a light source direction matrix L when the light sources are exposed in a rapid and sequential manner in different angles.
The reflectivity of the target object, i.e. the object to be surface defect detected, is unknown. When the three-dimensional scanner is used for scanning the surface of a target object with unknown reflectivity, the sequence of the LED light sources for sequentially and rapidly exposing at different angles on the three-dimensional scanner is the same as the sequence of calibrating the light source directions, meanwhile, the LED light sources are attached to the three-dimensional scanner, the camera and the light sources are kept relatively static, and meanwhile, the three-dimensional scanner can be considered to keep the light source direction matrix L unchanged in the process of calibrating the light source directions and the formal scanning process because the camera is used for acquiring images very rapidly and the three-dimensional scanner is used for scanning the target object with relatively slow movement.
Step 102: and acquiring a normal vector matrix and a reflectivity distribution matrix of the surface of the target object based on a light source direction matrix obtained when the light source direction is calibrated by a photometric stereo three-dimensional reconstruction technology.
The mode of obtaining the normal vector matrix and the reflectivity distribution matrix of the surface of the target object can be as follows:
determining a light source direction matrix L, scanning the surface of a target object with unknown reflectivity by using a three-dimensional scanner, setting an LED light source on the three-dimensional scanner to be exposed for f times in turn at different angles, completing image acquisition of the surface of the target object by the three-dimensional scanner, and setting the image resolution as w x h, wherein the behavior f of the gray matrix M of the surface image of the target object is listed as w x h;
the formula m=elnp is shown, where M is a target surface image gray matrix with the behavior f and the columns w×h, and E is a light source intensity matrix diag (E 1 、e 2 ...e f ) L is an LED light source direction matrix with a row of 3, N is a target object surface unit normal vector matrix with a row of 3, P is a reflectivity distribution matrix with a row of w, wherein M, E, L is a known quantity, and N, P is an unknown quantity.
The formula m=elnp can be expressed as m=l 'N', where L '=el, N' =np, where N 'is solved using the least squares method (not uniquely) with N' = (L '' T L′) -1 L′ T
After solving for N ', there is N' =np
Therefore there is ρ i (n ix +n iy +n iz )＝n′ i ，n′ i Is a unit normal vector, so there is (n) ix +n iy +n iz )＝1，
Therefore:
and solving the unit normal vector matrix N on the surface of the target object and the reflectivity matrix P on the surface of the target object.
Step 103: and generating a gradient distribution matrix of the surface of the target object based on the normal vector matrix.
One way to generate a gradient distribution matrix for the surface of the target object based on the normal vector matrix can be as follows:
first, based on the normal vector matrix, determining a distribution of gradients of the image in the X, Y direction;
secondly, obtaining gradient distribution of the image according to the distribution of the gradient of the image in the X, Y direction;
then, a target object surface gradient distribution matrix G is determined based on the image gradient distribution.
Step 104: and detecting concave-convex flaws and scratches existing on the surface of the target object based on the gradient distribution matrix and the reflectivity distribution matrix.
In the actual implementation process, the gradient distribution matrix, the reflectivity distribution matrix and the gray matrix of the image acquired by the three-dimensional scanner camera can be fused to form multichannel target surface information to be detected; and detecting defects on the surface of the target object based on the multi-channel target surface information to be detected.
The specific manner of detecting the defects existing on the surface of the target object based on the multi-channel target surface information to be detected can be flexibly set by those skilled in the art, and the specific limitation is not made in the embodiment of the present application. The existing related scheme for detecting the surface defects based on the surface information can be used for reference.
In an alternative embodiment, the method for calibrating the light source direction of the three-dimensional scanner through the space curved object with known reflectivity to obtain the light source direction matrix when the light sources are exposed in rapid sequence at different angles may include the following substeps:
the method comprises the following substeps: scanning a space curved object with known reflectivity by using a three-dimensional scanner, wherein a light source of the three-dimensional scanner is subjected to rapid and sequential exposure at different angles, and simultaneously the three-dimensional scanner performs image acquisition on the space curved object to obtain a second image;
wherein the number of second images is 3 or more.
Sub-step two: obtaining space curved surface object surface point cloud data based on the second image;
one way of feasibility is: determining a full bright image and an encoded image in the second image; and processing the full-bright image and the coded image through multi-eye imaging operation to obtain the surface point cloud data of the space curved object.
And a sub-step three: determining a normal vector matrix of the surface of the space curved surface object according to the point cloud data;
and a sub-step four: and obtaining a light source direction matrix when the light sources on the three-dimensional scanner are subjected to angle-division rapid and sequential exposure based on the gray matrix and the normal vector matrix of the second image.
One way of feasibility may be as follows:
firstly, solving a curved surface normal vector at each pixel based on a space curved surface equation of a preset reference object aiming at any pixel under illumination of a light source;
let the normal vector of the surface at any point be (a, B) normalized By the spatial surface equation ax+by+cz+d=0
Secondly, calculating to obtain a unit light source incidence vector at the pixel based on a curved surface normal vector, reflected illumination, reflectivity at the pixel, the normal vector at the illumination and a preset equation set;
for the lambertian reflection, there is an established I.alpha.cos θ
I is that reflected light is acquired by a camera, k is a proportionality coefficient, theta is the included angle between a unit normal vector n of the light and an incident vector l of a unit light source, e is the light intensity of the point, which can be regarded as the unit light intensity, ρ is the reflectivity of the point, and l= (l) is set x ，l y ，l z ) Then
Wherein I, ρ,
It is known that the incident vector l= (l) of the unit light source at the pixel can be calculated by three equations x ，l y ，l z )。
Finally, the light source direction matrix of the three-dimensional scanner is obtained by the incidence vector of the unit light source at each pixel under the irradiation of the light source when the light source is exposed in a rapid and sequential manner at different angles.
The calculation mode of determining the incidence vector of the unit light source at each pixel is applied to the calculation of the incidence vector of the unit light source at a plurality of pixels under a plurality of illumination angles, and the light source direction matrix L can be obtained from the target surface image gray matrix, the normal vector matrix and the reflectivity.
According to the object surface defect detection method, the defects such as concave-convex flaws and scratches on the surface of the target object are detected by combining the characteristic information such as normal vector distribution, reflectivity distribution and gradient distribution on the surface of the target object, and the defects are detected more comprehensively and positioned more accurately.
The method for detecting the surface defects of the object provided by the application is described below with reference to an embodiment.
The object surface defect detection method mainly comprises the following steps: and detecting the surface defects of the target object by formally scanning after the LED light source angle calibration and the light source calibration of the three-dimensional scanner.
The LED light source direction calibration aims at obtaining a light source direction matrix when the LED light source is subjected to angle division and rapid sequential exposure, and comprises the following steps:
s1: the metal balls of reflectivity ρ are scanned using a three-dimensional scanner. The LED light source on the three-dimensional scanner is subjected to rapid and sequential exposure in an angle mode, and meanwhile the three-dimensional scanner completes image acquisition of the surface of the metal ball, so that at least three pieces of image data are obtained.
Wherein the three pieces of image data are the second image. In the embodiments of the present application, a space curved object used as a reference object is described as an example of a metal ball. In the actual implementation process, the space curved object serving as the reference object is not limited to the metal ball, but can be any other suitable space curved object, and can be selected by a person skilled in the art according to actual requirements.
The scanner collects images on the surface of the metal ball through the camera, and the number of the collected images can be 3, 4 or 5 and any number greater than or equal to 3.
S2: acquiring metal ball surface point cloud data by using a metal ball surface image acquired by a three-dimensional scanner, and further acquiring a metal ball surface normal vector matrix N q ；
Specifically, acquiring a full-bright image and a coded image of the surface of the metal ball, and obtaining point cloud data information of the surface of the metal ball through multi-view imaging operation; obtaining a metal ball surface normal vector matrix N based on metal ball surface point cloud data information q ；
S3: gray matrix M based on metal ball surface image q And normal vector matrix N q The LED light source on the three-dimensional scanner is obtained at different angles in sequenceA light source direction matrix L during rapid exposure, and an illumination intensity matrix E.
The specific implementation steps of S3 may be as follows:
let the normal vector of the surface at any point be (a, B, C) normalized By the spatial surface equation ax+by+cz+d=0
For the lambertian reflection, there is an established I.alpha.cos θ
I is that reflected light is acquired by a camera, k is a proportionality coefficient, theta is the included angle between a unit normal vector n of the light and an incident vector l of a unit light source, e is the light intensity of the point, which can be regarded as the unit light intensity, ρ is the reflectivity of the point, and l= (l) is set x ，l y ，l z ) Then
Wherein I, ρ,
It is known that the unit light source incidence vector l= (l) at this point can be calculated from three equations x ，l y ，l z ). Metal ball surface image gray matrix M for the same reason q And normal vector matrix N q The reflectivity ρ yields a light source direction matrix L.
Thus, the LED light source angle calibration of the three-dimensional scanner is completed by executing S1-S3. The specific process of detecting the surface defects of the target object by formal scanning after the calibration of the light source is as follows:
s4: when the three-dimensional scanner is used for scanning the surface of a target object with unknown reflectivity, the sequence of the LED light sources for sequential and rapid exposure at different angles on the three-dimensional scanner is the same as the sequence of the light source direction calibration, meanwhile, the LED light sources are attached to the three-dimensional scanner, the camera and the light sources are kept relatively static, and meanwhile, the light source direction matrix L is kept unchanged in the light source direction calibration process and the formal scanning process because the camera acquires images very rapidly and the target object scanned by the three-dimensional scanner moves relatively slowly.
S5: the solution of the light source intensity matrix E is realized through the alternate minimization of the matrix, and the normal vector matrix N and the reflectivity distribution matrix P of the surface of the target to be scanned are obtained through the three-dimensional reconstruction of the photometric stereo method.
S5, the specific implementation steps are as follows;
after a light source direction matrix L and a light source intensity matrix E are determined, a three-dimensional scanner is used for scanning the surface of a target object with unknown reflectivity, an LED light source on the three-dimensional scanner is set to be subjected to angle-division rapid and sequential exposure for f times, meanwhile, the three-dimensional scanner completes image acquisition of the surface of the target object, the image resolution is set to be w x h, and then the behavior f of an image gray matrix M of the surface of the target object is listed as w x h;
the formula m=elnp is shown, wherein M is a target object surface image gray matrix with the behavior f and the columns w×h, and E is a light source intensity matrix diag (E 1 、e 2 ...e f ) L is an LED light source direction matrix with a row of 3, N is a target object surface unit normal vector matrix with a row of 3, P is a reflectivity distribution matrix with a row of w, wherein M, E, L is a known quantity, and N, P is an unknown quantity.
The formula m=elnp can be expressed as m=l 'N', where L '=el, N' =np, where N 'is solved using the least squares method (not uniquely) with N' = (L '' T L′) -1 L′ T
After solving for N ', there is N' =np
Therefore there are
n′ i Is a unit normal vector, so there is (n) ix +n iy +n iz )＝1，
Therefore:
and solving the unit normal vector matrix N on the surface of the target object and the reflectivity matrix P on the surface of the target object.
S6: and acquiring a gradient distribution matrix G of the surface of the target to be scanned through the normal vector matrix.
The specific implementation steps of S6 are as follows:
the distribution of the gradient of the image in the X, Y direction can be obtained from the normal vector matrix N obtained as described above:
there is a method of producing a liquid crystal display device,
the gradient distribution of the image can be obtained from the distribution of the gradient of the obtained image in the X, Y direction:
has the following components
So far, the gradient distribution matrix G of the surface of the target object is obtained.
S7: and detecting concave-convex flaws and scratches on the surface of the target object through gradient distribution and reflectivity distribution.
When detecting the concave-convex flaws and the division on the surface of the target object, the gradient distribution matrix, the reflectivity distribution matrix and the gray matrix acquired by the camera can be fused to form multi-channel target object surface information. And then carrying out defect detection on the surface of the target object through the obtained fused multi-channel target object surface information, wherein the specific method used for detection can be a traditional mode or a deep learning-based mode.
The specific mode for detecting the concave-convex flaws and scratches on the surface of the target object through the fused multichannel target object surface information of feasibility is as follows:
threshold segmentation is carried out on the reflectivity distribution image converted by the reflectivity distribution matrix P;
carrying out connected domain marking on the concave-convex flaw and scratch area by using a connected domain marking function;
searching the geometric center point of the concave-convex flaw and scratch area and the farthest distance from the area to the center;
and marking the areas where the concave-convex flaws and scratches are located in the reflectivity distribution image by using the geometric center points and the farthest distances.
Threshold segmentation is carried out on the gradient distribution image converted from the gradient distribution image G;
carrying out connected domain marking on the concave-convex flaw and scratch area by using a connected domain marking function;
searching the geometric center point of the concave-convex flaw and scratch area and the farthest distance from the area to the center;
marking the areas where the concave-convex flaws and scratches are located in the gradient distribution image by using the geometric center points and the farthest distances.
And displaying the union of the area where the concave-convex flaws and scratches are marked in the middle reflectivity distribution map image and the area where the concave-convex flaws and scratches are marked in the gradient distribution map image in the original image.
And finishing the detection of the concave-convex flaws and the scratches by the three-dimensional scanner.
Fig. 4 is a block diagram of an object surface defect detecting device according to an embodiment of the present application.
The object surface defect detection device provided by the embodiment of the application is applied to a three-dimensional scanner, and comprises the following functional modules:
the control module 401 is configured to scan a surface of a target object using a three-dimensional scanner, wherein a light source on the three-dimensional scanner is rapidly and sequentially exposed in an angle according to a calibrated sequence of calibrating a direction of the calibrated light source in a scanning process, and the three-dimensional scanner performs image acquisition on the surface of the target object to obtain a first image;
the acquisition module 402 is configured to acquire a normal vector matrix and a reflectivity distribution matrix on the surface of the target object based on a light source direction matrix obtained when the light source direction is calibrated by using a photometric stereo three-dimensional reconstruction technology;
a generating module 403, configured to generate a gradient distribution matrix of the target object surface based on the normal vector matrix;
and the detection module 404 is configured to detect the concave-convex flaws and scratches existing on the surface of the target object based on the gradient distribution matrix and the reflectivity distribution matrix.
Optionally, the apparatus further comprises:
the calibration module is used for calibrating the light source direction on the three-dimensional scanner through a space curved object with known reflectivity before the control module scans the surface of the target object by using the three-dimensional scanner, so as to obtain a light source direction matrix when the light sources are exposed in a rapid and sequential manner in different angles.
Optionally, the calibration module includes:
the first sub-module is used for scanning a space curved object with known reflectivity by using the three-dimensional scanner, the light source of the three-dimensional scanner is subjected to angle-division rapid and sequential exposure, and the three-dimensional scanner performs image acquisition on the space curved object to obtain a second image;
a second sub-module, configured to obtain the spatial curved surface object surface point cloud data based on the second image;
the third sub-module is used for determining a normal vector matrix of the surface of the space curved object according to the point cloud data;
and the fourth sub-module is used for obtaining a light source direction matrix when the light sources on the three-dimensional scanner are subjected to angle-division rapid and sequential exposure based on the gray matrix and the normal vector matrix of the second image.
Optionally, the second submodule is specifically configured to:
determining a full bright image and an encoded image in the second image; and processing the full-bright image and the coded image through multi-eye imaging operation to obtain the surface point cloud data of the space curved object.
Optionally, the fourth submodule is specifically configured to:
solving a curved surface normal vector at each pixel based on a space curved surface equation of a preset reference object aiming at any pixel under the illumination of a light source;
calculating to obtain a unit light source incidence vector at the pixel based on the normal vector of the curved surface, the reflected illumination, the reflectivity at the pixel, the normal vector at the illumination and a preset equation set;
and obtaining a light source direction matrix when the light sources are subjected to angle-division rapid and sequential exposure on the three-dimensional scanner through the incidence vectors of the unit light sources at each pixel under the irradiation of the light sources.
Optionally, the generating module includes:
a fifth sub-module for determining a distribution of gradients of the image in a X, Y direction based on the normal vector matrix;
a sixth sub-module, configured to obtain gradient distribution of the image according to distribution of gradients of the image in a X, Y direction;
and a seventh sub-module, configured to determine a gradient distribution matrix of the target object surface based on the image gradient distribution.
Optionally, the detection module includes:
an eighth sub-module, configured to fuse the gradient distribution matrix, the reflectivity distribution matrix, and the image gray matrix acquired by the three-dimensional scanner camera to form multi-channel surface information of the target to be detected;
and a ninth sub-module, configured to detect a defect existing on the surface of the target object based on the information of the target surface to be detected of the multiple channels.
According to the object surface defect detection device, the defects such as concave-convex flaws and scratches on the surface of the target object are detected by combining the characteristic information such as normal vector distribution, reflectivity distribution and gradient distribution on the surface of the target object, and the detection of the defects is more comprehensive and positioning is more accurate.
The object surface defect detection device shown in fig. 4 in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a three-dimensional scanner. The object surface defect detection device shown in fig. 4 in the embodiment of the present application may be a device having an operating system.
The object surface defect detection device shown in fig. 4 provided in this embodiment of the present application can implement each process implemented by the method embodiment of fig. 1, and in order to avoid repetition, a description is omitted here.
Optionally, the embodiment of the present application further provides a three-dimensional scanner, including a processor, a memory, and a program or an instruction stored in the memory and capable of running on the processor, where the program or the instruction when executed by the processor implements each process of the embodiment of the method for detecting a surface defect of an object, and the process can achieve the same technical effect, and for avoiding repetition, a description is omitted herein. Further, the three-dimensional scanner further includes a light source such as: LED light sources, cameras, and the like.
It should be noted that the electronic device in the embodiment of the present application includes the server described above.
The embodiment of the application further provides a readable storage medium, on which a program or an instruction is stored, where the program or the instruction realizes each process of the above embodiment of the object surface defect detection method when executed by a processor, and the same technical effects can be achieved, so that repetition is avoided, and no detailed description is given here.
Wherein the processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium such as a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.
The embodiment of the application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running a program or instructions, implementing each process of the object surface defect method embodiment, and achieving the same technical effect, so as to avoid repetition, and no redundant description is provided herein.
It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that various modifications and adaptations can be made without departing from the principles of the present invention, and such modifications and adaptations are intended to be comprehended within the scope of the present invention.
Claims (10)
1. A method for detecting surface defects of an object, applied to a three-dimensional scanner, the method comprising:
scanning the surface of a target object by using a three-dimensional scanner, wherein a light source on the three-dimensional scanner is subjected to angle-division rapid sequential exposure according to a calibrated light source direction calibration sequence in the scanning process, and simultaneously, the three-dimensional scanner performs image acquisition on the surface of the target object to obtain a first image;
acquiring a normal vector matrix and a reflectivity distribution matrix of the surface of a target object based on a light source direction matrix obtained when the light source direction is calibrated by a photometric stereo three-dimensional reconstruction technology;
generating a gradient distribution matrix of the surface of the target object based on the normal vector matrix;
and detecting concave-convex flaws and scratches existing on the surface of the target object based on the gradient distribution matrix and the reflectivity distribution matrix.
2. The method of claim 1, wherein prior to the step of scanning the surface of the target object using the three-dimensional scanner, the method further comprises:
and calibrating the direction of the light source on the three-dimensional scanner through a space curved object with known reflectivity to obtain a light source direction matrix when the light source is subjected to angle-division rapid and sequential exposure.
3. The method according to claim 2, characterized in that: the step of calibrating the light source direction of the three-dimensional scanner through the space curved object with known reflectivity to obtain a light source direction matrix when the light source is exposed in turn at different angles comprises the following steps:
scanning a space curved object with known reflectivity by using the three-dimensional scanner, enabling a light source of the three-dimensional scanner to be exposed at an angle in sequence, and acquiring an image of the space curved object by using the three-dimensional scanner to obtain a second image;
obtaining the surface point cloud data of the space curved surface object based on the second image;
determining a normal vector matrix of the surface of the space curved object according to the point cloud data;
and obtaining a light source direction matrix when the light sources on the three-dimensional scanner are subjected to angle-division rapid and sequential exposure based on the gray matrix and the normal vector matrix of the second image.
4. A method according to claim 3, wherein the step of obtaining the spatially curved object surface point cloud data based on the second image comprises:
determining a full bright image and an encoded image in the second image;
and processing the full-bright image and the coded image through multi-eye imaging operation to obtain the surface point cloud data of the space curved object.
5. A method according to claim 3, wherein the step of obtaining a light source direction matrix for the three-dimensional scanner when the light sources are exposed in rapid angular succession based on the gray matrix and the normal vector matrix of the second image comprises:
solving a curved surface normal vector at each pixel based on a space curved surface equation of a preset reference object aiming at any pixel under the illumination of a light source;
calculating to obtain a unit light source incidence vector at the pixel based on the normal vector of the curved surface, the reflected illumination, the reflectivity at the pixel, the normal vector at the illumination and a preset equation set;
and obtaining a light source direction matrix when the light sources are subjected to angle-division rapid and sequential exposure on the three-dimensional scanner through the incidence vectors of the unit light sources at each pixel under the irradiation of the light sources.
6. The method of claim 1, wherein the step of generating a gradient distribution matrix of the target object surface based on the normal vector matrix comprises:
determining a distribution of gradients of the image in a X, Y direction based on the normal vector matrix;
obtaining gradient distribution of the image according to the distribution of the gradient of the image in the X, Y direction;
and determining a gradient distribution matrix of the surface of the target object based on the image gradient distribution.
7. The method of claim 1, wherein the step of detecting the presence of the asperity flaws and scratches on the surface of the target object based on the gradient distribution matrix and the reflectance distribution matrix comprises:
fusing the gradient distribution matrix, the reflectivity distribution matrix and the gray matrix acquired by the three-position scanner camera to form multichannel target surface information to be detected;
and detecting defects existing on the surface of the target object based on the information of the surface of the target to be detected of the multiple channels.
8. An object surface defect detection apparatus, for use in a three-dimensional scanner, the apparatus comprising:
the control module is used for scanning the surface of the target object by using the three-dimensional scanner, the light source on the three-dimensional scanner is rapidly and sequentially exposed in an angle-dividing manner according to the calibrated light source direction calibration sequence in the scanning process, and the three-dimensional scanner acquires images of the surface of the target object to obtain a first image;
the acquisition module is used for acquiring a normal vector matrix and a reflectivity distribution matrix on the surface of the target object based on a light source direction matrix obtained when the light source direction is calibrated through a photometric stereo three-dimensional reconstruction technology;
the generation module is used for generating a gradient distribution matrix of the surface of the target object based on the normal vector matrix;
and the detection module is used for detecting concave-convex flaws and scratches on the surface of the target object based on the gradient distribution matrix and the reflectivity distribution matrix.
9. The apparatus of claim 8, wherein the apparatus further comprises:
the calibration module is used for calibrating the light source direction on the three-dimensional scanner through a space curved object with known reflectivity before the control module scans the surface of the target object by using the three-dimensional scanner, so as to obtain a light source direction matrix when the light sources are exposed in a rapid and sequential manner in different angles.
10. A three-dimensional scanner comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, which when executed by the processor, implement the steps of the object surface defect detection method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310104898.6A CN116106318A (en) | 2023-02-13 | 2023-02-13 | Object surface defect detection method and device and three-dimensional scanner |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310104898.6A CN116106318A (en) | 2023-02-13 | 2023-02-13 | Object surface defect detection method and device and three-dimensional scanner |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116106318A true CN116106318A (en) | 2023-05-12 |
Family
ID=86257807
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310104898.6A Pending CN116106318A (en) | 2023-02-13 | 2023-02-13 | Object surface defect detection method and device and three-dimensional scanner |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116106318A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116757713A (en) * | 2023-08-18 | 2023-09-15 | 画版文化科技集团有限公司 | Work estimation method, device, equipment and storage medium based on image recognition |
-
2023
- 2023-02-13 CN CN202310104898.6A patent/CN116106318A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116757713A (en) * | 2023-08-18 | 2023-09-15 | 画版文化科技集团有限公司 | Work estimation method, device, equipment and storage medium based on image recognition |
| CN116757713B (en) * | 2023-08-18 | 2024-01-12 | 画版文化科技集团有限公司 | Work estimation method, device, equipment and storage medium based on image recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6319329B2 (en) | Surface attribute estimation using plenoptic camera | |
| KR102287472B1 (en) | Acquisition Method for 3D Objects Using Unstructured Flash Photography and Apparatus Therefor | |
| EP1462992B1 (en) | System and method for shape reconstruction from optical images | |
| EP3382645B1 (en) | Method for generation of a 3d model based on structure from motion and photometric stereo of 2d sparse images | |
| CN109076148A (en) | Everyday scenes reconstruction engine | |
| JP5051493B2 (en) | 3D measurement marker and 3D measurement method using the same | |
| JP5133626B2 (en) | Surface reflection characteristic measuring device | |
| JP6519265B2 (en) | Image processing method | |
| JP2003130621A (en) | Method and system for measuring three-dimensional shape | |
| Marcin et al. | Hierarchical, three‐dimensional measurement system for crime scene scanning | |
| Ley et al. | Syb3r: A realistic synthetic benchmark for 3d reconstruction from images | |
| Logothetis et al. | A cnn based approach for the point-light photometric stereo problem | |
| Mecca et al. | Luces: A dataset for near-field point light source photometric stereo | |
| CN116106318A (en) | Object surface defect detection method and device and three-dimensional scanner | |
| Karami et al. | 3D digitization of transparent and glass surfaces: State of the art and analysis of some methods | |
| Park et al. | Surface light field fusion | |
| JP6237032B2 (en) | Color and three-dimensional shape measuring method and apparatus | |
| Stets et al. | Scene reassembly after multimodal digitization and pipeline evaluation using photorealistic rendering | |
| Knyaz | Multi-media projector–single camera photogrammetric system for fast 3D reconstruction | |
| Ji et al. | Calibration method of light-field camera for photogrammetry application | |
| Xu et al. | A unified spatial-angular structured light for single-view acquisition of shape and reflectance | |
| Taylor | Implementing high resolution structured light by exploiting projector blur | |
| JP2010243478A (en) | Method and apparatus for estimating 3d pose of 3d object in environment | |
| CN111033566A (en) | Method and system for the non-destructive inspection of an aircraft part | |
| Koch et al. | Hardware design and accurate simulation for benchmarking of 3D reconstruction algorithms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |