CN113658155A - Object surface flaw detection and analysis method and device based on photometric stereo - Google Patents
Object surface flaw detection and analysis method and device based on photometric stereo Download PDFInfo
- Publication number
- CN113658155A CN113658155A CN202110973139.4A CN202110973139A CN113658155A CN 113658155 A CN113658155 A CN 113658155A CN 202110973139 A CN202110973139 A CN 202110973139A CN 113658155 A CN113658155 A CN 113658155A
- Authority
- CN
- China
- Prior art keywords
- map
- gaussian
- surface curvature
- reflectivity
- derivative
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 40
- 238000004458 analytical method Methods 0.000 title claims description 36
- 230000007547 defect Effects 0.000 claims abstract description 112
- 238000002310 reflectometry Methods 0.000 claims abstract description 63
- 238000000034 method Methods 0.000 claims abstract description 45
- 238000013178 mathematical model Methods 0.000 claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 28
- 238000004364 calculation method Methods 0.000 claims abstract description 20
- 238000001914 filtration Methods 0.000 claims description 32
- 238000012800 visualization Methods 0.000 claims description 24
- 238000010586 diagram Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 230000003287 optical effect Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 10
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 description 21
- 229910052744 lithium Inorganic materials 0.000 description 21
- 239000000463 material Substances 0.000 description 7
- 230000002950 deficient Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000005286 illumination Methods 0.000 description 4
- 238000010191 image analysis Methods 0.000 description 4
- 239000010985 leather Substances 0.000 description 4
- ATJFFYVFTNAWJD-UHFFFAOYSA-N Tin Chemical compound [Sn] ATJFFYVFTNAWJD-UHFFFAOYSA-N 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 239000002453 shampoo Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- BQCADISMDOOEFD-UHFFFAOYSA-N Silver Chemical compound [Ag] BQCADISMDOOEFD-UHFFFAOYSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 239000002775 capsule Substances 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 235000019219 chocolate Nutrition 0.000 description 1
- 235000019504 cigarettes Nutrition 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 229910052709 silver Inorganic materials 0.000 description 1
- 239000004332 silver Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/514—Depth or shape recovery from specularities
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The application provides a method and a device for detecting and analyzing object surface flaws based on photometric stereo, wherein the method for detecting and analyzing the object surface flaws based on photometric stereo comprises the following steps: acquiring pixel values of the surface image of the object to be measured of at least three light sources with different angles, and constructing a luminosity three-dimensional mathematical model; performing reflectivity calculation on the luminosity three-dimensional mathematical model to construct a reflectivity graph; calculating to obtain a gradient field of the surface of the object to be measured; according to the gradient field of the surface of the object to be measured, Gaussian surface curvature calculation and average surface curvature calculation are carried out, and a Gaussian surface curvature defect map and an average surface curvature defect map are obtained; and analyzing the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map to obtain a detection result. The method avoids the problem that the traditional image processing technology is difficult to analyze and find out flaws on the surfaces of various different objects, and has the characteristics of high detection accuracy and strong universality.
Description
Technical Field
The application relates to the technical field of image detection, in particular to a method and a device for detecting and analyzing object surface flaws based on photometric stereo.
Background
The appearance inspection can find foreign matters, stains, flaws, defects and the like, and prevent the defective products from flowing out, but the visual inspection has a precision limit. The total number detection does not consume labor and cost, and precision deviation and manual operation errors are caused by personal difference. Moreover, fine flaws, stains, etc. are difficult to detect, and therefore, in order to maintain the quality, it is necessary to perform an amplification test by means of a microscope, etc. When the number of dots is small, the microscope can be offline for detection, but when thousands of dots are detected, enormous labor is required, and the production efficiency is greatly reduced. The vision system technology is an indispensable important link to consider quality and production efficiency.
With the progress of visual systems and visual system technologies, the detection of fine foreign objects, flaws, and defects has become possible for appearance detection that had to rely on human eye judgment in the past. The defect detection and analysis method based on the machine vision technology detects possible defects of products by utilizing image processing and analysis. When detecting the defects of the product, the defect image is characterized in that the gray value of the defect is different from the gray value of the standard image. Firstly, extracting and selecting the characteristics of the defective image, then comparing the gray value of the defective image with the gray value of the standard image, and judging whether the difference value, namely the difference degree of the gray values of the two images exceeds the preset threshold range, so that whether the product to be detected has defects or not can be judged by the method. This is one of the most basic methods for defect detection.
However, the conventional image processing techniques are susceptible to interference from surface materials, surface textures, patterns, surface colors, etc. of the object on which the defect or dent is detected. For example, the curved plastic surface of a shampoo bottle, the tinfoil surface for medical tablet packaging, the leather surface of clothes and the surface of a lithium battery are completely different in material texture and scratch marks, the types of formed flaws are diversified, some flaws are depressions, some flaws are cavities, and some flaws are dark marks.
Disclosure of Invention
In the prior art, the defects can not be found by analyzing the surfaces of different kinds of objects through one image processing technology. The surface material, texture and illumination difference of the object to be detected is large, the traditional algorithm design difficulty is large, the detection accuracy is low, the universality is poor, and defects such as scratches, dents and the like on the surface of a lithium battery are difficult to detect; the problem of the deep learning technique is high in hardware requirement and needs a large amount of and abundant samples. The method is a comprehensive detection method, integrates multiple calibration modes, can be suitable for the surface flaws of different objects, has the characteristics of high detection accuracy and strong universality, and discloses an object surface flaw detection analysis method based on photometric stereo through the following embodiments.
In a first aspect of the present invention, a method for detecting and analyzing object surface flaws based on photometric stereo is disclosed, comprising the following steps:
s10, acquiring all pixel values of the surface image of the object to be detected of at least three light sources with different angles;
s20, constructing a luminosity three-dimensional mathematical model according to the pixel values;
s30, calculating the photometric stereo mathematical model to obtain a reflectivity and a normal vector;
s40, constructing a reflectivity map by visualizing the reflectivity;
s50, calculating to obtain a gradient field of the surface of the object to be measured according to the normal vector;
s60, according to the gradient field of the surface of the object to be measured, Gaussian surface curvature calculation is carried out, and a Gaussian surface curvature defect map is obtained through visualization processing;
s70, according to the gradient field of the surface of the object to be measured, calculating the average surface curvature and obtaining an average surface curvature defect map through visualization processing;
and S80, analyzing the reflectivity diagram, the Gaussian surface curvature defect diagram and the average surface curvature defect diagram to obtain a detection result.
Optionally, in S20, the optical stereo mathematical model is,
I＝ρn·s
wherein, I is the image pixel value, n is the unit normal vector, s is the unit light source incidence vector, and rho is the object surface reflectivity.
Optionally, in S30, the reflectivity and the normal vector are obtained by calculating the photometric stereo mathematical model,
solving the photometric stereo mathematical model by adopting a least square estimation method,
order to
Then there is
Solved to obtain
Wherein T denotes transpose, STRepresenting the transpose of the matrix S.
Optionally, in S60, based on the gradient field of the surface of the object to be measured, gaussian surface curvature calculation is performed to calculate,
normalizing the unit normal vector n in the z direction to obtain a first derivative p in the x direction and a first derivative q in the y direction; carrying out Gaussian derivative filtering in the x direction on the gradient map in the x direction to obtain a second derivative p in the x directionx(ii) a Carrying out Gaussian derivative filtering in the y direction on the gradient map in the x direction to obtain a second derivative p in the x directiony；
Carrying out Gaussian derivative filtering in the y direction on the gradient map in the y direction to obtain a second derivative q in the y directiony(ii) a Carrying out Gaussian derivative filtering in the x direction on the gradient map in the y direction to obtain a second derivative q in the y directionx(ii) a Gaussian curvature
Optionally, in S70, according to the gradient field of the surface of the object to be measured, the average surface curvature is calculated as,
normalizing the unit normal vector n in the z direction to obtain a first derivative p in the x direction and a first derivative q in the y direction; carrying out Gaussian derivative filtering in the x direction on the gradient map in the x direction to obtain a second derivative p in the x directionx(ii) a Carrying out Gaussian derivative filtering in the y direction on the gradient map in the x direction to obtain a second derivative p in the x directiony；
Carrying out Gaussian derivative filtering in the y direction on the gradient map in the y direction to obtain a second derivative q in the y directiony(ii) a Carrying out Gaussian derivative filtering in the x direction on the gradient map in the y direction to obtain a second derivative q in the y directionx(ii) a The average surface curvature can be calculated
A＝(1+p2)qy
B＝pq(py+qx)
C＝(1+q2)px
Where H is the mean curvature.
Optionally, the analysis of the reflectivity map, the gaussian surface curvature defect map and the average surface curvature defect map in S80 is,
and carrying out binarization analysis on the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map.
Optionally, the analysis of the reflectivity map, the gaussian surface curvature defect map and the average surface curvature defect map in S80 is,
and carrying out connected domain analysis on the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map.
Optionally, the analysis of the reflectivity map, the gaussian surface curvature defect map and the average surface curvature defect map in S80 is,
and carrying out linear detection analysis on the reflectivity graph, the Gaussian surface curvature defect graph and the average surface curvature defect graph.
Optionally, the visualization process is to convert the gray scale image into a gray scale image for display by normalizing the gray scale value to be between 0 and 255.
In a second aspect of the present invention, a photometric stereo-based object surface flaw detection and analysis apparatus is disclosed, comprising:
the pixel module is used for acquiring all pixel values of the surface image of the object to be detected of at least three light sources with different angles;
the mathematical model module is used for constructing a luminosity three-dimensional mathematical model according to the pixel values;
the calculation module is used for calculating the photometric stereo mathematical model to obtain the reflectivity and normal vector;
the reflectivity module is used for constructing a reflectivity graph by performing visualization processing on the reflectivity;
the gradient field module is used for calculating to obtain a gradient field of the surface of the object to be measured according to the luminosity three-dimensional mathematical model;
the Gaussian surface curvature module is used for calculating the Gaussian surface curvature according to the gradient field of the surface of the object to be detected and acquiring a Gaussian surface curvature defect map through visualization processing;
the average surface curvature module is used for calculating the average surface curvature and acquiring an average surface curvature defect map through visualization processing;
and the detection analysis module is used for analyzing the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map to obtain a detection result.
By the scheme, the photometric stereo model can be rapidly solved by adopting least square estimation, the set of three output images can be better adapted to different types of flaw detection, and the method has good applicability. The reflectivity image can effectively reflect flaws such as scratches on the surface of an object, the average defect degree image can effectively reflect flaws such as bulges and pits on the surface of the object, the Gaussian depression degree image can effectively reflect flaws of holes on the surface of the object, and different flaws of various object images can be judged at one time. The method avoids the problem that the traditional image processing technology is difficult to analyze and find out flaws on the surfaces of different kinds of objects, and has the characteristics of high detection accuracy and strong universality.
Drawings
In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without any creative effort.
FIG. 1 is a flowchart of a method for detecting and analyzing defects on a surface of an object based on photometric stereo according to an embodiment of the present invention;
fig. 2 is a diagram of a defective lithium battery with a 0 degree light source according to an embodiment of the present invention;
fig. 3 is a diagram of a defective lithium battery with a 90-degree light source according to an embodiment of the present invention;
fig. 4 is a diagram of a defective lithium battery with a 270 degree light source according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating a Gaussian derivative filter kernel separation process according to an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating a comparison between an x-direction Gaussian derivative filter kernel and a y-direction Gaussian derivative filter kernel according to an embodiment of the present invention;
fig. 7 is a block diagram of an image defect detection apparatus based on frequency domain filtering according to an embodiment of the present invention.
Detailed Description
In the prior art, the defects can not be found by analyzing the surfaces of different kinds of objects through one image processing technology. The surface material, texture and illumination difference of an object to be detected is large, the types of defects are diversified, the traditional algorithm is high in design difficulty, low in detection accuracy and poor in universality, and defects such as scratches and dents on the surface of a lithium battery cannot be detected easily; the problem of the deep learning technique is high in hardware requirement and needs a large amount of and abundant samples. The invention discloses a method and a device for detecting and analyzing object surface flaws based on photometric stereo.
Referring to fig. 1, a flowchart of an object surface flaw detection and analysis method based on photometric stereo is provided in the present application, and is applied to a corresponding apparatus. As shown in fig. 1, the method comprises the following steps:
and S10, acquiring all pixel values of the surface image of the object to be measured of at least three light sources with different angles.
The purpose of this step is to obtain information for subsequent methods. The subsequent calculation methods of the present application are all based on the pixel values of the image. The pixel values are units or elements which are inseparable in the whole image, that is, the improved reflectivity map, the gaussian surface curvature defect map and the average surface curvature defect map of the application are spliced into a complete image after calculation and conversion of each pixel value. Secondly, the object surface image to be measured of at least three different angle light sources needs to be acquired in the method, so that a plurality of parameters are needed in the subsequent linear calculation process. The camera needs to be fixed, and the light source shoots at different angles. The linear equation system needs at least three sets of parameters to solve, and the pixel values of the same point on each picture are combined into a vector. The at least three different angles may be of any value, preferably evenly distributed over 0 ° to 360 °, so that the averaging is high and the error is small.
And S20, constructing a photometric stereo mathematical model according to the pixel values.
From the pixel values, the object surface conforms to the Lambertian reflection (ideal scattering) property according to the principle of photometric stereo, that is, the surface with the same brightness is observed from all the view field directions under a fixed illumination distribution, and the reflection model is as follows:
I＝ρn·s
wherein n is a unit normal vector, s represents a unit light source incidence vector, and ρ is the object surface reflectivity. It should be noted that I ═ ρ n · S is a linear formula, and corresponds to all pixel values on the surface image of the object to be measured, which requires at least three different angle light sources in step S10.
And S30, calculating the photometric stereo mathematical model to obtain the reflectivity and normal vector.
The step optimizes and solves the photometric stereo mathematical model formula I ═ ρ n · s by a least square estimation method,
order to
Then there is
Solved to obtain
Wherein T denotes transpose, STRepresenting the transpose of the matrix S. It should be noted that the calculation in this step is to calculate each pixel value on the surface image of the object to be measured, and the calculation of each pixel value is completed to finally form a complete image.
S40, a reflectance map is constructed by visualizing the reflectance.
The reflectivity of each pixel is obtained through the calculation of the pixel value in the step, complete reflectivity data of the surface image of the object to be detected can be formed, and then all the reflectivity data are subjected to visualization processing to obtain a reflectivity graph of the surface image of the object to be detected.
It should be noted that the visualization is a theory, method and technology that converts data into graphics or images to be displayed on a screen by using computer graphics and image processing technology, and performs interactive processing. All visualization processes in the application are to convert the images into gray images for display by normalizing the images to gray levels between 0 and 255.
The reflectance map is obtained to better reflect the degree of change in the surface material of the object. According to the reflectivity diagram, the method is suitable for detecting the surface of an object with obvious material change, namely the surface reflection difference between a flaw and the surface of the object is large, such as leather. The normal leather surface is relatively uniform, and the light reflection degree is uniform and consistent. The flaws on the leather are small cracks or irregular depressions, uniform light reflection cannot be formed, the color is darker, and the flaws can be more obviously identified in a reflectivity diagram.
And S50, calculating to obtain the gradient field of the surface of the object to be measured according to the normal vector.
And the normal vector of any point on the surface of the object to be measured is orthogonal to the tangent plane of the point. The tangent plane can be formed by stretching the gradient vector of the point in the x direction and the gradient vector of the point in the y direction, so that the normal vector is the unit vector which is orthogonal to the plane, and the gradient field of the surface of the three-dimensional object to be measured can be calculated. The gradient field is calculated in this step for the purpose of calculating the gaussian surface curvature and the mean surface curvature in subsequent steps. The gradient provides two pieces of information, magnitude and direction, representing the state of change in direction.
And S60, according to the gradient field of the surface of the object to be measured, Gaussian surface curvature calculation is carried out, and a Gaussian surface curvature defect map is obtained through visualization processing.
Specifically, the gaussian surface curvature is calculated by normalizing a unit normal vector n in the z direction to obtain a first derivative p in the x direction and a first derivative q in the y direction; carrying out Gaussian derivative filtering in the x direction on the gradient map in the x direction to obtain a second derivative p in the x directionx(ii) a Carrying out Gaussian derivative filtering in the y direction on the gradient map in the x direction to obtain a second derivative p in the x directiony(ii) a Carrying out Gaussian derivative filtering in the y direction on the gradient map in the y direction to obtain a second derivative q in the y directiony(ii) a Carrying out Gaussian derivative filtering in the x direction on the gradient map in the y direction to obtain a second derivative q in the y directionx(ii) a Gaussian curvature
Gaussian curvature is an intrinsic measure of curvature, i.e., the value of gaussian curvature depends only on how the distance on the surface is measured, rather than how the surface is embedded into space after the gaussian surface curvature is calculated. And performing visualization processing, namely converting the normalized gray scale to be between 0 and 255 gray scale images for displaying to obtain a Gaussian surface curvature defect map. The Gaussian surface curvature defect map can be applied to and used for identifying holes on the surface, such as tinfoil. Silver tinfoil is thin and can be used for packaging capsules, cigarettes, chocolate and other articles, and the defects are usually small holes. Hole type defects can be identified more clearly by using the formed Gaussian surface curvature defect map.
And S70, calculating the average surface curvature according to the gradient field of the surface of the object to be measured, and acquiring an average surface curvature defect map through visualization processing.
Specifically, the average surface curvature calculation was performed as,
normalizing the unit normal vector n in the z direction to obtainThe first derivative p to the x-direction and the first derivative q in the y-direction; carrying out Gaussian derivative filtering in the x direction on the gradient map in the x direction to obtain a second derivative p in the x directionx(ii) a Carrying out Gaussian derivative filtering in the y direction on the gradient map in the x direction to obtain a second derivative p in the x directiony；
Carrying out Gaussian derivative filtering in the y direction on the gradient map in the y direction to obtain a second derivative q in the y directiony(ii) a Carrying out Gaussian derivative filtering in the x direction on the gradient map in the y direction to obtain a second derivative q in the y directionx(ii) a The average surface curvature can be calculated,
A＝(1+p2)qy
B＝pq(py+qx)
C＝(1+q2)px
where H is the mean curvature. The curvature indicates the amount of curve at a certain point. The mean curvature of a curved surface, an extrinsic measure of curvature, describes locally the curvature of a curved surface fitting into the surrounding space. After the average surface curvature is calculated, the average surface curvature defect map is obtained by converting the average surface curvature into a gray image for display through visualization processing, namely normalization to the gray level of 0-255. Through the detection of the bending degree of the object, the defects with the concave-convex surface of the object can be distinguished. The plastic surface of a shampoo has defects which are pits and not flat. The mean surface curvature defect map has strong discriminating properties for the curved surface, and defects thereof can be more clearly identified.
And S80, analyzing the reflectivity diagram, the Gaussian surface curvature defect diagram and the average surface curvature defect diagram to obtain a detection result.
A reflectivity map, a gaussian surface curvature defect map, and an average surface curvature defect map, each of which has the advantage of identifying defects. The reflectivity graph is relatively suitable for the surface of an object with obvious material change defects; a Gaussian surface curvature defect map, which is more suitable for inspecting the surface of an object with hole defects; the average surface curvature defect map is suitable for identifying the surface of an article with concave-convex defects. At least one of the three output graphs can exert the identification advantages thereof, and the surface defects of the object to be detected can be more obviously identified. In practical application, one or more specific analyses in the three output maps can be selected according to the attributes of the flaws, namely, the flaws are weak in reflection, the flaws are cavities and the flaws are recesses. The three output graphs can cover a large range and have strong adaptability.
It should be noted that the reflectivity map, the gaussian surface curvature defect map and the average surface curvature defect map do not need human eyes to identify, but find flaws through a computer vision analysis program, and can be automatically identified through an image analysis algorithm. And performing binarization analysis, connected domain analysis, linear detection and other post-processing steps on the visualized reflectivity map, the visualized Gaussian surface curvature defect map and the average surface curvature defect map to judge whether flaws exist. Different picture analysis modes are selected by analyzing according to actual picture conditions, so that the defect part is highlighted, and the detection is more convenient.
In a specific example, a detailed application and solution of the method is provided. In the development process of science and technology, people have higher and higher quality requirements on lithium batteries for mobile phones, and generally speaking, before mobile phones enter the market, manufacturers need to correspondingly detect the appearance and the function of the lithium batteries for the mobile phones in order to meet better use effects.
The method can directly eliminate the interference of the background and the structure, reconstruct the gradient information of the object surface and highlight the scratch part, thereby simplifying the subsequent detection.
The camera needs to be fixed, and the light source shoots at different angles. Firstly, setting a horizontal line of the lithium battery 1 in the right direction as a reference point of 0 degree, and inputting an image of the lithium battery 1 acquired by a camera under the irradiation of 0 degree by a light source 2, referring to fig. 2; inputting images of the lithium battery 1 collected by a camera under the irradiation of 90 degrees by a light source 2, and referring to fig. 3; the input light source 2 is the 1 image captured by the camera under 270 degrees illumination, see fig. 4.
And acquiring all corresponding pixel values through three lithium battery 1 images, and constructing a luminosity three-dimensional mathematical model by using all the pixel values, wherein the value is I ═ rho · s.
Wherein, I is the image pixel value, n is the unit normal vector, s is the unit light source incidence vector, and rho is the object surface reflectivity.
Calculating the photometric stereo mathematical model to obtain the reflectivity and normal vector of each pixel, and solving the reflection model by adopting a least square estimation method to ensure that:
order to
Then there is
Solved to obtain
Wherein T denotes transpose, STRepresenting the transpose of the matrix S.
Then, the object surface reflectivity value corresponding to each point pixel of the image is obtained, and the reflectivity image can be obtained after normalization.
And calculating the gradient value of the pixels on the surface of the lithium battery 1. The normal vector of the pixel on the surface of the lithium battery 1 is orthogonal to the tangent plane of the point. The tangent plane can be formed by stretching the gradient vector of the pixel point in the x direction and the gradient vector of the pixel point in the y direction, so that the normal vector is the unit vector orthogonal to the plane, and the gradient field on the surface of the lithium battery 1 can be calculated.
And calculating the surface defect degree of the lithium battery 1. The pixels of each lithium battery 1 image are calculated. After the unit normal vector is normalized in the z direction, the values in the x direction and the y direction are the first derivative in the x direction and the first derivative in the y direction, respectively, and are denoted as p and q. The Gaussian derivative filtering is to use a filtering kernel as the filtering operation of the first Gaussian derivative, respectively perform the Gaussian derivative filtering in the x direction and the y direction on the gradient map in the x direction, and perform the Gaussian derivative filtering in the x direction and the y direction on the gradient map in the y direction, so as to obtain the second derivative p in the x directionxSecond derivative q in the y directionyFirst derivative in x-direction is derived p in y-directionyFirst derivative in the y-direction is derived q in the x-directionx。
The gaussian derivative filter kernels are separated for acceleration in consideration of separability of the gaussian and derivative filter kernels, in the manner shown with reference to fig. 5. Since the convolution operation is independent of the convolution order, the gaussian derivative filter kernel can be further reduced to a gaussian derivative kernel in the x-direction and a gaussian derivative kernel in the y-direction, as shown in fig. 6. That is, the x-direction gaussian derivative filtering may be performed by first convolving the input image with a gaussian derivative kernel along the x-direction, then convolving the convolved result with the gaussian kernel along the y-direction, and then convolving the convolved result with the gaussian derivative kernel along the x-direction. Similarly, the y-direction gaussian derivative filtering process is to perform convolution with the gaussian kernel along the x-direction and then perform convolution with the gaussian derivative kernel along the y-direction on the basis of the convolution result.
Then, by the formula
And (3) solving the Gaussian surface curvature of the lithium battery 1, and obtaining a Gaussian surface curvature defect map through visualization processing.
By setting A ═ 1+ p2)qy、B＝pq(py+qx)、C＝(1+q2)pxAnd
determining the average surface curvature of a lithium battery 1
And acquiring an average surface curvature defect map through visualization processing.
The method comprises the steps of obtaining a reflectivity map, a Gaussian surface curvature defect map and an average surface curvature defect map of the lithium battery 1, and mainly analyzing the average surface curvature defect map by combining the cognition that the surface of the lithium battery 1 is generally concave.
And performing connected domain analysis on the obtained average surface curvature defect map by using a Blob tool to obtain a part marked with a defect, wherein a white point part represents the defect in the image subjected to the connected domain analysis.
The reflectivity map, the gaussian surface curvature defect map, and the average surface curvature defect map can be automatically identified by Blob analysis image analysis algorithm without human eye identification. The purpose is to detect and analyze 2D shapes in images to obtain information such as the location, shape, orientation of objects and topological relationships between objects, i.e. containment relationships. From this information, the target can be identified. In some applications we need to use not only the shape features of 2D, but also the feature relationships between Blob analyses.
The Blob analysis employed in the present embodiment is not limited to this analysis tool, as the image analysis. The method has the essence that the image is differentiated by the method, so that the purpose of clearness is achieved, the characteristics of the image can be amplified obviously, and the image can be identified automatically through an image analysis algorithm.
Further, referring to fig. 7, the device for detecting and analyzing the surface flaws of the object based on the photometric stereo disclosed in the embodiment of the present invention further includes:
the pixel module is used for acquiring all pixel values of the surface image of the object to be detected of the light sources with different angles;
the mathematical model module is used for constructing a luminosity three-dimensional mathematical model according to the pixel values;
a calculation module for calculating the photometric stereo mathematical model to obtain the reflectivity and normal vector
The reflectivity module is used for constructing a reflectivity graph by performing visualization processing on the reflectivity;
the gradient field module is used for calculating to obtain a gradient field of the surface of the object to be measured according to the luminosity three-dimensional mathematical model;
the Gaussian surface curvature module is used for calculating the Gaussian surface curvature according to the gradient field of the surface of the object to be detected and acquiring a Gaussian surface curvature defect map through visualization processing;
the average surface curvature module is used for calculating the average surface curvature and acquiring an average surface curvature defect map through visualization processing;
and the detection analysis module is used for analyzing the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map to obtain a detection result.
In the previous description, numerous specific details were set forth in order to provide a thorough understanding of the present invention. The foregoing description is only a preferred embodiment of the invention, which can be embodied in many different forms than described herein, and therefore the invention is not limited to the specific embodiments disclosed above. And that those skilled in the art may, using the methods and techniques disclosed above, make numerous possible variations and modifications to the disclosed embodiments, or modify equivalents thereof, without departing from the scope of the claimed embodiments. Any simple modification, equivalent change and modification of the above embodiments according to the technical essence of the present invention are within the scope of the technical solution of the present invention.
Claims (10)
1. An object surface flaw detection and analysis method based on photometric stereo is characterized by comprising the following steps:
s10, acquiring all pixel values of the surface image of the object to be detected of at least three light sources with different angles;
s20, constructing a luminosity three-dimensional mathematical model according to the pixel values;
s30, calculating the photometric stereo mathematical model to obtain a reflectivity and a normal vector;
s40, constructing a reflectivity map by visualizing the reflectivity;
s50, calculating to obtain a gradient field of the surface of the object to be measured according to the normal vector;
s60, according to the gradient field of the surface of the object to be measured, Gaussian surface curvature calculation is carried out, and a Gaussian surface curvature defect map is obtained through visualization processing;
s70, according to the gradient field of the surface of the object to be measured, calculating the average surface curvature and obtaining an average surface curvature defect map through visualization processing;
and S80, analyzing the reflectivity diagram, the Gaussian surface curvature defect diagram and the average surface curvature defect diagram to obtain a detection result.
2. The method of claim 1, wherein the object surface defect detection and analysis method based on photometric stereo is characterized in that,
the optical stereo mathematical model in S20 is,
I＝ρn·s
wherein, I is the image pixel value, n is the unit normal vector, s is the unit light source incidence vector, and rho is the object surface reflectivity.
3. The method as claimed in claim 2, wherein the reflectivity and normal vector obtained by the calculation of the photometric stereo mathematical model in S30 are,
solving the photometric stereo mathematical model by adopting a least square estimation method,
order to
Then there is
Solved to obtain
Wherein T denotes transpose, STRepresenting the transpose of the matrix S.
4. The method for detecting and analyzing surface flaws of an object based on photometric stereo as claimed in claim 2 or 3 wherein said step of S60 is based on gradient field of the object surface to be measured, and is characterized in that the Gaussian surface curvature calculation is performed as,
normalizing the unit normal vector n in the z direction to obtain a first derivative p in the x direction and a first derivative q in the y direction; carrying out Gaussian derivative filtering in the x direction on the gradient map in the x direction to obtain a second derivative p in the x directionx(ii) a Carrying out Gaussian derivative filtering in the y direction on the gradient map in the x direction to obtain a second derivative p in the x directiony；
Carrying out Gaussian derivative filtering in the y direction on the gradient map in the y direction to obtain a second derivative q in the y directiony(ii) a Carrying out Gaussian derivative filtering in the x direction on the gradient map in the y direction to obtain a second derivative q in the y directionx(ii) a Gaussian curvature
5. The method for detecting and analyzing surface flaws of an object based on photometric stereo as claimed in claim 2 or 3 wherein the average surface curvature calculation according to the gradient field of the surface of the object to be measured in S70 is performed as,
normalization in the z-direction for a unit normal vector nObtaining a first derivative p in the x direction and a first derivative q in the y direction; carrying out Gaussian derivative filtering in the x direction on the gradient map in the x direction to obtain a second derivative p in the x directionx(ii) a Carrying out Gaussian derivative filtering in the y direction on the gradient map in the x direction to obtain a second derivative p in the x directiony；
Carrying out Gaussian derivative filtering in the y direction on the gradient map in the y direction to obtain a second derivative q in the y directiony(ii) a Carrying out Gaussian derivative filtering in the x direction on the gradient map in the y direction to obtain a second derivative q in the y directionx(ii) a The mean curvature can be calculated
A＝(1+p2)qy
B＝pq(py+qx)
C＝(1+q2)px
Where H is the mean curvature.
6. The method for detecting and analyzing the surface defects of the object based on the photometric stereo according to any one of claims 1 to 5, wherein the analysis of the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map in S80 is,
and carrying out binarization analysis on the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map.
7. The method for detecting and analyzing the surface defects of the object based on the photometric stereo according to any one of claims 1 to 5, wherein the analysis of the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map in S80 is,
and carrying out connected domain analysis on the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map.
8. The method for detecting and analyzing the surface defects of the object based on the photometric stereo according to any one of claims 1 to 5, wherein the analysis of the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map in S80 is,
and carrying out linear detection analysis on the reflectivity graph, the Gaussian surface curvature defect graph and the average surface curvature defect graph.
9. The method for object surface defect detection and analysis based on photometric stereo according to any one of claims 1-8 characterized in that the visualization process is converted to grey scale map for display by normalization to grey scale value between 0-255.
10. An object surface flaw detection and analysis device based on photometric stereo is characterized by comprising:
the pixel module is used for acquiring all pixel values of the surface image of the object to be detected of at least three light sources with different angles;
the mathematical model module is used for constructing a luminosity three-dimensional mathematical model according to the pixel values;
the calculation module is used for calculating the photometric stereo mathematical model to obtain the reflectivity and normal vector;
the reflectivity module is used for constructing a reflectivity graph by performing visualization processing on the reflectivity;
the gradient field module is used for calculating to obtain a gradient field of the surface of the object to be measured according to the luminosity three-dimensional mathematical model;
the Gaussian surface curvature module is used for calculating the Gaussian surface curvature according to the gradient field of the surface of the object to be detected and acquiring a Gaussian surface curvature defect map through visualization processing;
the average surface curvature module is used for calculating the average surface curvature and acquiring an average surface curvature defect map through visualization processing;
and the detection analysis module is used for analyzing the reflectivity map, the Gaussian surface curvature defect map and the average surface curvature defect map to obtain a detection result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110973139.4A CN113658155A (en) | 2021-08-24 | 2021-08-24 | Object surface flaw detection and analysis method and device based on photometric stereo |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110973139.4A CN113658155A (en) | 2021-08-24 | 2021-08-24 | Object surface flaw detection and analysis method and device based on photometric stereo |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113658155A true CN113658155A (en) | 2021-11-16 |
Family
ID=78492619
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110973139.4A Pending CN113658155A (en) | 2021-08-24 | 2021-08-24 | Object surface flaw detection and analysis method and device based on photometric stereo |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113658155A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723727A (en) * | 2022-04-21 | 2022-07-08 | 广州大学 | Visual detection method and device for surface dents |
| CN115170804A (en) * | 2022-07-26 | 2022-10-11 | 无锡九霄科技有限公司 | Surface defect detection method, device, system and medium based on deep learning |
| CN115575399A (en) * | 2022-09-29 | 2023-01-06 | 梅卡曼德(上海)机器人科技有限公司 | Surface defect detection method and surface defect detection system |
| CN115656189A (en) * | 2022-12-12 | 2023-01-31 | 易斯德(福建)智能科技有限公司 | Defect detection method and device based on luminosity stereo and deep learning algorithm |
| CN116402789A (en) * | 2023-03-30 | 2023-07-07 | 开思时代科技(深圳)有限公司 | Vehicle paint spraying demand data analysis method and system |
| CN116559181A (en) * | 2023-07-07 | 2023-08-08 | 杭州灵西机器人智能科技有限公司 | Defect detection method, system, device and medium based on luminosity stereoscopic vision |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6219438B1 (en) * | 1997-09-02 | 2001-04-17 | Lucent Technologies Inc. | Produce indentifier using barcode scanner and wavelet image processing and having compensation for dirt accumulated on viewing window |
| US20050111713A1 (en) * | 2003-08-13 | 2005-05-26 | Anna Jerebko | Method of analysis of local patterns of curvature distributions |
| JP2009180597A (en) * | 2008-01-30 | 2009-08-13 | Dainippon Screen Mfg Co Ltd | Defect detecting device and defect detecting method |
| CN106780726A (en) * | 2016-12-23 | 2017-05-31 | 陕西科技大学 | The dynamic non-rigid three-dimensional digital method of fusion RGB D cameras and colored stereo photometry |
| CN108593710A (en) * | 2018-06-14 | 2018-09-28 | 湖南大学 | A kind of the thermal imaging detecting system and method for highly reflective material surface defect |
| CN109523541A (en) * | 2018-11-23 | 2019-03-26 | 五邑大学 | A kind of metal surface fine defects detection method of view-based access control model |
| CN110308153A (en) * | 2019-08-03 | 2019-10-08 | 广西师范大学 | Metal works defect inspection method, system, storage medium and device based on single eye stereo vision |
| CN111192225A (en) * | 2020-04-14 | 2020-05-22 | 北京智拓视界科技有限责任公司 | Method, apparatus and computer program medium for determining whether a pipe is defective |
| CN112232109A (en) * | 2020-08-31 | 2021-01-15 | 深圳奥比中光科技有限公司 | Living body face detection method and system |
| CN112435248A (en) * | 2020-11-30 | 2021-03-02 | 深圳市泰沃德技术有限公司 | Defect detection method, defect detection device, control device and readable storage medium |
| CN112734756A (en) * | 2021-03-29 | 2021-04-30 | 惠州高视科技有限公司 | Detection method and system based on photometric stereo vision |
-
2021
- 2021-08-24 CN CN202110973139.4A patent/CN113658155A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6219438B1 (en) * | 1997-09-02 | 2001-04-17 | Lucent Technologies Inc. | Produce indentifier using barcode scanner and wavelet image processing and having compensation for dirt accumulated on viewing window |
| US20050111713A1 (en) * | 2003-08-13 | 2005-05-26 | Anna Jerebko | Method of analysis of local patterns of curvature distributions |
| JP2009180597A (en) * | 2008-01-30 | 2009-08-13 | Dainippon Screen Mfg Co Ltd | Defect detecting device and defect detecting method |
| CN106780726A (en) * | 2016-12-23 | 2017-05-31 | 陕西科技大学 | The dynamic non-rigid three-dimensional digital method of fusion RGB D cameras and colored stereo photometry |
| CN108593710A (en) * | 2018-06-14 | 2018-09-28 | 湖南大学 | A kind of the thermal imaging detecting system and method for highly reflective material surface defect |
| CN109523541A (en) * | 2018-11-23 | 2019-03-26 | 五邑大学 | A kind of metal surface fine defects detection method of view-based access control model |
| CN110308153A (en) * | 2019-08-03 | 2019-10-08 | 广西师范大学 | Metal works defect inspection method, system, storage medium and device based on single eye stereo vision |
| CN111192225A (en) * | 2020-04-14 | 2020-05-22 | 北京智拓视界科技有限责任公司 | Method, apparatus and computer program medium for determining whether a pipe is defective |
| CN112232109A (en) * | 2020-08-31 | 2021-01-15 | 深圳奥比中光科技有限公司 | Living body face detection method and system |
| CN112435248A (en) * | 2020-11-30 | 2021-03-02 | 深圳市泰沃德技术有限公司 | Defect detection method, defect detection device, control device and readable storage medium |
| CN112734756A (en) * | 2021-03-29 | 2021-04-30 | 惠州高视科技有限公司 | Detection method and system based on photometric stereo vision |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723727A (en) * | 2022-04-21 | 2022-07-08 | 广州大学 | Visual detection method and device for surface dents |
| CN115170804A (en) * | 2022-07-26 | 2022-10-11 | 无锡九霄科技有限公司 | Surface defect detection method, device, system and medium based on deep learning |
| CN115170804B (en) * | 2022-07-26 | 2024-01-26 | 无锡九霄科技有限公司 | Surface defect detection method, device, system and medium based on deep learning |
| CN115575399A (en) * | 2022-09-29 | 2023-01-06 | 梅卡曼德(上海)机器人科技有限公司 | Surface defect detection method and surface defect detection system |
| CN115656189A (en) * | 2022-12-12 | 2023-01-31 | 易斯德(福建)智能科技有限公司 | Defect detection method and device based on luminosity stereo and deep learning algorithm |
| CN115656189B (en) * | 2022-12-12 | 2023-03-17 | 易斯德(福建)智能科技有限公司 | Defect detection method and device based on luminosity stereo and deep learning algorithm |
| CN116402789A (en) * | 2023-03-30 | 2023-07-07 | 开思时代科技(深圳)有限公司 | Vehicle paint spraying demand data analysis method and system |
| CN116402789B (en) * | 2023-03-30 | 2023-11-28 | 开思时代科技(深圳)有限公司 | Vehicle paint spraying demand data analysis method and system |
| CN116559181A (en) * | 2023-07-07 | 2023-08-08 | 杭州灵西机器人智能科技有限公司 | Defect detection method, system, device and medium based on luminosity stereoscopic vision |
| CN116559181B (en) * | 2023-07-07 | 2023-10-10 | 杭州灵西机器人智能科技有限公司 | Defect detection method, system, device and medium based on luminosity stereoscopic vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113658155A (en) | Object surface flaw detection and analysis method and device based on photometric stereo | |
| CN106650770B (en) | Mura defect detection method based on sample learning and human eye visual characteristics | |
| Ming et al. | A comprehensive review of defect detection in 3C glass components | |
| CN107678192B (en) | Mura defect detection method based on machine vision | |
| KR101477665B1 (en) | Defect detection method in heterogeneously textured surface | |
| CN111047655A (en) | High-definition camera cloth defect detection method based on convolutional neural network | |
| CN111307819B (en) | Wafer edge defect detection system and method | |
| CN108827597B (en) | Light spot uniformity detection method and detection system of structured light projector | |
| CN117392042A (en) | Defect detection method, defect detection apparatus, and storage medium | |
| CN115775236A (en) | Surface tiny defect visual detection method and system based on multi-scale feature fusion | |
| CN114219758A (en) | Defect detection method, system, electronic device and computer readable storage medium | |
| Dominguez-Nicolas et al. | Indentation image analysis for Vickers hardness testing | |
| KR101677070B1 (en) | System and Method for Automatically Detecting a Mura Defect using Morphological Image Processing and Labeling | |
| CN113763322B (en) | Pin Pin coplanarity visual detection method and device | |
| CN117173107A (en) | Product defect detection method, device, equipment, system and readable storage medium | |
| JP7411155B2 (en) | Color unevenness inspection device and color unevenness inspection method | |
| Lin et al. | Computer-aided transmitted deformation inspection system for see-through glass products | |
| Chiu et al. | Effective image models for inspecting profile flaws of car mirrors with applications | |
| CN114882027B (en) | Electronic equipment chip pin defect detection method and system | |
| Tsai et al. | Surface defect detection of 3D objects using robot vision | |
| CN114428412B (en) | Image recognition device and image recognition method | |
| Xu et al. | Non-Contact Extensometer Deformation Detection via Deep Learning and Edge Feature Analysis | |
| Zhang et al. | Defect Inspection for Curved Surface with Highly Specular Reflection | |
| Lin et al. | Vision-based deformation inspection system for automotive glass using Hough circle detectors | |
| Khatyreva et al. | Unsupervised anomaly detection for industrial manufacturing using multiple perspectives of free falling parts |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |