CN102288613B - Surface defect detecting method for fusing grey and depth information - Google Patents

Abstract

The invention relates to an on-line detecting method for surface defects of an object and a device for realizing the method. The accuracy for the detection and the distinguishing of the defects is improved through the fusion of grey and depth information, and the method and the device can be applied to the detection of the object with a complicated shape and a complicated surface. A grey image and a depth image of the surface of the object are collected by utilizing the combination of a single colored area array CCD (charge-coupled device) camera and a plurality of light sources with different colors, wherein obtaining of the depth information is achieved through a surface structured light way. The division and the defect edge extraction of the images are carried out through the pixel level fusion of the depth image and the grey image, so that the area where the defects are positioned can be detected more accurately. According to the detected area with the defects, the grey characteristics, the texture characteristics and the two-dimensional geometrical characteristics of the defects are extracted from the grey image; the three-dimensional geometrical characteristics of the defects are extracted from the depth image; further, the fusion of characteristic levels is carried out; and a fused characteristic quantity is used as the input of a classifier to classify the defects, thereby achieving the distinguishing of the defects.

Description

The detection method of surface flaw that a kind of gray scale and depth information merge

Technical field

The device that the present invention relates to a kind of body surface defect online detection method and realize the method, especially the product surface for shape or surface appearance more complicated detects, and can improve the accuracy rate of defects detection.

Background technology

Surface imperfection Automatic Measurement Technique starts from 20 century 70s, and to 20 end of the centurys, the developed countries such as Germany, the U.S., Japan have developed the machine vision surface on-line detecting system with practical value.Mechanical vision inspection technology is divided by its handled data type, can be divided into the detection based on bianry image, gray level image, coloured image and depth image, and current surface detecting system generally adopts gray level image detection method.Gray level image detection method is by camera acquisition body surface gray level image, and the variation according to defect area in gray scale, processes gray level image, and utilizes the gray feature of defect to detect and identification defect.This gray scale detection method exists following problem:

(1) gray level image detection method is applicable to the body surface detection that background is single, surface for background complexity, owing to being difficult to judge from background by half-tone information the region at defect place, therefore can produce a large amount of undetected and flase drops, affect the reliability of testing result.

(2) gray level image detection method is applicable to the surface detection of planar object, is the object of rule or irregular surface for surface, adopts gray level image detection method can cause even eclipse phenomena of shade, and impact detects effect, produces undetected.

(3) degree of depth of surface imperfection is not only and is characterized defect order of severity important indicator, and is the important parameter that carries out defects detection and identification.Adopt gray level image detection method cannot obtain the quantitative information of depth of defect, can impact the correct assessment of the accurate identification of defect and surface quality.

By depth image, can obtain case depth information, and detect by depth image the impact that defect is not subject to body surface background situation and body form, thereby solves gray level image and detect the problem existing, but the application of depth image detection method in surface is detected also exists following problem:

(1) by depth image, detect the defect that defect is only applicable to change in depth, for some, there is no the defect of the degree of depth, as spot class defect, cannot detect by depth image.

(2) some interfering materials of body surface also can cause the variation on case depth, adopt depth image detection method these interfering objects can be identified as to defect, cause mistake to know.

(3) current depth image acquisition methods cannot reach high resolution, therefore, is difficult to detect some tiny defects by depth image, as crackle, impression etc.

Therefore, adopt merely depth image detection method also cannot realize the reliable detection of surface imperfection.Gray level image detection method and depth image detection method are combined, by the fusion of half-tone information and depth information, can be in conjunction with the advantage of two kinds of methods, thus improve the reliability of defects detection.

Traditional information fusion often refers to the information fusion of multisensor, if obtain half-tone information and depth information by independent video camera respectively, not only need to increase the quantity of video camera, but also the image that need to collect different cameras carries out registration, increased the complicacy of system.The present invention is by the combination of single colourful video camera and many different colored light sources, gray level image and depth image that separate unit video camera obtains body surface have simultaneously been realized, not only reduced the quantity of video camera, and gray level image corresponding with depth image be the same area of body surface, without the registration that carries out image, reduced the difficulty of information fusion.

Summary of the invention

The present invention utilizes separate unit video camera to obtain gray level image and the depth image of body surface simultaneously, the region of the fusion detection defect by half-tone information and depth information, and defect is identified, thereby improve the accuracy rate of defects detection and identification.

The present invention is achieved in that

The present invention gathers gray level image and the depth image of body surface by the combination of single colourful Array CCD Camera and blue light or red illumination light source, the same area of described blue light or red illumination light source irradiating object, described depth image is to obtain by area-structure light mode, the light source green glow of area-structure light, blue light or ruddiness for the lighting source of described gray level image, from the coloured image collecting, isolate the image of blue light or ruddiness and green glow, wherein blue light or ruddiness image are exactly surperficial gray level image, and green glow image is exactly area-structure light projected image; By the Pixel-level fusion of depth image and gray level image, carrying out cutting apart with Defect Edge of image extracts, thereby can detect more accurately the region at defect place, according to the defect area detecting, from gray level image, extract gray feature, textural characteristics and the two-dimensional geometry feature of defect, from depth image, extract the three-dimensional geometry feature of defect, and carry out feature level fusion, using the characteristic quantity after fusion as the input of sorter, carry out the Classification and Identification of defect.

Another technical scheme of the present invention is that above-mentioned feature level fusion is by the fusion of depth image and gray level image, to carry out cutting apart with Defect Edge of image to extract, because surface imperfection has caused the variation of object surface depth, therefore in depth image, find the point that the degree of depth is undergone mutation, according to these degree of depth catastrophe points to Image Segmentation Using, because the resolution of gray level image is than depth map image height, therefore can in gray level image, according to the segmentation result of depth image, carry out the edge of defect, thereby obtain the accurate edge of defect.

Another technical scheme of the present invention is that the Classification and Identification of above-mentioned defect is that feature level by depth image and gray level image merges, from gray level image, extract defect gray feature, textural characteristics and two-dimensional geometry feature, described defect gray feature comprises mean value, variance, flexure, kurtosis, energy, the entropy of gray scale; Described textural characteristics comprises second moment, entropy, contrast and the homogeneity of texture; Described two-dimensional geometry feature comprises simple descriptor, shape description symbols and bending moment etc. not, and the described three-dimensional geometry feature of extracting defect from depth image, comprises not bending moment and Fourier descriptor.

The present invention mainly improves the reliability of defects detection and identification by following two aspects:

(1) in carrying out the Pixel-level fusion process of half-tone information and depth information, utilize surperficial depth information to cut apart gray level image, and extract the edge of defect, can improve like this gray level image and cut apart the accuracy of extracting with Defect Edge.The difficult point that image is cut apart is the uncertainty of half-tone information, namely space three-dimensional object is projected to two dimensional image, has lost a lot of information.Utilizing depth information to carry out gray level image cuts apart and has improved the accuracy cut apart and the efficiency of partitioning algorithm.Edge extracting is also the difficult point during image is processed, and because the edge of defect can produce shade or uneven illumination phenomenon, is difficult to accurately extract complete Defect Edge.The accuracy that raising image is cut apart can reduce the undetected of defect, and the accurate extraction that the accuracy that raising Defect Edge extracts can be defect characteristic provides basis, thereby reduces the flase drop of defect.

(2) by traditional defect characteristic based on gray level image, as gray feature, textural characteristics, two-dimensional geometry feature etc. with by the defect three-dimensional geometry Fusion Features of extraction of depth information, can improve the accuracy of classification of defects, reduce the mistake of defect and know.

Meanwhile, half-tone information is extracted and also can improve reliability, precision and the efficiency that stripe edge is extracted for the stripe edge of area-structure light projected image, thus the accuracy and efficiency of raising extraction of depth information.

Adopting blue light is that blue light wavelength is short as the object of gray level image lighting source, for small defect, will have better detection effect.While adopting green glow to be the illumination for high temp objects as the object of area-structure light light source, because ruddiness and the infrared light of high temp objects radiation exists the different of wave band from the green glow of lighting source, green glow can be extracted from radiant light, thereby avoid the impact of bias light.For some, need surface bright, that details in a play not acted out on stage, but told through dialogues combination detects, light source that can be using blue diffused light as bright field illumination, red directional light is as the light source of dark ground illumination, camera acquisition to B and R passage be respectively light field image and darkfield image, the reflection of two kinds of images be all surperficial half-tone information.The depth information obtaining in conjunction with G passage detects defect simultaneously, can further improve the accuracy of defects detection.

The invention has the beneficial effects as follows: by the fusion of gray scale and depth information, not only can improve the accuracy rate of defects detection, and can improve the efficiency of algorithm, meet the online requirement detecting.

Accompanying drawing explanation

Fig. 1 is the device that obtains surperficial gray level image and depth image.In Fig. 1: 1 is object to be detected, 2 is colored Array CCD Camera, and 3 is blue diffused light light source, and 4 is green area-structure light light source.

Fig. 2 is the detection method of surface flaw process flow diagram that half-tone information and depth information merge.

Embodiment

In Fig. 1, blue diffused light light source 3 and green area-structure light light source 4 are irradiated to the same area on examined object 1 surface, and colored Array CCD Camera 2 gathers the image of examined object 1 surperficial light areas.In the coloured image collecting at video camera 2, blue channel (being B passage) is exactly the reflected light image that light source 3 is irradiated to object 1 surface, and green channel (G passage) is exactly the reflected light image that light source 4 is irradiated to object 1 surface.Due to light source 3 transmitting be diffused light, so B channel image is the surperficial gray level image of object 1.Due to light source 3 transmitting be area-structure light, so G channel image is the area-structure light image on object 1 surface.Area-structure light projected image also needs to carry out depth extraction step, just can convert surperficial depth image to.Its step is as follows:

(1) extract area-structure light projected image stripe edge coordinate, this coordinate is the image coordinate of stripe edge in two dimensional image plane.

(2) according to formula 1, calculate the three dimensional space coordinate of stripe edge:

$s\left[\begin{array}{c}{u}_i\\ v_i\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{Wi}}\\ Y_{\mathrm{Wi}}\\ Z_{\mathrm{Wi}}\\ 1\end{array}\right]---\left(1\right)$

In formula: s is a scale factor, (X _wi, T _wi, Z _wi) be the three dimensional space coordinate of i unique point; (u _i, v _i) be the image coordinate of i unique point; m _ijfor the camera parameters obtaining through demarcation in advance.

(3) Z step 2 being obtained _winormalize to [0,255],

will

project to (u _i, v _i) plane, with

for gray-scale value, can obtain surperficial depth image.

Due to depth image corresponding with gray level image be the same area of body surface, without registration, just can carry out image co-registration.Image co-registration process is divided into two processes:

(1) Pixel-level merges.Pixel-level merges the region for detection of defect place, and its main performing step is that image is cut apart with Defect Edge and extracted.Therefore image segmentation step completes by depth image, and its principle is due to surface imperfection, to have caused the variation of object surface depth, finds the point that the degree of depth is undergone mutation in depth image, according to these degree of depth catastrophe points to Image Segmentation Using.In depth image, carry out the advantage that image cuts apart and be that depth image is not subject to the impact of body surface situation and surround lighting, the reliability of segmentation result is high.But, because the resolution of depth image is low, therefore in depth image, be difficult to extract the accurate edge of defect, need in gray level image, according to the segmentation result of depth image, carry out the edge extracting of defect.Due to depth image corresponding with gray level image be the same area, so the segmentation result of depth image can be directly used in Defect Edge in gray level image, extract.

(2) feature level merges.Feature level fusion is for the Classification and Identification of defect, and the two dimensional character that its main step is defect extracts with three-dimensional feature and extracts, and will after two dimensional character and three-dimensional feature fusion, as characteristic quantity, defect be classified.According to the defect area detecting, from gray level image, extract gray feature, textural characteristics and the two-dimensional geometry feature of defect, from depth image, extract the three-dimensional geometry feature of defect, and carry out the fusion of feature level, characteristic quantity after merging carries out classification of defects as the input of sorter, thereby realizes the identification of defect.

Gray feature obtains by the grey level histogram of image, comprising:

Mean value:

$\mathrm{Mean}=\underset{b=0}{\overset{L-1}{\mathrm{\Σ}}}\mathrm{bP}\left(b\right)---\left(2\right)$

Variance:

${\mathrm{Var}}^2=\underset{b=0}{\overset{L-1}{\mathrm{\Σ}}}{(b-\stackrel{\‾}{b})}^{2}P\left(b\right)---\left(3\right)$

Flexure:

$\mathrm{Skewness}=\frac{1}{{\mathrm{Var}}^3}\underset{b=0}{\overset{L-1}{\mathrm{\Σ}}}{(b-\stackrel{\‾}{b})}^{3}P\left(b\right)---\left(4\right)$

Kurtosis:

$\mathrm{Kurtosis}=\frac{1}{{\mathrm{<figure-callout\; id="4"\; label="Var"\; filenames="HDA0000060486420000011.png"\; state="\{\{state\}\}">Var</figure-callout>}}^4}\underset{b=0}{\overset{L-1}{\mathrm{\&Sigma;}}}{(b-\stackrel{\&OverBar;}{b})}^{4}P\left(b\right)-3---\left(5\right)$

Energy:

$\mathrm{Power}=\underset{b=0}{\overset{L-1}{\mathrm{\&Sigma;}}}P{\left(b\right)}^2---\left(6\right)$

Entropy:

$\mathrm{Entropy}=-\underset{b=0}{\overset{L-1}{\mathrm{\&Sigma;}}}P\left(b\right){\log }_2\left[P\left(b\right)\right]---\left(7\right)$

In formula 2-7, the single order probability distribution that P (b) is gradation of image, it is defined as:

P(b)＝P{g(i，j)＝b}(0≤b≤L-1) (8)

In formula 8, b is the quantized level of image, is total to L level, and g (i, j) is the gray-scale value of pixel (i, j).

Textural characteristics adopts statistic law, by gray level co-occurrence matrixes P (g ₁, g ₂) obtain P (g ₁, g ₂) represent normalization after gray-scale value be respectively g ₁and g ₂pixel to number.Textural characteristics comprises:

Second moment

$W_M=\underset{g_1}{\mathrm{\&Sigma;}}\underset{g_2}{\mathrm{\&Sigma;}}{P}^2(g_1,g_2)---\left(9\right)$

Entropy

$W_E=-\underset{g_1}{\mathrm{\&Sigma;}}\underset{g_2}{\mathrm{\&Sigma;}}P(g_1,g_2)\log P(g_1,g_2)---\left(10\right)$

Contrast

$W_C=\underset{g_1}{\mathrm{\&Sigma;}}\underset{g_2}{\mathrm{\&Sigma;}}|g_1-g_2|P(g_1,g_2)---\left(11\right)$

Homogeneity

$W_H=\underset{g_1}{\mathrm{\&Sigma;}}\underset{g_2}{\mathrm{\&Sigma;}}\frac{P(g_1,g_2)}{k+|g_1-g_2|}---\left(12\right)$

In formula 12, k is a non-zero constant.

Two-dimentional geometrical shape by simple descriptor, shape description symbols and not bending moment etc. represent, simple descriptor comprises region area, border girth, regional barycenter etc., shape description symbols comprises compactedness, excentricity, spherical property and circle.Three-dimensional geometry feature comprises not bending moment and the Fourier descriptor of defect area in depth image.

Claims (3)

1. the detection method of surface flaw that a gray scale and depth information merge, it is characterized in that: the gray level image that gathers body surface by the combination of single colourful Array CCD Camera and blue light or red illumination light source, the same area of described blue light or red illumination light source irradiating object, described depth image is to obtain by area-structure light mode, the light source green glow of area-structure light, blue light or ruddiness for the lighting source of described gray level image, from the coloured image collecting, isolate the image of blue light and green glow or the image of ruddiness and green glow, wherein blue light or ruddiness image are exactly surperficial gray level image, green glow image is exactly area-structure light projected image, by the Pixel-level fusion of depth image and gray level image, carrying out cutting apart with Defect Edge of image extracts, thereby can detect more accurately the region at defect place, according to the defect area detecting, from gray level image, extract gray feature, textural characteristics and the two-dimensional geometry feature of defect, from depth image, extract the three-dimensional geometry feature of defect, and carry out feature level fusion, using the characteristic quantity after fusion as the input of sorter, carry out the Classification and Identification of defect, described depth extraction step is as follows: (1) extracts area-structure light projected image stripe edge coordinate, and this coordinate is the image coordinate of stripe edge in two dimensional image plane,

(2) according to formula 1, calculate the three dimensional space coordinate of stripe edge:

In formula: s is a scale factor, (X _wi, Y _wi, Z _wi) be the three dimensional space coordinate of i unique point; (u _i, v _i) be the image coordinate of i unique point; m _ijfor the camera parameters obtaining through demarcation in advance;

(3) Z step 2 being obtained _winormalize to [0,255],

, will project to (u _i, v _i) plane, with

for gray-scale value, can obtain surperficial depth image.

2. the detection method of surface flaw that a kind of gray scale as claimed in claim 1 and depth information merge, it is characterized in that: it is by the defect characteristic of traditional gray level image and by the defect three-dimensional geometry Fusion Features of extraction of depth information that described feature level merges, because surface imperfection has caused the variation of object surface depth, therefore in depth image, find the point that the degree of depth is undergone mutation, according to these degree of depth catastrophe points to Image Segmentation Using, because the resolution of gray level image is than depth map image height, therefore can in gray level image, according to the segmentation result of depth image, carry out the edge extracting of defect, thereby obtain the accurate edge of defect.

3. the detection method of surface flaw that a kind of gray scale as claimed in claim 1 and depth information merge, it is characterized in that: the Classification and Identification of described defect is that the feature level by depth image and gray level image merges, from gray level image, extract defect gray feature, textural characteristics and two-dimensional geometry feature, described defect gray feature comprises mean value, variance, flexure, kurtosis, energy, the entropy of gray scale; Described textural characteristics comprises second moment, entropy, contrast and the homogeneity of texture; Described two-dimensional geometry feature comprises simple descriptor, shape description symbols and bending moment not, and the described three-dimensional geometry feature of extracting defect from depth image, comprises not bending moment and Fourier descriptor.

Images