CN114219797A - MEMS acoustic film surface defect detection method based on frequency domain transformation - Google Patents

Abstract

The invention discloses a MEMS acoustic film surface defect detection method based on frequency domain transformation, which comprises the steps of calculating a Fourier spectrogram and a frequency spectrum gradient chart of an MEMS acoustic film surface defect image through two-dimensional discrete Fourier transformation, calculating a segmentation threshold Tg between 2 peak values in a superposition gradient histogram through a maximum inter-class variance method, establishing a Boolean mask and carrying out morphological operation on the Boolean mask, eliminating a main frequency component corresponding to a periodic structure identified by gradient calculation in a frequency domain according to the Boolean mask, carrying out two-dimensional discrete Fourier inverse transformation on a residual frequency spectrum to obtain a reconstructed image containing a defect region, and finally carrying out binary segmentation by adopting a threshold method to output a binary image containing defect information. The invention not only solves the problem that the traditional defect detection algorithm depends on an imaging light source and a reference image, but also has certain noise resistance and is not interfered by factors such as image translation, rotation, scaling and the like.

Description

MEMS acoustic film surface defect detection method based on frequency domain transformation

Technical Field

The invention belongs to the technical field of machine vision, and particularly relates to a method for detecting surface defects of an MEMS acoustic film based on frequency domain transformation.

Background

Micro-electro-mechanical systems (MEMS) are high-technology sensing systems that integrate multiple disciplines such as microelectronics, mechanics, etc., and have been widely used in the fields of aerospace, communications, etc. The MEMS acoustic membrane is an important implementation device for mutual conversion of acoustic (mechanical vibration)/electrical (voltage, current) signals as a core device in earphones and microphones. However, the device has the disadvantages of small size, various structures, complex production process and extremely high requirements on the environment of tape-out, storage and packaging. Impurities such as water vapor, fibers and dust suspended in the air may adhere to the surface of the acoustic membrane in the form of surface defects and negatively affect the vibration mode thereof. In order to ensure the product quality, detection is required before and after the flow sheet process, the cutting and the packaging.

The traditional MEMS surface defect detection method (photoelectric detection, eddy current inspection and high-resolution X-ray diffraction HRXRD) has extremely high requirements on detection equipment and is likely to cause secondary damage to the micro-component. The manual detection method satisfies the principle of no contact, but is affected by subjective factors with low efficiency. In recent years, researchers have proposed many surface defect detection algorithms, which can be roughly classified into a reference method that relies on a defect-free template image and a non-reference method that does not rely on a template. However, the performance of the reference method is affected by factors such as image distortion caused by image rotation, scaling and uneven illumination conditions, and when the device structure is complex, mismatching of feature vectors is obvious, so that a large amount of noise is generated after image difference calculation, and the stability is poor. In the non-reference method, analysis methods such as fourier transform, wavelet transform and Gabor transform are generally used to convert the two-dimensional image signal into a frequency domain, and texture features of the background are extracted according to a related frequency domain distribution criterion, so that research can be performed by combining frequency domain characteristics and spatial domain characteristics of background textures. Although Gabor transformation conforms to the visual perception characteristic of human and has better orientation selectivity, the calculated amount for formulating filter parameters is large, and the requirement of rapid detection is not met.

Disclosure of Invention

The invention provides a method for detecting surface defects of an MEMS (micro-electromechanical systems) acoustic film based on frequency domain transformation, which is based on a machine vision detection technology, detects common defect types (patch, strip and radial) existing on the surface of the MEMS acoustic film by a non-contact and non-supervision detection means, and outputs a binary image of a defect area.

In order to achieve the purpose, the technical scheme of the invention is realized as follows:

a MEMS acoustic film surface defect detection method based on frequency domain transformation comprises the following steps:

s1, calculating a Fourier spectrogram and a frequency spectrum gradient chart of the MEMS acoustic film surface defect image through two-dimensional discrete Fourier transform, and performing bidirectional maximum filtering with the size of 3X 3 to highlight a peak value and inhibit noise;

s2, calculating a segmentation threshold Tg between 2 peak values in the superimposed gradient histogram by a maximum inter-class variance method, establishing a Boolean mask and carrying out morphological operation on the Boolean mask to ensure that the peak values highlighted in the frequency spectrum are all covered by the mask;

and S3, eliminating the main frequency component corresponding to the periodic structure identified by gradient calculation in the frequency domain according to the Boolean mask, performing two-dimensional inverse discrete Fourier transform on the residual frequency spectrum to obtain a reconstructed image containing the defect region, and finally performing binarization segmentation by using a threshold method to output a binarized image containing the defect information.

Further, the method for calculating the fourier spectrogram in step S1 includes:

the DFT result of the image is represented by F (u, v), and the algorithm is as follows:

the amplitude spectrum of the spectrum is defined as:

A(u,v)＝|F(u,v)|

the numerical range is reduced by adopting a logarithmic spectrum method:

L(u,v)＝log[A(u,v)+1]。

further, the method for calculating the spectral gradient map in step S1 includes:

g_uand g_vRespectively representing gradient values at the coordinate points, calculating a gradient map according to the gradient values at each coordinate point:

further, the rule of the boolean mask in step S2 is:

wherein t is_gIs a scaling constant.

Further, the morphological operation in step S2 includes:

and expanding based on the morphological operation of the square structural elements, and adding a square area in the center of the mask, wherein the square area corresponds to a DC (direct current) item in a frequency spectrum, so that the defect information is reduced.

Further, the method of performing two-dimensional inverse discrete fourier transform in step S3 includes:

calculating the frequency spectrum of the aperiodic pattern frequency component:

and performing two-dimensional inverse discrete Fourier transform on the defect image to reconstruct the defect image, wherein the formula is as follows:

further, the threshold method in step S3 includes:

the following method is adopted for threshold segmentation:

wherein mu_dIs the mean, σ, of the image_dIs the variance of the image, t_dAre control constants.

Compared with the prior art, the invention has the following beneficial effects:

(1) the method can effectively remove the periodic structure texture on the surface of the acoustic film and is not dependent on a defect-free reference image;

(2) aiming at the input defect image, the invention eliminates the frequency component corresponding to the larger gradient value through the expanded Boolean mask, reconstructs the defect image through inverse Fourier transform, sets a segmentation threshold value for the inverse Fourier transform reconstructed image to output a binary defect image, discusses and summarizes a scaling constant t in the experimental part_gThe detection results of the MEMS acoustic film defect images of different models show that the defect detection effect of the MEMS acoustic film defect detection device is different in size and type, and the statistical result shows that the defect detection rate of the MEMS acoustic film defect detection device is more than 95%.

(3) The invention not only solves the problem that the traditional defect detection algorithm depends on an imaging light source and a reference image, but also has certain noise resistance and is not interfered by factors such as image translation, rotation, scaling and the like. The detection task of the surface defects of the MEMS acoustic film is well finished, and the method has certain engineering practical value.

Drawings

FIG. 1 is a flowchart of a defect detection algorithm according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a defect image structure according to an embodiment of the present invention;

FIG. 3 is an exemplary image and its spectrogram of an embodiment of the present invention;

FIG. 4 is a graph of bi-directionally filtered spectral gradients in accordance with an embodiment of the present invention;

FIG. 5 is a superimposed gradient histogram of an embodiment of the invention;

FIG. 6 is a Boolean mask diagram of an embodiment of the present invention; where FIGS. 6(a), 6(b), 6(c), 6(d) are all Boolean masks of example image spectra;

FIG. 7 is a Fourier reconstructed image and its spectrogram according to an embodiment of the present invention; wherein FIGS. 7(a) - (b) are defect-free reconstructed images and their masked spectrograms, FIGS. 7(d) - (e) are defect reconstructed images and their masked spectrograms, and FIGS. 7(c) and (f) are Fourier reconstructed difference images;

FIG. 8 is a flow chart of periodic structure texture removal according to an embodiment of the present invention;

FIG. 9 is a diagram of binarization results of an embodiment of the present invention;

FIG. 10 is a comparison graph of reconstruction effect of different scaling constants according to the embodiment of the present invention, wherein FIG. 10(a1) is a radial defect image, and FIG. 10(a2) is a spectrum diagram of a defect image; fig. 10(b1) is a reconstructed image with a scaling constant of 0.5, fig. 10(c1) is a reconstructed image with a scaling constant of 0.7, fig. 10(d1) is a reconstructed image with a scaling constant of 1.0, fig. 10(e1) is a reconstructed image with a scaling constant of 1.5, fig. 10(b2) is a reconstructed image spectrum with a scaling constant of 0.5, fig. 10(c2) is a reconstructed image spectrum with a scaling constant of 0.7, fig. 10(d2) is a reconstructed image spectrum with a scaling constant of 1.0, and fig. 10(e2) is a reconstructed image spectrum with a scaling constant of 1.5;

FIG. 11 is a defect image detection diagram for model XC-25 MEMS of an embodiment of the present invention; fig. 11(a1) is a defect image a, fig. 11(a2) is a reconstructed image of the defect image a, fig. 11(a3) is a binarization result map of the defect image a, fig. 11(b1) is a defect image b, fig. 11(b2) is a reconstructed image of the defect image b, fig. 11(b3) is a binarization result map of the defect image b, fig. 11(c1) is a defect image c, fig. 11(c2) is a reconstructed image of the defect image c, and fig. 11(c3) is a binarization result map of the defect image c;

FIG. 12 is a defect image detection diagram for model XC-29 MEMS of an embodiment of the present invention; fig. 12(a1) is a defect image a, fig. 12(a2) is a reconstructed image of the defect image a, fig. 12(a3) is a binarization result map of the defect image a, fig. 12(b1) is a defect image b, fig. 12(b2) is a reconstructed image of the defect image b, fig. 12(b3) is a binarization result map of the defect image b, fig. 12(c1) is a defect image c, fig. 12(c2) is a reconstructed image of the defect image c, and fig. 12(c3) is a binarization result map of the defect image c.

Detailed Description

It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.

The machine vision detection technology is taken as a multi-field cross comprehensive technology relating to optics, computer technology and image processing, has important research significance and practical value for improving the manufacturing level of MEMS micro-components and reducing the production cost by virtue of the advantages of no contact and high precision, and simultaneously has certain periodicity due to the background texture distribution of the MEMS acoustic membrane, so that the Fourier spectrum of the MEMS acoustic membrane has discreteness and regularity, and the background texture characteristics can be extracted by related frequency distribution criteria to eliminate the periodic structure texture in the background.

Fig. 1 shows a flow chart of the present invention, which mainly comprises three steps:

(1) calculating a Fourier spectrogram and a spectrum gradient map of the defect image through two-dimensional Discrete Fourier Transform (DFT), and performing bidirectional maximum filtering with 3 x 3 size to highlight peaks and inhibit noise;

(2) calculating a segmentation threshold Tg between 2 peaks in the superimposed gradient histogram by using an inter-class variance method (OTSU), establishing a Boolean mask and performing morphological operation 'expansion' on the Boolean mask to ensure that the peaks highlighted in the spectrum are covered by the mask;

(3) and eliminating the main frequency component corresponding to the periodic structure identified by gradient calculation in the frequency domain according to the Boolean mask, performing two-dimensional Inverse Discrete Fourier Transform (IDFT) on the residual frequency spectrum to obtain a reconstructed image containing the defect region, and finally performing binarization segmentation by using a threshold method to output a binarized image containing the defect information.

The specific technical scheme content of each step is as follows:

firstly, Fourier spectrum analysis:

the image containing defects is represented by f (x, y), and the following relationship can be obtained:

f(x,y)＝f_p(x,y)+f_d(x,y) (1)

wherein f is_d(x, y) denotes defects embedded in the sample, f_p(x, y) denotes a periodic texture and the image containing the defect can be regarded as the sum of two pieces of information (as shown in fig. 2). The object of the invention is to separate from the image containing defects without prior information on the texture of the periodic structure. Since the fourier spectrum is suitable for describing the periodicity of an image, f (x, y) can be described by a DFT operation of f_d(x, y) are estimated, respectively

As f_d(x, y) and f_pEstimated value of (x, y):

in that

The medium defect information is preserved and most of the periodic structure texture information is eliminated.

The two-dimensional discrete Fourier transform has wide application in the field of image processing, and realizes the conversion of image signals in a space domain and a frequency domain. The DFT result of the image is represented by F (u, v), and the algorithm is as follows:

the amplitude spectrum of the spectrum is defined as:

A(u,v)＝|F(u,v)| (4)

in general, A (u, v) has a relatively wide dynamic range. However, the gradient value in the high frequency region is much smaller than that in the low frequency region, which is not beneficial to the analysis of the gradient, so the numerical range is reduced by using a logarithmic spectrum method:

L(u,v)＝log[A(u,v)+1] (5)

the fourier spectrum of the example image is shown in fig. 3. It can be seen that the fourier spectrum has good applicability to periodic texture in grayscale images. For a defect-free image containing only periodic signals, the spectrum consists of a regularly distributed series of harmonic frequency points, which are represented by a series of prominent regular peaks in the spectrum containing the defective image. In different cases, however, the frequency component distributions of different periodic structure textures are more complex and varied due to leakage effects or Gibbs phenomenon (Gibbs), yet they are still prominent regular peaks in the frequency spectrum. By correcting these peaks and their neighboring peaks to 0, the periodic pattern can be effectively removed. The frequency components associated with the periodic pattern in the frequency spectrum have steeper peaks, i.e., larger gradient values, than those of the defect region. Therefore, the information of the periodic pattern can be extracted from the frequency spectrum of the defect image by the method of measuring the gradient, i.e. the frequency components in the frequency spectrum are separated by using the gradient distribution. The spectrum characteristics of Fourier transform show that the spectrum of the image is unchanged after the image is translated; when the image F (x, y) rotates, the phase of the image changes, and the frequency spectrum F (u, v) rotates by the same angle, but the overall distribution of frequency components in the frequency spectrum is not changed; the scaling of the image results in a scaling of the spectrum without affecting the gradient distribution in the spectrum. Therefore, it can be shown that the algorithm based on frequency domain transformation is not affected by the factors of image translation, rotation and scaling.

Secondly, removing the periodic structure texture:

fig. 8 is a flowchart illustrating the removal of the periodic structure texture, which includes:

the gradient values are calculated as defined below:

g_uand g_vRepresenting the gradient values of the coordinate points in the sum direction, respectively. Calculating a gradient map from the gradient values at each coordinate point:

however, the central gradient value of the peak in the spectrum may be smaller than the gradient values of the edges. In order for the region around the peak to show a near uniform response, a3 x 3 max filter is applied to the gradient map G (u, v) to achieve a smoothing effect, called post filtering. Since noise may still be generated in the gradient map due to the logarithm operation performed before, the same filtering is performed on L (u, v) to further highlight the peak, which is called pre-filtering. The gradient map peak combining pre and post filtering is prominent and less noisy. And the frequency components corresponding to the periodic structure texture in the spectral gradient map of the defect image and the spectral gradient map of the defect-free image both show larger gradient values, while the gradient values of the remaining frequency components are close to 0.

As can be seen from fig. 4, the spectral gradient maps of the 2 images are visually similar, and the frequency components corresponding to the periodic background texture show larger gradient values, while the remaining components are small values close to 0. By setting the fourier components of those low gradient values to 0, periodic background texture information can be extracted and eliminated. FIG. 5 is a histogram of superimposed gradients after bi-directional filtering, the histogram of gradients for 2 images each including two peaks, where the higher peak corresponds to a frequency component with a low gradient value; the lower peaks correspond to the frequency components of the periodic structure texture correlation,the segmentation threshold T between two peaks can be estimated by using the OTSU threshold segmentation method_g。

The Boolean mask M (u, v) is commonly used for judging the condition of array elements, the Boolean mask is used for screening the elements in the spectrum gradient diagram, and the establishment rule is as follows:

wherein is t_gThe scaling constant, which can be empirically determined, has a value of 0.8. To ensure that the prominent peaks in the spectrum are all covered by the mask, morphological operations dilation based on square structuring elements are applied to the boolean mask (as shown in fig. 6). A square region is added to the center of the mask (e.g., fig. 6(a) -6 (d), with a rectangular region at the center of each figure) because its region corresponds to the DC term in the spectrum. Since the information of the defect region is mostly concentrated in the low frequency region near the DC term (refer to fig. 3), but the morphological operation will cause the loss of the frequency component of the region, a small square mask of 3 × 3 pixels is intentionally added to reduce the missing of the defect information.

After the boolean mask M (u, v) is determined, the spectrum of the aperiodic pattern frequency component is calculated according to the following rule:

and performing two-dimensional Inverse Discrete Fourier Transform (IDFT) on the defect image to reconstruct the defect image, wherein the formula is as follows:

as shown in fig. 7, fig. 7(a) - (b) are a defect-free reconstructed image and its masked spectrogram, fig. 7(d) - (e) are a defect reconstructed image and its masked spectrogram, and fig. 7(c) and (f) are difference images after fourier reconstruction. The frequency spectrum after the mask has higher coincidence degree because the background textures of the two images are the same; the two difference images have higher integrity, and the removal effect of the periodic structure texture is proved to be better; most of background textures in the defect-free image sample are removed, and only a relatively complete defect area is left in the defect image sample and contains most of defect information, which shows that the periodic structure textures of the MEMS acoustic film image can be effectively removed.

Thirdly, image reconstruction binarization:

in order to separate the defect information from the reconstructed image, a binarization operation needs to be performed thereon. Because the gray value distribution of the defect area in the image is uniform and the difference between the gray value distribution of the defect area and the gray value of the background part is large, the following method can be adopted for threshold segmentation:

wherein mu_dIs the mean, σ, of the image_dIs the variance of the image. t is t_dTo control the constants, mainly for scaling the variance, t is usually taken_d7. Through the binarization operation, the defect part in the denoised image can be segmented and output relatively completely. Fig. 9 is a graph showing the binarization result.

Fourthly, experimental result display:

scaling constant t_gMainly used for limiting the segmentation threshold T obtained by the OTSU algorithm_gIts value will affect the generation of the boolean mask and thus the removal of the background texture. FIG. 10 shows t_gThe reconstruction effect of different defect images is 0.5, 0.7, 1.0 and 1.5. When t is_gWhen the value of (a) is too large, unremoved periodic structure textures remain in the defect reconstructed image, because a part of high-frequency components corresponding to background textures still exist in the Fourier spectrum and are not removed; when t is_gWhen the value of (a) is too small, a part of frequency components of defect information may be deleted, resulting in reduction of defect area information and occurrence of noise, affecting reconstruction effect, and thus t may be set to be smaller_gThe value range of (A) is defined in the range of 0.7-1.0.

In order to verify the detection effect of the invention on MEMS acoustic films with different periodic structure textures, a defect image of 2 types of MEMS is selected and introduced into the invention for detection, and a reconstructed image and a binarization result are shown in FIGS. 11 and 12. The algorithm has good removal effect on 2 background textures, and the defect area is completely reserved; the algorithm is insensitive to the type and the size of the defect, and the banding, the plaque-shaped defect and the tiny defect in the image can be accurately segmented; when the imaging quality is higher, the detection effect of the image defects is better.

As can be seen in fig. 12(b1), the slight "virtual focus" phenomenon occurred during image acquisition resulted in poor image quality of the image, which is also a problem in particular industrial detection and can be regarded as noise adulterated in the image. The detection result shows that the defect area can still be accurately segmented, and the method has certain anti-noise performance. However, the reconstructed image output by the invention has ripple-like noise around a large defect, and it can be seen from the spectrum in the foregoing that the masking operation of the spectrum causes the frequency components of the partial region to be missing, thereby affecting the inverse fourier transform reconstruction of the image. However, after wavelet denoising and binarization processing, the noises are also eliminated, and the final image binarization result is not influenced.

In conclusion, the invention provides an unsupervised MEMS acoustic film defect detection algorithm based on frequency domain transformation, which is characterized in that periodic structure textures on the surface of an acoustic film can be effectively removed and a defect-free reference image is not relied on. Aiming at the input defect image, the invention eliminates the frequency component corresponding to the larger gradient value through the Boolean mask after expansion and reconstructs the defect image through inverse Fourier transform. And then setting a segmentation threshold value for the inverse Fourier transform reconstructed image and outputting a binary defect image. The scaling constant t is discussed and summarized in the experimental part_gThe value range of (A) is 0.7-1.0. The detection results of the MEMS acoustic film defect images of different models show that the defect detection method has the detection effect on defects of different sizes and types, and the statistical result shows that the defect detection rate of the defect detection method reaches over 95 percent.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (7)

1. A MEMS acoustic film surface defect detection method based on frequency domain transformation is characterized by comprising the following steps:

s1, calculating a Fourier spectrogram and a frequency spectrum gradient chart of the MEMS acoustic film surface defect image through two-dimensional discrete Fourier transform, and performing bidirectional maximum filtering with the size of 3X 3 to highlight a peak value and inhibit noise;

s2, calculating a segmentation threshold Tg between 2 peak values in the superimposed gradient histogram by a maximum inter-class variance method, establishing a Boolean mask and carrying out morphological operation on the Boolean mask to ensure that the peak values highlighted in the frequency spectrum are all covered by the mask;

and S3, eliminating the main frequency component corresponding to the periodic structure identified by gradient calculation in the frequency domain according to the Boolean mask, performing two-dimensional inverse discrete Fourier transform on the residual frequency spectrum to obtain a reconstructed image containing the defect region, and finally performing binarization segmentation by using a threshold method to output a binarized image containing the defect information.

2. The method for detecting the surface defects of the MEMS acoustic membrane based on the frequency domain transformation as recited in claim 1, wherein the method for calculating the Fourier spectrogram in the step S1 comprises:

the DFT result of the image is represented by F (u, v), and the algorithm is as follows:

the amplitude spectrum of the spectrum is defined as:

A(u,v)＝|F(u,v)|

the numerical range is reduced by adopting a logarithmic spectrum method:

L(u,v)＝log[A(u,v)+1]。

3. the method for detecting the surface defect of the MEMS acoustic film based on the frequency domain transformation as claimed in claim 1, wherein the method for calculating the spectrum gradient map in the step S1 comprises:

g_uand g_vRespectively representing gradient values at the coordinate points, calculating a gradient map according to the gradient values at each coordinate point:

4. the method for detecting the surface defect of the MEMS acoustic thin film based on the frequency domain transformation as claimed in claim 1, wherein the rule of the boolean mask in step S2 is:

wherein t is_gIs a scaling constant.

5. The method for detecting the surface defect of the MEMS acoustic film based on the frequency domain transformation as claimed in claim 1, wherein the morphological operation in the step S2 includes:

and expanding based on the morphological operation of the square structural elements, and adding a square area in the center of the mask, wherein the square area corresponds to a DC (direct current) item in a frequency spectrum, so that the defect information is reduced.

6. The method for detecting the surface defect of the MEMS acoustic film based on the frequency domain transformation as claimed in claim 1, wherein the method for performing the two-dimensional inverse discrete Fourier transform in step S3 comprises:

calculating the frequency spectrum of the aperiodic pattern frequency component:

and performing two-dimensional inverse discrete Fourier transform on the defect image to reconstruct the defect image, wherein the formula is as follows:

7. the method for detecting the surface defects of the MEMS acoustic film based on the frequency domain transformation as claimed in claim 1, wherein the threshold method in step S3 comprises:

the following method is adopted for threshold segmentation:

wherein mu_dIs the mean, σ, of the image_dIs the variance of the image, t_dAre control constants.

Images