CN114136975A

CN114136975A - Intelligent detection system and method for surface defects of microwave bare chip

Info

Publication number: CN114136975A
Application number: CN202111298816.3A
Authority: CN
Inventors: 季思蔚; 圣冬冬; 江芸; 王海涛; 杜林�; 王昆黍; 刘港港
Original assignee: SHANGHAI PRECISION METROLOGY AND TEST RESEARCH INSTITUTE
Current assignee: SHANGHAI PRECISION METROLOGY AND TEST RESEARCH INSTITUTE
Priority date: 2021-11-04
Filing date: 2021-11-04
Publication date: 2022-03-04

Abstract

The invention provides an intelligent detection system for surface defects of a microwave bare chip, which comprises a camera, a light source, an industrial personal computer, a motion platform and a platform driver, wherein the camera is used for detecting the surface defects of the microwave bare chip; the bare chip is fixed in the plane of the motion platform, and the motion platform is driven to move by the platform driver, so that the camera acquires the surface image of the chip; the camera is connected with the industrial personal computer, and transmits the acquired chip surface image to the industrial personal computer for processing; the industrial personal computer is connected with the platform driver and is used for setting parameters of the camera and the platform driver. The invention effectively avoids the condition of missing detection, greatly improves the defect identification and detection efficiency and reduces the labor intensity of workers.

Description

Intelligent detection system and method for surface defects of microwave bare chip

Technical Field

The invention belongs to the technical field of chip detection, and particularly relates to a microwave bare chip surface defect intelligent detection system and method.

Background

The microwave has the characteristics of short wavelength, wide frequency band, strong penetrating power and the like, plays an important role in the current microelectronic technology, becomes a great factor for restricting the development of high-end technology and promotes the development of electronic technology. The application of microwave technology is derived from high reliability application, but with the development of information technology, civil and commercial microwave technology is more and more emphasized by people. The rapid development of science and technology and economy at present not only continuously raises the requirements for the functions and performance of the microwave system, but also has higher and higher requirements for the reliability of the microwave system.

With the development of electronic packaging technology, the use of a large number of bare chips makes it an important subject to provide high-quality bare chips, especially in the fields of military industry, space technology, etc. with high requirements for chip quality grade. The quality and reliability guarantee technology of the bare chip has important research significance because the bare chip is not packaged and is produced well and the quality and reliability guarantee technology of the bare chip is not protected and electrically connected. The assurance of good chips prior to fabrication of electronic systems and assembly of electronic components can minimize waste, optimize materials, provide a base source of chips, and reduce costs. The packaging test of high performance devices nowadays takes up an increasing proportion of the cost of the device, and therefore it is important to sort out the failed or potentially defective chips before use, so that manufacturers can better control the cost of the package. These factors have led to the development and maturation of testing and aging techniques for bare chips, which have passed quality and reliability certification as known good chips.

The surface defect detection technology is used for screening out defective microwave bare chips and ensuring that the quality and reliability of the finally selected microwave bare chips meet the quality and reliability grade requirements of packaged finished products. Although many studies at present realize preliminary chip automatic defect detection, most of the studies are used by a traditional computational vision method, and mainly aiming at packaged chips, the realized functions can only be suitable for single-type chip defect detection tasks and are difficult to realize defect positioning, and the classification tasks of aerospace microwave chips with various types lack a learning and summarizing process and are difficult to meet the requirements. Particularly for microwave bare chips, at present, related researches are few, a user unit with high reliability requirement performs quality control on the microwave bare chips, related data are firstly monitored and reviewed, different from packaged devices, the bare chips cannot be aged and screened, an artificial intelligent detection system capable of automatically identifying defects is urgently needed to be established, and the method has important significance for breaking high-end equipment and technical monopoly of foreign semiconductor package testing and improving the independent research and development capability of China in semiconductor chip package testing equipment.

Disclosure of Invention

In view of the above disadvantages of the prior art, an object of the present invention is to provide an intelligent detection system for surface defects of a bare microwave chip, which solves the technical problems in the prior art, such as high labor intensity of manual machines, low detection efficiency, low yield, non-uniform judgment standards of the existing detection equipment, and easy omission and false detection.

The technical scheme adopted by the invention is as follows:

the invention provides an intelligent detection system for surface defects of a microwave bare chip, which comprises a camera, an industrial personal computer, a motion platform, a platform driver and a light source, wherein the camera is connected with the industrial personal computer;

the industrial personal computer drives the motion platform to move and rotate through the platform driver, so that the camera acquires the surface image of the bare chip; the motion platform moves back and forth and left and right through the guide rail; the moving platform comprises an object stage, a platform holding unit and a rotary driving unit, the rotary driving unit is positioned above the guide rail and is embedded in the platform holding unit, the object stage is arranged above the platform holding unit and comprises a platform plate and a supporting plate, the platform plate is arranged above the supporting plate and is used for placing a bare chip, and the rotary driving unit drives the platform holding unit and the object stage to rotate through rotation;

the camera amplifies the collected surface image of the bare chip through a lens and sends the amplified surface image to an industrial personal computer for processing;

the industrial personal computer controls the camera, the platform driver and the light source through the controller;

the light source is positioned above the motion platform and used for illuminating the bare chip and comprises a first light projection unit, a second light projection unit and a stripe light source; the first light projection unit and the second light projection unit have the same structure and are symmetrically distributed on two sides of the camera and the lens, and the stripe light source is positioned between the first light projection unit and the second light projection unit;

the first light projection unit and the second light projection unit respectively comprise a projection light source, a condensing lens, a pattern generation unit and a light projection lens; the light emitted from the projection light source enters the pattern generation unit through the condensing lens, and then the measurement light emitted from the pattern generation unit is irradiated on the bare chip on the objective table through the light projection lens;

the fringe light source is used for emitting uniform illuminating light of visible light to the bare chip on the objective table and detecting surface texture information of the bare chip, and the surface texture information comprises colors or patterns; the striped light sources are distributed around the camera and lens, and the light projection axis of the striped light sources is parallel to the axis of the camera and lens.

Furthermore, the industrial personal computer is connected with the camera through a network cable, the bare chip image collected by the camera is processed, the bare chip image is identified through the detection model, whether the bare chip has defects or not is judged, and when the bare chip has defects, the computer alarms through the connected alarm.

Further, the bedplate comprises a rotating mechanism, a mother board and a daughter board; a plurality of daughter boards are arranged on the mother board; the rotating mechanism is connected with the motherboard; each daughter board is used for placing a bare chip; the rotating mechanism is used for acquiring a control request and driving the mother board to rotate or stop according to the control request, so that the bare chip on each daughter board can reach the optimal position for the camera to acquire images.

The invention also provides an intelligent detection device for the surface defects of the microwave bare chip, which comprises an intelligent detection system and a shell, wherein the intelligent detection system for the surface defects of the microwave bare chip is arranged in the shell;

the camera, the first light projecting unit and the second light projecting unit are arranged in the shell through the lifting support; the camera moves up and down through the lifting support, and the distance between the camera and the bare chip is adjusted; the first light projecting unit and the second light projecting unit are both mounted on the lifting support through connecting rods, universal sleeves and universal ball heads are arranged between the first light projecting unit and the connecting rods and between the second light projecting unit and the connecting rods, the universal ball heads are arranged in the universal sleeves, and the angles of the projection light sources irradiating the bare chips are adjusted through the universal ball heads.

Further, the intelligent detection device for the surface defects of the bare chip further comprises a manipulator used for grabbing the bare chip.

The invention also provides an intelligent detection method for the surface defects of the microwave bare chip, which comprises the following steps:

step S1, selecting a qualified microwave bare chip as a sample bare chip to be fixed in a bedplate, driving the bedplate to move and rotate by a motion platform through a guide rail and a rotation driving unit, enabling each part of the sample bare chip to sequentially pass through a camera field fixed above and take a picture, and completing the acquisition of the sample bare chip image;

step S2, splicing all parts of images of the sample bare chip collected by the camera into a complete full-width standard template image;

step S3, when detecting the microwave bare chip to be detected, sequentially collecting images of each part of the microwave bare chip to be detected, matching a corresponding area in the full-width standard template image as a sub-template for each local image, carrying out image registration on the local image to be detected and the sub-template, and then differentiating to extract suspected defects;

and step S4, performing primary defect classification on the suspected defects, separating true defects and false defects, and performing secondary defect classification on the true defects and the false defects.

Further, the step S1 further includes performing image preprocessing on the acquired sample bare chip image, where the image preprocessing employs median filtering to remove image noise, and specifically includes the following steps:

step S11, selecting a filtering template to scan the target pixel point and then calculating the value of the new pixel point replacing the original pixel point;

step S12, the target pixel point and the adjacent pixel points are sorted according to the gray value, the median of the gray value and the gray value of the target pixel point are selected for replacement, and the formula is as follows:

g(x，y)＝median{f(x-i，y-i)}，(i，j)∈W

wherein f (x-i, y-i) is the pixel value of one point in the image before filtering, g (x, y) is the pixel value of the point obtained after median filtering is carried out on the target image, and W is a median filter template.

Further, in the step S2, the image mosaic algorithm is used to mosaic and combine the local sample bare chip images collected in the subareas into a complete chip image, that is, a full-width standard template, and the method specifically includes the following steps:

step S21, extracting angular points in the image as registration features;

step S22, registering common corner points in the images to be spliced through a matching algorithm, and fusing the registered images to manufacture a full-breadth standard template;

and step S23, correcting splicing errors.

Further, the step S3 specifically includes the following steps:

step S31, performing threshold segmentation on the microwave bare chip image to be detected, and extracting an ROI (region of interest) in the microwave bare chip image to be detected;

step S32, using the ROI area in the microwave bare chip image to be detected as a matching template, searching a corresponding area in a full-width standard template as a sub-template, wherein the sub-template has the same size as the matching template and possibly has a difference in direction and position;

step S33, image transformation is carried out, and the sub-template is registered with the pixel coordinate of the matching template;

and step S34, in the process of area separation, the difference is made between the sub-template and the matched pixels through a difference method, and the image area of the suspected defect after the difference is extracted.

Further, in step S4, after the image area of the suspected defect is obtained, the suspected defect is classified and determined by an SVM classifier, including a first-level defect classification and a second-level defect classification, which specifically includes the following steps:

s41, a primary defect classification module: in the first-stage defect classification process, samples are divided into two types, and first-stage sample labels are set, wherein one type of label is a false defect, and the other type of label is a true defect;

s42, secondary defect classification module: in the secondary defect classification process, classifying the samples into 6 classes, setting secondary sample labels, including metal loss, scratching, lead bending, lead pollution, insufficient soldering and qualification, obtaining characteristic parameters of the 6 classes of secondary sample labels, and inputting the parameters into a secondary SVM defect classifier for training; and the defect classifier after training can detect the defects of the microwave bare chip.

Compared with the prior art, the invention has the following beneficial effects: the condition of missing detection is effectively avoided, the defect identification and detection efficiency is greatly improved, and the labor intensity of workers is reduced.

Drawings

FIG. 1 is a block diagram of an intelligent detection system for surface defects of a microwave bare chip according to the present invention.

FIG. 2 is a structural diagram of the intelligent detection system for surface defects of a microwave bare chip according to the present invention.

Fig. 3 is a power supply circuit of the controller of the present invention.

Fig. 4 is a schematic top view of one embodiment of a platen of the present invention.

FIG. 5 is a flow chart of the intelligent extraction and classification of surface defects of a bare chip according to the present invention.

FIG. 6 is a schematic diagram of suspected defect extraction according to the present invention.

FIG. 7 is a flowchart illustrating the process of creating a full-width template image according to the present invention.

Fig. 8 is a schematic diagram of template matching.

FIG. 9 is a diagram of a suspected defect classification module according to the present invention.

FIG. 10 is a schematic diagram of decision tree classification according to the present invention.

Detailed Description

The following description of the embodiments of the present invention will be further described with reference to the accompanying drawings and specific examples.

The first embodiment is as follows:

the invention provides an intelligent detection system for surface defects of a microwave bare chip, which is shown in figure 1. To ensure that images of each portion of the die 10 can be captured by the camera 20, it is necessary to have a relative displacement between the camera 20 and the die 10. Compared with the case of taking pictures by moving the position of the camera 20, it is more convenient to complete the whole acquisition of the images of all parts of the chip by moving the bare chip 10. Therefore, by installing a motion platform 70, the bare chip is fixed in the plane of the motion platform 70 to drive the movement of the bare chip, so that the camera 20 can acquire images of each area of the bare chip. In the process of acquiring a bare chip image, the camera 20 and the motion platform 70 are indispensable hardware devices, and in order to ensure the quality of an image acquired by the camera, a light source 30 needs to be provided for illumination, the motion of the motion platform needs the platform driver 60 to provide power, and finally, the camera 20 and the platform driver 60 need to set parameters through the industrial personal computer 40. Accordingly, the bare chip image acquiring apparatus includes a camera 20, a light source 30, a motion stage 70, a stage driver 60, and an industrial personal computer 40.

The light source 30 provides the illumination condition required for image acquisition, the driver controls the motion platform 70 to move in two directions, the bare chip 10 is placed on the motion platform 60, the camera 20 acquires the surface image of the bare chip, and then the chip image is sent to the industrial personal computer 40, so that the process of extracting and classifying suspected defects is completed.

Preferably, the industrial personal computer 40 drives the motion platform 70 to move and rotate through the platform driver 60, so that the camera 20 acquires the surface image of the bare chip 10; the moving platform 70 moves back and forth and left and right through the guide rails 80, and the guide rails 80 include a front and rear moving guide rail and a left and right moving guide rail. The moving platform 70 includes a stage 710, a platform holding unit 720, and a rotation driving unit 730, as shown in fig. 2. The rotary driving unit 730 is disposed above the guide rail 80, the rotary driving unit 730 is nested inside the stage holding unit 720, the stage 710 is disposed above the stage holding unit 720, the stage includes a platen 7110 and a support plate 7120, the platen 7110 is disposed above the support plate 7120, the platen 7110 has a disk shape and is used for placing the bare chip 10, and the rotary driving unit 730 rotates the stage holding unit 720 and the stage 710, thereby changing the angle of the camera 20 shooting the bare chip 10.

The camera 20 amplifies the collected bare chip surface image through the lens 201, and sends the amplified image to the industrial personal computer 40 for processing. The camera 20 employs a CCD camera or a CMOS camera. The resolution of the camera in this embodiment of the invention requires at least 2200 by 500 pixels. According to the comprehensive consideration of factors such as detection precision, cost performance of a camera, test results of different cameras and the like, the German Basler Ace ACA2500-14gm industrial camera is preferably adopted, the resolution of the camera is adjusted to 2256 x 1500 pixels, and therefore the subsequent processing speed of images can be greatly improved while the detection precision of bare chips is met.

In a defect detection system, the suitability of lens selection directly affects the overall performance of the system. In general, parameters such as an imaging plane, a working distance, a field of view, and a depth of field need to be considered when selecting a lens. The lens is an optical focalizer located in front of the camera, and the lens mainly has the function of enabling the camera to flexibly change the view angle, brightness and the like of a picture by adjusting the focal length and the aperture of the lens, and is an important element for image acquisition. The model of the lens must be determined according to the detected working distance.

In a specific embodiment, the distance between the camera and the bare chip is approximately 60mm, and a lens of Japanese computer PM-1614M is preferably used together with the camera, and the lens is a zoom lens which can realize the focal length conversion within the range of 30mm-46 mm. The camera and the lens are reasonably matched to acquire images of the bare chip.

The industrial personal computer 40 controls the camera 20, the platform driver 60, and the light source 30 through the controller 50.

The light source 30 is located above the moving platform 70, and is used for illuminating the bare chip 10, and includes a first light projecting unit 310, a second light projecting unit 320, and a stripe light source 330; the first light projection unit 310 and the second light projection unit 320 have the same structure, and are symmetrically distributed on both sides of the camera 20 and the lens 201, and the stripe light source 330 is located between the first light projection unit 310 and the second light projection unit 320.

Each of the first and second light projecting units 310 and 320 includes a projection light source 3110, a condensing lens 3120, a pattern generating unit 3130, and a light projecting lens 3140. Light emitted from the projection light source 3110 enters the pattern generation unit 3130 through the condenser lens 3120, and then measurement light emitted from the pattern generation unit 3130 is irradiated onto a die on the stage 710 through the light projection lens 3140.

The projection light source 3110 is an artificial light source including a fluorescent lamp, an incandescent lamp, a halogen tungsten lamp, a metal halide lamp, a Light Emitting Diode (LED), a laser lamp, and the like. The influence of external environment light on the detection process can be greatly eliminated by adopting an artificial light source. The light source effect directly determines the imaging quality of the workpiece. Preferably, a Light Emitting Diode (LED) or a halogen lamp that generates light of one color is used. The monochromatic projection light source 3110 is more advantageous than the white light source in terms of ease of chromatic aberration correction. In addition, shorter wavelengths are advantageous to improve the resolution of the shape measurement, and therefore, a blue light source (e.g., a blue LED) is used as the projection light source 3110 in a specific embodiment of the invention. By adopting an imaging mode of backlight irradiation of an LED light source, the definition and the resolution of projection imaging can be greatly improved, and the imaging distortion phenomenon of the sensitive grid of the strain gauge caused by mechanical vibration is effectively reduced.

The pattern generation unit 3130 is a device for generating pattern light for structured illumination, and is capable of switching between uniform measurement light and measurement light of a two-dimensional pattern. For example, a Digital Micromirror Device (DMD) or a liquid crystal panel is used for the pattern generating unit 3130. The DMD is a display element having a large number of very small mirrors arranged two-dimensionally and capable of switching between a bright state and a dark state on a per-pixel basis by controlling the inclination of each mirror.

The structured illumination method for measuring the three-dimensional shape of the bare chip using the triangulation principle may be a sinusoidal phase shift method, a multi-slit method, a spatial encoding method, or the like.

The sinusoidal phase shift method is an illumination method that projects a sinusoidal fringe pattern onto a die and acquires a captured image each time the fringe pattern is moved at a pitch smaller than a sinusoidal cycle. The three-dimensional shape data is obtained by determining a phase value of each pixel from a luminance value of each captured image and converting the value into height information.

The multi-slit method is an illumination method in which a fine stripe pattern is projected onto a die and a captured image is acquired every time the stripe pattern moves at a pitch smaller than a gap between stripes. The three-dimensional shape data is obtained by determining the capturing timing of the maximum luminance of each pixel from the luminance value of each captured image and converting the timing into height information.

The spatial coding method is an illumination method in which a plurality of stripe patterns having different stripe widths and a black/white duty ratio of 50% are sequentially projected onto a die to obtain a captured image. The three-dimensional shape data is obtained by determining a code value of each pixel from a luminance value of each captured image and converting the value into height information.

The pattern generating unit 3130 may generate the above stripe pattern into a two-dimensional pattern, and acquire high-resolution three-dimensional shape data with high accuracy using a multi-slit method and a spatial encoding method in combination.

The stripe light source 330 emits a uniform illumination light of visible light to the bare chip 10 on the stage 710 for detecting surface texture information, including color or pattern information, of the bare chip 10. The striped light sources 330 are distributed around the camera and the lens in an amount of between 2 and 12, and the light projection axis of the striped light sources 330 is substantially parallel to the axis of the camera and the lens. Therefore, a shadow is less likely to be formed on the die than illumination of the projection light source 3110, and a dead angle at the time of photographing is made smaller. The fringe light sources 330 are distributed around the camera and lens in a ring shape, and with the use of LED lamps, uniform diffuse illumination can be achieved across the entire bare chip surface.

The controller 50 includes a control circuit of the stage driver 60, a driving circuit of driving the projection light source 30 and the camera 20, and a processing circuit of processing an image. The controller 50 receives the light signal from the camera 20. The controller also includes a power circuit, as shown in fig. 3, for supplying power to the controller. The power circuit adopts a TPS54310 chip, the TPS54310 is a switch power supply adjusting chip produced by Texas instruments, and the TPS54310 can realize low-voltage input and high-current output (the input voltage range is 3V-6V, the output voltage can be adjusted between 0.9V-3.3V according to requirements, and the output current is 3A). The voltage error amplifier is arranged inside the circuit, so that the working performance under the transient response condition can be improved. The slow start mode may be provided from the inside or the outside, respectively. Its good voltage output characteristics can be used for processor/logic reset, fault signal detection and continuous power supply.

In the power supply circuit, one ends of decoupling capacitors C1 and C3 are grounded in parallel, and the other ends of the decoupling capacitors are connected with a VIN port of a TPS54310 chip; one end of the resistor R1 is connected with the RT port of the TPS54310 chip, and the other end is grounded and used for setting the switching frequency of the module; one end of an inductor L1 is connected with a PH port of the TPS54310 chip, and the other end of the inductor L1 is connected with a capacitor C2 and then grounded and used for filtering output voltage; one end of the capacitor C4 is connected with a VBIAS port of the TPS54310 chip, and the other end of the capacitor C4 is grounded; one end of the capacitor C6 is connected with an SS/ENA port of the TPS54310 chip, and the other end is grounded; the capacitor C5 is connected between the PH port and the BOOT port of the TPS54310 chip; resistors R2, R3, R5, capacitors C7, C8 and C9 form a loop compensation circuit, wherein a capacitor C8 is connected in parallel with the resistor R5 and the capacitor C9 and is connected between a VSENSE port and a COMP port of a TPS54310 chip, and a resistor R2 is connected in parallel with the capacitor C7 and the resistor R3 and is connected between the VSENSE port and a PH port of the TPS54310 chip; the resistor R4 and the resistor R2 are used as voltage dividing resistors to control the output voltage of the power supply module, and the voltage dividing resistors are specifically as follows:

wherein V_outIs the output voltage.

The industrial personal computer 40 is used for controlling the detection system and performing surface defect processing and identification on the bare chip, and comprises a display 410, a keyboard 420 and a mouse 430. The display 410 is a monitor device for displaying an image of the defect detection result and setting information on a screen. The keyboard 420 and the mouse 430 are input devices used by a user to make operation inputs. The industrial personal computer 40 may be a personal computer and is connected to the controller 50.

Preferably, the inventive platen 7110 includes a rotating mechanism 7111, a master plate 7112, and a daughter plate 7113, as shown in fig. 4. Wherein: a plurality of sub-boards 7113 are arranged on the mother board 7112; the rotating mechanism 7111 is connected with the motherboard 7112; each daughter board 7113 is used for placing the bare chip 10; the rotating mechanism 7111 is used for acquiring a control request and driving the motherboard 7112 to rotate or stop according to the control request, so that the bare chip on each daughter board can reach the optimal position for the camera to acquire images. The rotating mechanism can be in various implementation forms, and can be used for acquiring a control request and driving the motherboard to rotate or stop.

The mother board is driven by the rotating mechanism, so that a plurality of daughter boards can be arranged on the mother board, the number of the daughter boards is increased, the number of the bare chips to be tested is increased, the frequency of replacing the devices to be tested is reduced, and the problem of complex operation caused by frequent replacement of the devices to be tested in the prior art is solved.

Preferably, the industrial personal computer is connected with the camera through a network cable, the bare chip image collected by the camera is processed, the bare chip image is identified through the detection model, whether the bare chip has defects or not is judged, and when the bare chip has defects, the computer alarms through the connected alarm.

Example two:

the invention also provides an intelligent detection device for the surface defects of the bare chip, which comprises an intelligent microwave detection system for the surface defects of the bare chip and a shell, wherein the intelligent microwave detection system for the surface defects of the bare chip is arranged in the shell;

Preferably, the intelligent detection device for the surface defects of the bare chip further comprises a manipulator, wherein the manipulator is used for grabbing the bare chip and can automatically carry the bare chip.

Preferably, the manipulator is driven by a linear motor, a plurality of vacuum suction nozzles are arranged on the manipulator, the vacuum suction nozzles are communicated with an external vacuum generator, the bare chips are sucked by the vacuum suction nozzles, and the manipulator has the characteristics of high-precision positioning, taking and placing and high-speed movement.

Example three:

the invention also provides an intelligent detection method for the surface defects of the bare chip, which is shown in figure 5. Firstly, a qualified full-width image is required to be made as a standard template. A qualified chip sample is placed in a motion platform in the manufacturing process, a translation platform moving path is reasonably designed, all parts of the chip can sequentially pass through the camera field fixed above and take pictures, and images of all parts of the sample wafer collected by the camera are spliced into a complete full-breadth standard template image. When a chip sample to be detected is detected, image acquisition is carried out on all parts of the sample to be detected in sequence in the same mode, for each local image, a corresponding area is matched in a full-width standard template to be used as a sub-template, the local image to be detected and the sub-template are subjected to image registration and then difference, suspected defects are extracted to carry out primary defect classification, and the separated true defects and false defects are subjected to secondary defect classification.

1. Image acquisition

In the microwave bare chip image acquisition process, noise exists in the chip image due to the interference of the external environment. The noise can cause redundant information with higher proportion in the chip image, thereby improving the difficulty of the processes of full-width template manufacturing, suspected defect extraction and suspected defect classification. In order to reduce the calculation amount in the subsequent image processing process, the microwave bare chip image is subjected to median filtering in the specific embodiment of the invention, so that the fuzzy details in the image can be eliminated, and the effect on noise caused by pulse interference is obvious. The operation process does not need to count the characteristics of the image, so the calculation is convenient.

The median filtering is to select a filtering template to scan a target pixel point, then calculate the value of the new pixel point for replacing the original pixel point, sort the target pixel point and the adjacent pixel points according to the gray value, and select the median of the gray values and the gray value of the target pixel point for replacement. Because the noise point and the edge point in the image belong to the point with the sudden change of the gray value in the image pixel point, the numerical difference is larger compared with the adjacent pixel point. However, the edge points in the image are not the extreme values of the adjacent pixels, and most of the noise points in the image are the extreme values of the adjacent pixel points. Accordingly, the noise removing method which can be adopted according to the characteristic of the noise pixel point is a median filtering method. The common templates for median filtering have linear, square and cross shapes, and the size and shape of the template need to be adjusted continuously in the practical application process to obtain the optimal effect. The mathematical expression of the median filtering algorithm is as follows:

g(x,y)＝median(f(x-i,y-i)},(i,j)∈W

The median filtering can not only remove noise in the realizable image, but also can more completely keep detail information in the image. The present invention prefers in particular embodiments a modified adaptive weighted median filtering algorithm (WAMF). The technique of dynamically changing window filtering and center weighted median filtering is utilized. Firstly, moving a 3 x 3 window on an image to detect noise, and dividing pixel points into two types of noise points and non-noise points; then, the size of a filtering window is adjusted in a self-adaptive mode according to the number of the noise points, all pixel points are reasonably grouped in a self-adaptive mode according to a certain rule based on similarity, and each group of pixel points are endowed with weighted values; and finally, performing weighted median filtering on the noise points in the image. The algorithm can solve the contradiction between noise suppression and detail retention, and effectively improves the self-adaptive image processing and detail retention capacity by adaptively adjusting the filtering window and the grouping of each pixel point and giving corresponding weight to each group of pixel points.

The algorithm is divided into three steps: firstly, carrying out noise detection on pixel points of an image window; secondly, adaptively adjusting the size of a filtering window according to the number of noise pixel points of the image window; and finally, determining the weight value of each pixel point in the filtering window, and removing noise by using a weighted median filtering algorithm.

(1) Image noise detection

Noise processing is performed in a 3 × 3 square image window, and if the gray value of the pixel center point (i, j) is f (i, j), the gray value set of all the pixel points in the current window is: s_i，jCalculating the average of all pixels in the window, i (i + k, j + r) | k, r ═ 1, 0, 1 }:

suppose S_i，jWherein the minimum gray value is Z_minThe maximum gray value is Z_maxThen, the noise pixel point can be determined by the following conditions: if the pixel value of the center point is f (i, j) ═ Z_maxOr f (i, j) ═ Z_minAnd | f (i, j) -Average (S)_i,j)|＞d_i，jIf so, judging that (i, j) is a noise pixel point, marking N (i, j) as 1, and marking a non-noise pixel point as N (i, j) as 0. Human visual effect defines the noise detection threshold as:

(2) determination of filter window size

The algorithm self-adaptively determines the size of a filtering window according to the number of noise pixel points of the current image window. Assuming that the number of noise pixels to be found in a 3 × 3 image window, when the center pixel is a noise pixel, the number of calculations is:

the size L of the filter window_i，jIs adaptively determined as Num (S)_i,j) Dependent on 3X 3 windowsThe number of noisy pixels. The calculation formula is as follows:

(3) noise pixel filtering

After the size of a filtering window and noise pixel points are determined, image pixel points are divided into two types, namely noise pixel points and non-noise pixel points, wherein the non-noise pixel points keep the original gray value, and the noise pixel points adopt a new weighted median filtering algorithm to process noise. Generally, in an adjacent region, if there is a certain correlation between the central pixel and its surrounding pixels, then the similarity between the gray value f (i + k, j + r) of the specific pixel (i + k, j + r) in the filtering window and the gray value (i, j) of the central pixel is calculated

Wherein

Is a similar function, | f (i + k, j + r) -f (i, j) | is an independent variable, and the following condition needs to be satisfied:

1)

in the interval [0, ∞]Must be a monotonically decreasing function;

2) the similarity function is

Get

3) The value of the similarity is in the range of [0, 1 ]. If the gray value of the specific pixel point in the filtering window is closer to the gray value of the pixel at the central point, the similarity is higher, and otherwise, the similarity is lower.

And calculating similarity values of all pixel points in the filtering window by utilizing a similarity function, sequencing the similarity values from small to large, grouping the pixel points according to the similarity values, giving corresponding weight values, and processing noise by adopting a weighted median filtering algorithm.

Assuming a noise point pixel gray scale value of f (i, j), the filter window size is L_i,jAfter (2n +1) × (2n +1), n ∈ {1, 2, 3}, the adaptive operation is performed, and the filtering process is as follows:

1) calculating the similarity value of each pixel point (i + k, j + r) in the filtering window:

2) sorting similarity values (2n +1) × (2n +1) of all pixel points (i + k, j + r) in a filtering window from small to large, and then grouping into 2n +1 pixels. Each group has u pixels (u is 2n +2), and the serial number of each group pixel is (w-1)_u+1To w_u(w is 1, 2, …, 2n), and when the group number w is 2n +1, there is only one pixel (i.e. the point of maximum similarity). The gray value f (i + k, j + r) of the pixel point (i + k, j + r) in the group number w can be assigned with a corresponding weight value w (w is 1, 2, …, 2n + 1).

3) Carrying out weighted median filtering on the pixel (i, j) at the center point of the filtering window, wherein the gray value after noise point filtering is as follows:

g(i，j)＝weighted_Med{f(i-n，j-n)，f(i-n，j-n+1)，…，f(i+n，j+n)}。

2. suspected defect extraction

The suspected defect extraction is used for extracting a suspected defect area in the chip, and comprises a full-width standard template manufacturing module and a suspected defect separating module, as shown in fig. 6.

(1) Full-width standard template manufacturing

In the manufacture of the full-width surface mark template, the specific task is to splice and combine the local chip images collected in the subareas into a complete chip image through an image splicing algorithm. The image splicing object is only a qualified and defect-free chip, and a qualified complete chip image is also called a full-width standard template. The stitching process mainly includes corner extraction, image registration and fusion, as shown in fig. 7. Firstly, extracting angular points in the images as registration features, then registering common angular points in the images to be spliced through a matching algorithm, and fusing the registered images to manufacture a full-width standard template. Splicing is completed off line, and splicing errors can be corrected in time. And fusing the obtained full-width chip image to eliminate the gap generated at the transition region of the image edge.

1) Feature corner detection

A corner point in an image refers to a local maximum curvature point on a contour line, which belongs to one of the important features in the image. In the corner detection process, firstly, the corners in the images are extracted, then the registration algorithm is utilized to pair the corners in the images to be spliced, and finally, the correct registration corners are taken as registration objects to execute a splicing program. The angular point features have rich feature information and are easy to measure and represent, the angular points are insensitive to the change of illumination, and the registration algorithm based on the angular points has higher splicing accuracy and robustness. The corner features are thus used as features in the images to be stitched.

In the embodiment of the invention, the corner points in the microwave bare chip image are detected by a Foerstner corner point extraction method, the Foerstner corner point extraction method searches points which are as small as possible and close to a circle in the image as characteristic points by designing a smooth matrix, calculates the unevenness and the isotropy of the image and then extracts the characteristic points by using a method for inhibiting local minimum values. The Forstner corner detection method effectively removes the dummy corners. When the image has obvious edge information, the Forstner corner detection method avoids multiple iterations, reduces the calculated amount and greatly retains the image edge information.

2) RANSAC image registration method

After the angular points in the images to be spliced are extracted, the angular points are used as matching objects to be matched, and then the corresponding angular points in the images to be spliced are sequentially matched through an image registration algorithm, so that the two images are combined into a relatively complete image. The registration of the corner points in the images belongs to the correlation problem, and the occurrence of false registration is avoided as much as possible while correctly registering the corner points in the two images. In the embodiment of the invention, the collected corners are registered by a random sample consensus (RANSAC) corner matching algorithm. As a parameter estimation method with higher robustness, the RANSAC corner matching algorithm can better process noise generated in the process of constructing an image pyramid and eliminate mismatching points. The RANSAC angular point matching algorithm is divided into two steps by establishing a space conversion model and matching common characteristic points in images to be spliced:

firstly, determining the gray value correlation near the angular points of two images, and determining the initial pairing relationship between common angular points by a normalized cross-correlation (NCC) method;

secondly, continuously performing an iteration process from the obtained initial matching point set, finding out an optimal parameter model, and completing final registration of the angular points.

3) Weighted average based image fusion

Due to the influence of external lighting conditions, the chip image after registration may include slight gray scale and brightness changes, which are displayed as gaps in the picture. Due to the existence of the gaps, a large amount of redundant information exists in the image, which may cause the inaccuracy of the template matching process and influence the subsequent suspected defect extraction process. Therefore, a fusion process is required to remove redundant gap information in the image, so that the final image has a good visual effect. The image fusion algorithm should ensure that the images after image splicing are visually consistent, and simultaneously, the loss of the original image information is reduced as much as possible.

(2) Separation of suspected defects

In the suspected defect separation module, the specific task is to compare the chip image to be detected with the manufactured full-width standard template image and extract a suspected defect area in the chip to be detected, wherein the chip to be detected is also in a mode of acquiring images in a subarea mode. The suspected defect separation module mainly comprises ROI (region of interest) extraction, template matching, image transformation and region separation. Firstly, extracting an ROI (region of interest) in a to-be-detected chip image through threshold segmentation; taking the ROI area in the to-be-detected chip diagram as a matching template, and searching a corresponding area in a full-width standard template, wherein the corresponding area is called as a sub-template, the sub-template and the matching template have the same size, and the direction and the position of the sub-template are possibly different; registering the sub-template and the pixel coordinates of the matched template through image transformation; and finally, in the process of area separation, the difference is made between the sub-template and the matched pixels through a difference method, and the image area with the suspected defects after the difference is extracted.

In the embodiment of the invention, a smooth histogram method is adopted to extract the matching template in the chip to be detected. Firstly, manually setting an initial threshold value, extracting a target chip region from a chip image, separating the background, then respectively solving the average gray value of the target chip region and the background region, determining a relative histogram, then performing smooth calculation through a guass filter, and taking the minimum value as the threshold value.

And searching and extracting the area corresponding to the matching template on the full-width standard template to be used as the sub-template. In the embodiment of the invention, a template matching algorithm based on gray scale is adopted to carry out a matching process. And searching a sub-template with the highest similarity in the full-width template image by using a template matching algorithm based on gray scale so as to find a correct matching area. And (3) the matching template is named as T, the T is translated on the full-width template, the matched area is named as a sub-template S, and the extraction process of the sub-template with the highest similarity is completed by calculating the gray level similarity of the template T and the template S. A schematic diagram of the grayscale-based template matching algorithm is shown in fig. 8.

Although the sub-template searched from the full-width standard template and the matching template have the same size, the orientation and the position of the sub-template and the matching template may be different. And establishing a pixel coordinate conversion relation between the two templates to complete the calibration process by establishing an image transformation model between the matching template T and the sub-template S.

And after the matching template T corresponds to the coordinates in the sub-template image S one by one, comparing the T with the S to obtain the difference information between the two images. Before comparison, the S direction T needs to be subjected to specified treatment, and a suspected defect area is extracted through difference.

False defects exist in the suspected defect area obtained through the difference, generally, the false defects contain a large amount of noise point information, noise point interference can be eliminated through threshold segmentation preliminarily, and the false defect information in the suspected defects is reduced.

Among the suspected defects before threshold segmentation, a rectangular area is marked as a true defect area, and a circular area is marked as a false defect which is relatively difficult to distinguish. After threshold segmentation, real defects and partial pseudo defects are reserved in suspected defect areas, the areas are marked, and corresponding sample labels are made to define categories so as to define the categories of the defects appearing on the surface of the chip. The false defects and the real defects are extracted to be used as training and testing samples for subsequent defect classification. The false defects can be used as qualified samples, and other true defects are labeled according to actual defect types. In the classifier training process, the false defect cannot influence the precision of the classifier. The false defects in the training are used as samples of the qualified class for training, so that the false defects can be distinguished from the true defects after training.

3. Suspected defect classification

After the image sample of the suspected defect is obtained, the suspected defect is classified and judged through an SVM (support vector machine) classifier. The suspected defect classification includes a first-level defect classification and a second-level defect classification, as shown in fig. 9.

(1) First order defect classification

In the first-level defect detection process, samples are divided into two types and sample labels are set, wherein one type of label is a false defect, and the other type of label is a true defect. And calculating image characteristic parameters of the sample, wherein the image characteristic parameters comprise 6 types of parameters including an image mean, an image gray standard deviation sigma, a gray average gradient (GWG), a Zernike moment and a minimum circumscribed rectangle length-to-volume ratio, and are used as input vectors of a primary defect classifier. After training is carried out through the SVM first-level defect classifier, the false defects can be filtered out. And further classifying the classified true and false defects and a sample set serving as a secondary defect through an SVM classifier.

SVM is based on the theory of minimization of structural risk. The method has the main idea that a sample space is mapped into a high-dimensional space through a nonlinear mapping function, so that the problem that the original cannot be classified can be solved in the high-dimensional space. And dividing the positive and negative data sets by utilizing the hyperplanes, and obtaining an optimal solution when the distance between the two parallel hyperplanes is maximum.

The defects are described by the defined 6 characteristics from shapes and gray distribution respectively, and are selected through optimization, otherwise, the defects are difficult to construct hyperplane classification.

1) The image mean value mean is commonly used for verifying the image quality and reflects the brightness index of the image, and the larger the mean value is, the higher the image brightness is.

2) The image gray standard deviation sigma shows the dispersion process of the image pixel value and mean value, and the larger the value of sigma is, the higher the image quality is. The following formula:

where M × N represents the image size, f is the image pixel value, and mean represents the image mean.

3) The gray average gradient is expressed as a value obtained by performing first-order weighting calculation between the gray value of each point in the region and the gray value of the adjacent point, and the gray average gradient can well express the region with fine gray in the image. Defined as follows:

where f (i, j) represents the gray scale value of the image at the ith row and j column, M, N represents the row and column length, and G is the gray scale average gradient value of the image.

4) Zernike moments are commonly used to extract image edge contours and have the advantage of rotational invariance.

5) The Minimum Bounding Rectangle (MBR) refers to the maximum rectangular range representing an arbitrary two-dimensional shape by two-dimensional coordinates, ra and rb represent the major axis and the minor axis of the matrix, and the aspect ratio is represented as rb/ra.

(2) Secondary defect classification

In the secondary defect detection process, the samples are classified into 6 types and sample labels are set, wherein the 6 types of sample labels are metal loss, scratch, lead bending, lead pollution, cold joint and qualified respectively. And designing a decision tree multi-classification method to enable the SVM classifier to be applied to multi-classification problems with more sample types, and continuously inputting 6 types of characteristic parameters obtained from the samples into a secondary SVM defect classifier for learning. And the classifier after training can carry out classification test on the defects of the microwave bare chip.

The multi-classification problem can be solved by converting into a series of two-classification problems. In the specific embodiment, 6 types of classifiers for classifying lead contamination, metal deficiency, cold solder joint, scratch, lead bending and qualification are sequentially designed by a decision tree classification method, as shown in fig. 10.

It should be noted that the foregoing is only illustrative and illustrative of the present invention, and that any modifications and alterations to the present invention are within the scope of the present invention as those skilled in the art will recognize.

Claims

1. An intelligent detection system for surface defects of a microwave bare chip is characterized by comprising a camera, an industrial personal computer, a motion platform, a platform driver and a light source;

2. The microwave bare chip surface defect intelligent detection system of claim 1, wherein the industrial personal computer is connected with the camera through a network cable, the bare chip image collected by the camera is processed, the bare chip image is identified through the detection model, whether the bare chip has defects or not is judged, and when the bare chip has defects, the computer alarms through the connected alarm.

3. A microwave bare chip surface defect intelligent detection system according to claim 2, wherein the platen comprises a rotation mechanism, a mother board, a daughter board; a plurality of daughter boards are arranged on the mother board; the rotating mechanism is connected with the motherboard; each daughter board is used for placing a bare chip; the rotating mechanism is used for acquiring a control request and driving the mother board to rotate or stop according to the control request, so that the bare chip on each daughter board can reach the optimal position for the camera to acquire images.

4. An intelligent microwave bare chip surface defect detection device, which is characterized by comprising the intelligent microwave bare chip surface defect detection system and a shell, wherein the intelligent microwave bare chip surface defect detection system is installed inside the shell;

5. The apparatus of claim 4, wherein the apparatus further comprises a manipulator for grasping the bare chip.

6. An intelligent detection method for surface defects of a microwave bare chip, which is applied to the intelligent detection system for surface defects of a microwave bare chip as claimed in any one of claims 1 to 3, and comprises the following steps:

7. The method as claimed in claim 6, wherein the step S1 further comprises image preprocessing for the collected sample die image, the image preprocessing employs median filtering to remove image noise, and the method specifically comprises the following steps:

g(x，y)＝median{f(x-i，y-i)}，(i，j)∈W

8. The method according to claim 6, wherein the step S2 of splicing and combining the local sample bare chip images collected by the partitions into a complete chip image, namely a full-width standard template, by an image splicing algorithm specifically comprises the following steps:

step S21, extracting angular points in the image as registration features;

and step S23, correcting splicing errors.

9. The method of claim 8, wherein the step S3 specifically comprises the following steps:

10. The method as claimed in claim 9, wherein in step S4, after the image area of the suspected defect is obtained, the suspected defect is classified and determined by an SVM classifier, including a first-level defect classification and a second-level defect classification, and the method specifically includes the following steps: