Disclosure of Invention
The invention aims to provide a chip appearance detection device and method based on motion time association control, which aim to solve the limitation of the existing chip appearance detection system hardware on imaging resolution and view field parameters, and realize high-resolution and large-view field imaging detection which can only be realized by high-price hardware configuration with lower hardware configuration. According to the invention, high-precision coding micro-motion is introduced into the image acquisition part, a preliminary detected sample surface image is obtained through synchronous time sequence triggering, interpolation and pixel point value redistribution are carried out on the image after the preliminary detected sample surface image is subjected to associated calculation with coding motion information, a higher-resolution image is obtained under the condition of keeping the original view field, and more detail information of the sample is displayed, so that the image effect which can be realized only by a high-magnification system is realized.
In order to achieve the above purpose, the present invention provides the following technical solutions: a chip appearance detection method based on motion time association control comprises the following steps:
step one: placing the chip to be tested on a precise adjustment table, and adjusting the position and the height of the precise adjustment table to precisely and three-dimensionally adjust the position of the chip to be tested;
step two: the chip to be detected is adjusted to enable the microscopic imaging system and the imaging and image acquisition module to clearly image the chip, so that the chip is located at the focal plane of the microscopic imaging system, namely, the sensor plane of the imaging and image acquisition module, and the microscopic imaging system performs optical microscopic amplification on the chip to be detected;
step three: the information processing analysis module is used for setting the coding control mode of the motion control module and realizing the motion control of the micro-motion module by using the subsequent trigger output;
step four: setting exposure time parameters of an imaging and image acquisition module by using the information processing analysis module for subsequent triggering image acquisition;
step five: the information processing analysis module is used for sending a starting instruction to the synchronous time sequence trigger generation module, and the synchronous time sequence trigger generation module generates a single synchronous pulse trigger signal to respectively trigger the motion control module and the imaging and image acquisition module at the same time;
step six: inputting the image acquired by the imaging and image acquisition module into an information processing analysis module, performing association calculation on the acquired image and the coding information of the motion control module in the third step to acquire a higher-precision image, and simultaneously performing image processing based on a preset algorithm to extract sample characteristic information;
step seven: and outputting a corresponding detection result according to the processing analysis result of the information processing analysis module, and making a corresponding alarm prompt based on the detection result so as to realize the detection and analysis of the appearance of the chip.
Preferably, the method of setting the coding control mode in the third step is to set the displacement sequence of the motion control module on the X-Y plane in the period of the exposure period of the image sensor, and set the distance of each step displacement and the time kept at the current position.
Preferably, the method for setting the exposure time parameter in the fourth step is that the exposure period of the image sensor is set to be consistent with the total time of the coding control displacement in the third step, and other parameters are kept unchanged in the period.
Preferably, the method for extracting the characteristic information of the sample in the sixth step comprises the following steps:
acquiring data of a sample image amplified by a microscopic imaging system to obtain an appearance image sample I0 of the detection chip;
performing association calculation on the original image I0 and a matrix M corresponding to the coding control mode of the motion control module in the third step, so as to realize reassignment and interpolation of values of I0 pixels of the original image and obtain an image with higher degree of update;
and extracting the characteristic information of the chip to be detected according to the image sample obtained after the association calculation to obtain the appearance characteristic information of the detection chip.
Compared with the prior art, the invention has the beneficial effects that:
1. the device structure of the invention carries out optical microscopic amplification on a chip sample through a microscopic imaging system with lower magnification and lower configuration, carries out data acquisition on a sample image amplified by the microscopic imaging system in cooperation with the accurate coding motion of an image acquisition module, carries out association calculation on the acquired image combined with coding information, carries out interpolation and pixel value redistribution on the image, and realizes the high-resolution imaging effect through a calculation imaging mode. The motion time coding information is introduced, the effect of high resolution is realized through an algorithm, the dependence of imaging resolution on system hardware configuration is reduced, and the method has simpler hardware requirements and lower system cost
2. The invention introduces motion time coding information, realizes the effect of high resolution through an algorithm, keeps the original field angle while realizing high resolution, and has smaller magnification and higher field angle compared with a system with the same level resolution, thereby enabling the system to detect a larger chip surface range more quickly.
3. According to the invention, by introducing motion time coding information and associated imaging, the acquired chip sample characteristics are compared and judged, whether the chip is abnormal or not is determined, the abnormal chip sample characteristics of the chip with the abnormality are matched, the abnormality category is determined based on the matched parameters, and the corresponding abnormality alarm prompt is carried out, so that a manager can conveniently and timely make a response scheme, the abnormal chip is timely processed, the processing efficiency is improved, and the production quality of the chip is ensured.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1-4, the present invention provides a technical solution: a chip appearance detection device based on motion time association control comprises a precision adjusting table, a microscopic imaging system, a micro-motion module, an imaging and image acquisition module, a motion control module, a synchronous time sequence trigger generation module and an information processing analysis module. The imaging and image acquisition module is integrated in the micro-motion module, the micro-motion module is installed on the micro-imaging system, the micro-motion module, the imaging and image acquisition module and the micro-imaging system are coaxially arranged, the micro-motion module is electrically connected with the motion control module, the motion control module is electrically connected with the synchronous time sequence trigger generation module and the information processing analysis module respectively, the imaging and image acquisition module is electrically connected with the synchronous time sequence trigger generation module and the information processing analysis module respectively, and the synchronous time sequence trigger generation module is electrically connected with the information processing analysis module.
In the present invention, the precision adjusting table: for supporting and precisely adjusting the position of the chip to ensure that an accurate image is obtained during the inspection process.
Microscopic imaging system: providing a high resolution microscopic imaging function for viewing microscopic details of the chip, wherein the microscopic imaging system comprises an Objective lens, an eyepiece, a light source, a focusing system, a sample stage, a detector, a filter, an image processing unit, and software, an Objective lens (Objective): the objective lens is one of the core optical elements of a microscopic imaging system, responsible for focusing the light to form a magnified image. Different objectives have different magnifications and working distances for achieving different degrees of microstructure observation. Eyepiece (Eyepiece): eyepiece is used for direct observation of the sample by an observer, although in modern microscopic imaging systems digital cameras and displays are also often used for image display. Light source: the light source provides illumination to illuminate the sample so that it is visible under the microscope. Common light sources include white LEDs, halogen lamps or mercury arc sources, the particular choice depending on the application requirements. Focusing system: the focusing system is used for adjusting the focal length between the objective lens and the sample so as to obtain a clear image. This may be achieved by means of manual knobs, auto-focusing or electric focusing, etc. Sample Stage (Stage): the sample stage supports and positions a sample to be observed. It has an adjustment of the X, Y, Z axis allowing the user to locate and view the sample at different positions. The detector comprises: the detector captures the optical signal after passing through the objective lens and the sample, converting it into an electrical signal. Digital cameras are often used as detectors in modern microscopy imaging systems. An optical filter: the filters are used to select light of a particular wavelength to aid in enhancing the view of a particular feature of the sample, such as a fluorescence filter in a fluorescence microscope. An image processing unit: the image processing unit is used for processing, storing and displaying the image captured by the imaging system. This includes adjusting contrast, brightness, color balance, etc. and performing digital image analysis and processing. Digital camera and computer: modern microscopic imaging systems use digital cameras to capture images and process and display them in real time by a computer. The system allows for digitizing the images for convenient storage, sharing and further analysis. Software: the microscopic imaging system is accompanied by specific software for controlling the system, acquiring and processing the images, and contains some image analysis tools.
And the micro-motion module is used for: coaxially placed with the microscopic imaging system for tiny motion control. Which may be a micro-motion stage or other device capable of achieving micro-motion.
Imaging and image acquisition module: the micro-motion module is integrated in the micro-motion module and is responsible for acquiring images in real time in the motion process. This helps to capture minute details in motion.
A motion control module: and controlling the movement of the micro-movement module to ensure accurate positioning and movement of the chip. Including motion control of the XYZ axes.
A synchronous time sequence trigger generation module: and generating a synchronous time sequence trigger signal to ensure that the micro-motion module, the motion control module and the imaging and image acquisition module have synchronous time sequences when in cooperative work.
And the information processing and analyzing module is used for: the images acquired from the imaging and image acquisition modules are processed and analyzed. This includes algorithms such as image recognition, feature extraction, defect detection, etc., to enable efficient detection and analysis of chip appearance.
The operation method of the device comprises the following steps: placing a chip to be tested: the chip to be tested is placed on the precision adjustment stage in the field of view of the microscopic imaging system, i.e. the chip area to be tested. Adjusting the distance between the precise adjusting table and the microscopic imaging system: by adjusting the distance between the precise adjusting table and the microscopic imaging system, the imaging system is ensured to obtain clear and high-resolution images. Presetting motion control and imaging parameters: the information processing and analyzing module presets the coding control mode of the motion control module and the parameters of the imaging and image acquisition module. This includes parameters such as displacement step size, imaging frequency, etc. Transmitting a synchronous trigger signal instruction: the information processing analysis module sends an instruction to the synchronous time sequence trigger generation module, and the synchronous time sequence trigger generation module is triggered to generate a synchronous trigger signal. Synchronous trigger signal transmission: the synchronous trigger signal generated by the synchronous time sequence trigger generation module is transmitted to the motion control module and the imaging and image acquisition module. Displacement control and image acquisition begin: after receiving the synchronous trigger signal, the motion control module starts to execute preset displacement control work. Meanwhile, the imaging and image acquisition module starts to acquire a single image under the triggering of the synchronous signal. Image acquisition and motion control information association calculation: the information processing and analyzing module acquires the images acquired by the imaging and image acquisition module. And carrying out association calculation by combining coding control information of the motion control module so as to ensure that images of each step are associated with corresponding displacement information. Generating a high resolution image: the information processing analysis module generates a sample appearance image with higher resolution by combining all the acquired images. This helps to improve the accuracy and clarity of detection. Image recognition and analysis: and carrying out image recognition and analysis on the generated high-resolution image. This includes operations such as defect detection, feature extraction, shape analysis, etc. Outputting a detection report: and generating a detection report by the information processing analysis module according to the identification and analysis results. The report contains detailed information about the appearance of the chip, such as defect location, shape, size, etc.
Fig. 2 illustrates the manner in which the imaging and image acquisition modules of the present invention are integrated on a micro-motion module.
In order to better embody the detection flow of a chip appearance detection device based on motion time association control, as shown in fig. 3, the invention provides a chip appearance detection method based on motion time association control, which comprises the following steps:
step one: placing the processed chip to be tested on an upper surface area to be tested of a precise adjustment table, and adjusting the position and the height of the precise adjustment table to realize precise three-dimensional adjustment of the position of the chip to be tested;
in the above steps of the present invention, the chip to be measured is carefully placed on the upper surface of the precise adjustment table to ensure that the chip is located in the area to be measured, and the adjustment device is used, and the adjustment mode may be a manual knob, an electric device or other adjustment mechanisms to fine-adjust the position of the precise adjustment table so as to ensure that the chip to be measured is located in the desired observation position. The fine adjustment is carried out on the X, Y, Z shaft so as to ensure that the chip is in the center of the visual field, and the height of the precise adjustment table is adjusted so as to ensure that the focal plane is on the surface of the chip to be measured, thereby conveniently obtaining clear and well-focused images.
Step two: this step requires the realization of a clear amplified imaging of the chip to be tested. The method comprises the specific operation that a chip to be detected is adjusted to enable a microscopic imaging system and an imaging and image acquisition module to clearly image the chip, so that the chip is ensured to be positioned at a focal plane of the microscopic imaging system, namely a sensor plane of the imaging and image acquisition module, and the microscopic imaging system carries out optical microscopic amplification on the chip to be detected;
it should be noted that in the above steps of the present invention, the focal length adjusting function of the microscope system is used to ensure that the chip to be tested is located on the focal plane. By manually or automatically adjusting the focal length of the microscope. The definition change of the image can be observed in the process of adjusting the focal length so as to find the optimal focal point; the light source is ensured to fully illuminate the chip to be tested so as to obtain a clear image. The brightness and the direction of illumination can be adjusted, so that the light can be ensured to uniformly and brightly irradiate on the surface of the chip; during the adjustment, the image under the microscope is observed in real time. Ensuring that the microstructure of the chip is clearly visible so as to meet the requirements of subsequent analysis or detection; and adjusting parameters of the imaging and image acquisition module, including exposure time, gain and the like, according to the characteristics of the chip to be tested and imaging requirements.
Step three: and setting a coding control mode of the motion control module by using the information processing analysis module, and realizing motion control of the micro-motion module by using the subsequent trigger output. Specifically, the displacement order of the motion control module on the X-Y plane, the distance of each step displacement, and the holding time at the current position can be controlled by setting the exposure period of the image sensor.
In the above steps of the present invention, the information processing analysis module is used to set the exposure period of the image sensor. The exposure period determines the length of time the image sensor receives the light signal at each imaging. The distance of each step displacement, i.e. the step size of the minute displacement, is determined. This step size may be adjusted according to the specific application requirements. The smaller step size can improve the accuracy of position control; the length of time that is maintained at each location is determined. This is the time that remains stable at the current position after the fine displacement is performed to ensure adequate exposure and image acquisition; transmitting a setting instruction of coding control to a motion control module by utilizing an information processing analysis module; the motion control module starts to execute micro displacement on the X-Y plane according to the setting provided by the information processing analysis module, and keeps the set time at each position; after the micro displacement is executed at each position and kept for a certain time, the imaging and image acquisition module acquires images according to the set exposure period.
Step four: and setting exposure time parameters of the imaging and image acquisition module by utilizing the information processing analysis module for subsequent triggering of image acquisition. Specifically, the exposure period of the image sensor is set to be consistent with the total time of the code control displacement in the third step, and other parameters are kept unchanged in the period.
In the step of the present invention, the total time of the code control displacement is calculated according to the step length, the displacement sequence and the time kept at each position of the code control displacement in the step three; and setting the exposure period of the image sensor as the calculated total time of the coding control displacement by using an information processing analysis module. Ensuring that the exposure period can cover the whole process of tiny displacement so as to obtain a clear image; when setting the exposure period, it is ensured that other parameters related to the image acquisition remain unchanged. Including gain, frame rate, etc.; and sending a setting instruction to the imaging and image acquisition module through the information processing and analysis module, and transmitting the new exposure period parameter to the image sensor. After the exposure period parameter is set, the information processing analysis module can trigger the imaging and image acquisition module to execute image acquisition operation. At this time, the image sensor will perform image acquisition in accordance with the set exposure period.
Step five: and the information processing analysis module is used for sending a starting instruction to the synchronous time sequence trigger generation module, generating a single synchronous pulse trigger signal by the synchronous time sequence trigger generation module after receiving the instruction, respectively and simultaneously transmitting the single synchronous pulse trigger signal to the motion control module and the imaging and image acquisition module, and starting to operate corresponding preset working contents by each module after detecting a pulse rising edge.
In the above steps, parameters of the synchronous time sequence trigger generation module are set in the information processing analysis module, including the frequency, the starting time and the like of the trigger signal; ensuring that these parameters match the requirements of the system; when the information processing analysis module determines to start synchronous operation, a starting instruction is sent to the synchronous time sequence trigger generation module; after receiving the starting instruction, the synchronous time sequence trigger generation module starts to generate a synchronous pulse trigger signal; this signal will be used to synchronize the operation of the motion control module and the imaging and image acquisition module; the synchronous pulse trigger signal generated by the synchronous time sequence trigger generation module is transmitted to the motion control module and the imaging and image acquisition module at the same time; ensuring that signals can accurately reach each module; the motion control module and the imaging and image acquisition module respectively detect the rising edge of the synchronous pulse trigger signal; this rising edge is typically used as a point in time to begin executing the preset work content; the motion control module starts to execute preset working contents after detecting the rising edge of the synchronous pulse trigger signal, wherein the preset working contents comprise motion control of tiny displacement, and the imaging and image acquisition module starts to execute the preset working contents, such as image acquisition, after detecting the rising edge of the synchronous pulse trigger signal; after each module completes the preset work content, a corresponding completion signal or state information may be generated, so that the information processing analysis module or other modules can further process the work content.
Step six: the method mainly comprises the steps of firstly collecting a chip surface image, then obtaining a high-resolution image through association calculation, and finally processing and analyzing the high-resolution image. The method comprises the steps of inputting an image acquired by an imaging and image acquisition module into an information processing and analyzing module, and performing association calculation on coding information of a motion control module in the third step so as to acquire a higher-precision image, and simultaneously performing image processing based on a preset algorithm to extract sample characteristic information. The specific method comprises the following steps: and acquiring data of the image of the sample to be detected amplified by the microscopic imaging system to obtain an appearance image sample I0 of the chip to be detected. And then, carrying out association calculation on the obtained original image I0 and a matrix M corresponding to the coding control mode of the motion control module in the step three, so as to realize the redistribution and interpolation of the values of the I0 pixel points of the original image and obtain a higher-precision image. And extracting the characteristic information of the chip to be detected according to the high-precision image sample obtained after the correlation calculation to obtain the appearance characteristic information of the chip to be detected. And determining the abnormal category of the chip sample characteristics based on the matching parameters, determining the corresponding abnormal alarm prompt according to the abnormal category of the chip sample characteristics, and carrying out abnormal alarm.
It should be noted that, in the above steps of the present invention, an imaging and image acquisition module is used to acquire an image of a surface of a chip to be tested, so as to obtain an original image sample I0. The image sample may be magnified by a microscopic imaging system. And (3) performing association calculation on the original image I0 and a matrix M corresponding to the coding control mode of the motion control module in the step three. The purpose of the correlation calculation is to reassign and interpolate pixel values in the original image to obtain a higher precision image. And generating a high-precision image through a result obtained by the correlation calculation. The image has higher resolution and precision, and is helpful for improving the capturing capability of the chip details to be tested. And extracting characteristic information of the high-precision image by utilizing an information processing and analyzing module. Which is used to identify and quantify the appearance characteristics of the chip to be inspected, such as shape, color, texture, etc. Based on the extracted feature information, the matching parameters are used to determine an anomaly class of the chip sample feature for comparing the extracted feature with the desired feature of the normal sample. Once the anomaly category of the chip sample feature is determined, the system may generate a corresponding anomaly alert prompt. Including displaying the abnormal region, providing a description of the type of abnormality, etc.
Step seven: this step is used to enable detection and analysis of the chip appearance. And outputting a corresponding detection result according to the processing analysis result of the information processing analysis module, and judging whether the chip sample to be detected has an out-of-standard flaw or not based on the detection result and the set flaw standard. If yes, marking the defect area, making a corresponding alarm prompt and outputting a detection report; if no out-of-standard flaw exists, the detection report is directly output.
It should be noted that, in the above steps of the present invention, the information processing and analyzing module outputs the detection result according to the previous image processing and analyzing steps. The results include various quantitative and qualitative analyses of chip appearance characteristics, such as shape, color, texture, and the like. Based on the information processing analysis result, the system judges whether the chip sample to be tested has an exceeding flaw. This can be achieved by comparison with a preset flaw criterion. And if the detection result meets the standard exceeding condition defined in the flaw standard, judging that the standard exceeding flaw exists. If there is an out-of-limit flaw, the system may mark the corresponding defect area for display in the final inspection report. The display mode includes indicating the positions of flaws on the image by marks or bounding boxes. For the case of an out-of-limit flaw, the system may generate a corresponding alert prompt. The prompts may include textual descriptions, graphical indicia, or other forms that alert the operator to a particular area or take further action. The system outputs a final detection report which comprises detection results, whether out-of-standard flaws exist, marked defect areas and the like. Reports may be used to record the detection process, provide visual feedback, and serve as a reference for quality control. If the chip sample to be tested does not detect out-of-standard flaws, the system directly outputs a detection report without marking the flaw area or generating additional alarms. In this case, the report reflects the result that the sample to be tested meets the expected standard in appearance.
According to the above embodiments, the final experimental system obtained the target detection results as shown in fig. 4 and 5. The embodiment result shows that the method and the device can conveniently and rapidly realize the reconstruction of the high-resolution image with lower hardware configuration, simultaneously keep a higher field angle, finally realize better and faster measurement results, can be widely applied to optical imaging and detection, can be applied to biological sample detection and object type identification scenes, and provide a simple, feasible and low-cost detection method and device for improving the accuracy of detection results. The method is convenient for a manager to process the abnormal chips in time, improves the processing efficiency and ensures the production quality of the chips.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations may be made therein without departing from the spirit and scope of the invention as defined by the appended claims and their equivalents.