CN109448109B - Three-dimensional reconstruction method of scanning electron microscope image - Google Patents
Three-dimensional reconstruction method of scanning electron microscope image Download PDFInfo
- Publication number
- CN109448109B CN109448109B CN201811230563.4A CN201811230563A CN109448109B CN 109448109 B CN109448109 B CN 109448109B CN 201811230563 A CN201811230563 A CN 201811230563A CN 109448109 B CN109448109 B CN 109448109B
- Authority
- CN
- China
- Prior art keywords
- image
- template
- value
- dimensional
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Abstract
The invention discloses a three-dimensional reconstruction method of a scanning electron microscope image. The invention comprises the following steps: setting an image number threshold value, and selecting a template area for a first frame of an image acquired by SEM; moving the sample platform with a predefined step length to obtain a real-time image, and performing template matching by adopting a template area of a first frame; adding the matched images into a matched image sequence; selecting an interested region for the matched image sequence, and calculating the focusing measurement of each pixel point by adopting modified Laplacian; calculating the maximum focusing measured value of each pixel point by adopting Gaussian interpolation, and calculating the height of the corresponding point according to the maximum focusing measured value to obtain a three-dimensional point cloud; and according to the three-dimensional point cloud, interpolating and fitting the three-dimensional graph to realize three-dimensional reconstruction.
Description
Technical Field
The invention relates to the field of electron microscope image processing, in particular to a three-dimensional reconstruction method of a scanning electron microscope image
Background
Scanning Electron Microscopy (SEM) has been widely used as a large-scale analytical instrument for microscopic morphology observation and microstructure analysis, for accurate measurement of nanomaterial size, performance characterization, and 3D morphology recovery of materials. In addition, by using the SEM image as a visual sensor, great help is provided for development of automatic nano operation, such as automatic detection of I C chips, automatic pickup of nanowires and automatic measurement of nanowire impedance characteristics by four probes, so that people are liberated from complicated manual nano operation, and the working efficiency is greatly improved. However, due to lack of height information in SEM images, damage to the end effector is caused, greatly reducing the success rate of automated nanomanipulation.
However, with the development of machine vision, measurement of three-dimensional dimensions based on two-dimensional images has been greatly developed. Because of its high precision, the focus morphology recovery (SSF) has been widely used in robot obstacle avoidance, biological entity, and three-dimensional reconstruction of mineral particles with simple implementation. However, in the process of acquiring a large number of graphic sequences by using the SSF, the sample platform cannot move up and down absolutely due to vibration of the executing mechanism, and the image frames are laterally moved, so that errors of focus measurement values are caused, and the accuracy of three-dimensional reconstruction is affected.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a three-dimensional reconstruction method of a scanning electron microscope image, which can eliminate errors of three-dimensional reconstruction of a traditional SSF algorithm caused by vibration of an actuating mechanism, and is simple and easy to realize.
In order to solve the technical problems, the invention provides a three-dimensional reconstruction method of a scanning electron microscope image, which comprises the following steps:
setting an image number threshold value, and selecting a template area for a first frame of an image acquired by SEM;
moving the sample platform with a predefined step length to obtain a real-time image, and performing template matching by adopting a template area of a first frame;
adding the matched images into a matched image sequence;
repeating the steps when the threshold of the number of the images is not reached;
selecting an interested region from the matching image sequence, and calculating a focusing measured value of each pixel point by adopting modified Laplacian;
calculating the maximum focusing measured value of each pixel point by adopting Gaussian interpolation, and calculating the height of the corresponding point according to the maximum focusing measured value to obtain a three-dimensional point cloud;
and according to the three-dimensional point cloud, interpolating and fitting the three-dimensional graph to realize three-dimensional reconstruction.
In one embodiment, the moving the sample platform with a predefined step length to obtain a real-time image, and performing template matching by using a template area of a first frame specifically includes:
moving the sample stage in a preset step length to acquire a real-time image;
and (3) adopting a coefficient matching algorithm, and taking the first frame template area as a template to extract the template area of the real-time image.
In one embodiment, the matching image sequence specifically includes:
and adding the template image area acquired from the real-time image into the first frame image, and combining the template image area into a new image sequence, namely a matching image sequence.
In one embodiment, the selecting a region of interest for the matching image sequence, and calculating the focus measurement value of each pixel point by using modified laplace specifically includes:
selecting the same position from the image matching sequence as an interested region to form an interested region sequence;
the corrected laplace operator is used to calculate a focus measurement for each pixel within the sequence of regions of interest.
In one embodiment, a maximum focusing measurement value of each pixel point is calculated by Gaussian interpolation, and the height of a corresponding point is calculated according to the maximum focusing measurement value to obtain a three-dimensional point cloud;
using gaussian interpolation to calculate a maximum focus measurement for each pixel of the region of interest sequence;
calculating the height of the pixel by the maximum focus measurement value and the position of the region of interest sequence where the maximum focus measurement value is positioned;
and acquiring the heights of all pixels to obtain a three-dimensional point cloud.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any of the methods when the program is executed.
A computer readable storage medium having stored thereon a computer program which when executed by a processor realizes the steps of any of the methods.
A processor for running a program, wherein the program runs on performing any one of the methods.
The invention has the beneficial effects that:
1. the invention improves the traditional SSF three-dimensional reconstruction method, has simple algorithm and small operand.
2. The invention can effectively eliminate the problem that the sample platform cannot move up and down absolutely due to the vibration of the actuating mechanism, and the error of focus measurement value is caused along with the lateral movement of the image frame, thereby influencing the accuracy of three-dimensional reconstruction.
Drawings
FIG. 1 is a diagram of SEM actuator vibration (image side shift, resulting in error in pixel focus measurements) in a three-dimensional reconstruction method of a scanning electron microscope image according to the present invention.
Fig. 2 is a flow chart of a three-dimensional reconstruction method of a scanning electron microscope image according to the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and specific examples, which are not intended to be limiting, so that those skilled in the art will better understand the invention and practice it.
Referring to fig. 2, the invention discloses a three-dimensional reconstruction method of a scanning electron microscope image, which comprises the following steps:
step S101, setting an image number threshold value, and selecting a template area for a first frame of an image acquired by SEM;
step S102, moving the sample platform with a predefined step length to obtain a real-time image, and performing template matching by using a template area of a first frame. Wherein the coefficient matching method matches the relative value of the template to its mean with the relative value of the image to its mean, 1 represents a perfect match, -1 represents a bad match, and 0 represents no correlation (random sequence). The calculation formula is as follows:
R (x，y) ＝∑ x′，y′ (T′(x′，y′)·I(x+x′，y+y′)) (1)
wherein:
T′(x′，y′)＝T(x′，y′)-I/(w·h)·∑ x″，y″ T (x″，y″) (2)
I′(x+x′，y+y′)＝I(x+x′，y+y′)-I/(w·h)·∑ x″，y″ I(x+x″，y+y″) (3)
step S103, adding the matched images into a matched image sequence;
step S104, repeating the steps when the image number threshold is not reached;
step 105, selecting a region of interest for the matching image sequence, calculating a focus measurement value of each pixel point by using modified laplace, and calculating by using a variable step S to adapt to the texture change of the image, namely:
L M (x，y)＝|2I(x，y)-I(x-s，y)-I(x+s，y)|+|2I(x，y)-I(x，y-s)-I(x，y+s)| (4)
wherein: l (L) M (x, y) is the corrected Laplace's seed value at point (x, y); i (x, y) is the gray value of the (x, y) point on the image.
The final focus measurement at point (i, j) is at [ ] i J) sum of corrected Laplace calculated values in field, i.e
Wherein: n is the window size calculated by the (x, y) field reference; t is a threshold. Laplace calculation values greater than T participate in the calculation, where N determines the window size at the time of calculation.
And S106, calculating the maximum focusing measured value of each pixel point by adopting Gaussian interpolation, and calculating the height of the corresponding point according to the maximum focusing measured value to obtain the three-dimensional point cloud. Since the height obtained by the maximum method is discrete, the resolution is limited by the number of layers of the image taken, and the focus maximum of each pixel is calculated by using Gaussian interpolation fitting. The focus measurement degree of the adjacent three images is F m-1 ，F m ，F m+1 But the maximum measurement of the object is not in these three images. Fitting a Gaussian function according to a Gaussian function fitting model to obtain the maximum focus measurement, corresponding toThe height value of the point on the object at this time is also obtained.
Wherein: d, d m-1 ，d m ，d m+1 F is the focal position m-1 ，F m ，F m+1 In order to focus the measured values,
focal position for Gaussian fitting, F p Is the fitted focus measurement.
And S107, interpolating and fitting the three-dimensional graph according to the three-dimensional point cloud to realize three-dimensional reconstruction.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any of the methods when the program is executed.
A computer readable storage medium having stored thereon a computer program which when executed by a processor realizes the steps of any of the methods.
A processor for running a program, wherein the program runs on performing any one of the methods.
The above-described embodiments are merely preferred embodiments for fully explaining the present invention, and the scope of the present invention is not limited thereto. Equivalent substitutions and modifications will occur to those skilled in the art based on the present invention, and are intended to be within the scope of the present invention. The protection scope of the invention is subject to the claims.
Claims (5)
1. The three-dimensional reconstruction method of the scanning electron microscope image is characterized by comprising the following steps of:
setting an image number threshold value, and selecting a template area for a first frame of an image acquired by SEM;
moving a sample platform with a predefined step length to acquire a real-time image, and performing template matching by adopting a template area of a first frame, wherein the template area extraction of the real-time image is performed by adopting a coefficient matching algorithm and taking the template area of the first frame as a template, the coefficient matching algorithm matches the relative value of the template to the mean value of the template with the relative value of the image to the mean value of the image, 1 represents perfect matching, -1 represents bad matching, 0 represents no correlation, and the calculation formula is as follows:
R (x，y) ＝Σ x′，y′ (T′(x′，y′)·I(x+x′，y+y′))
wherein:
T′(x′，y′)＝T(x′，y′)-I/(w·h)·∑ x″，y″ T(x″，y″)
I′(x+x′，y+y′)＝I(x+x′，y+y′)-I/(w·h)·∑ x″，y″ I(x+x″，y+y″)；
adding the matched images into a matched image sequence;
repeating the steps when the threshold of the number of the images is not reached;
selecting a region of interest for a matching image sequence, calculating a focus measurement value for each pixel point using modified Laplacian, and calculating with a variable step size s to adapt to the texture variation of the image, i.e. L M (x, y) = |2I (x, y) -I (x-s, y) -I (x+s, y) |+|2I (x, y) -I (x, y-s) -I (x, y+s) |, wherein L M (x, y) is the corrected Laplace's seed value at point (x, y); i (x, y) is the gray value of the (x, y) point on the image; the final focus measurement at point (i, j) is the sum of corrected Laplacian calculations in the (i, j) field, i.e
Wherein N is the window size calculated by the (x, y) field internal reference; t is oneA threshold, wherein the Laplace calculated value larger than T participates in calculation, and N determines the window size during calculation;
calculating the maximum focusing measurement value of each pixel point by Gaussian interpolation, and calculating the height of the corresponding point according to the maximum focusing measurement value to obtain a three-dimensional point cloud, wherein the focusing measurement degrees of the adjacent three images are F respectively m-1 ,F m ,F m+1 However, the maximum measurement degree of the object is not in the three images, the maximum focusing measurement degree can be calculated by fitting a Gaussian function according to a Gaussian function fitting model, and accordingly, the height value of the point on the object at the moment is also obtained:
wherein d is m-1 ,d m ,d m+1 F is the focal position m-1 ,F m ,F m+1 In order to focus the measured values,
focal position for Gaussian fitting, F p Is the fitted focus measurement;
and according to the three-dimensional point cloud, interpolating and fitting the three-dimensional graph to realize three-dimensional reconstruction.
2. The method for three-dimensional reconstruction of scanning electron microscope images according to claim 1, wherein the matching image sequence specifically comprises:
and adding the template image area acquired from the real-time image into the first frame image, and combining the template image area into a new image sequence, namely a matching image sequence.
3. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of claim 1 or 2 when executing the program.
4. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method of claim 1 or 2.
5. A processor for running a program, wherein the program when run performs the method of claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811230563.4A CN109448109B (en) | 2018-10-22 | 2018-10-22 | Three-dimensional reconstruction method of scanning electron microscope image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811230563.4A CN109448109B (en) | 2018-10-22 | 2018-10-22 | Three-dimensional reconstruction method of scanning electron microscope image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109448109A CN109448109A (en) | 2019-03-08 |
| CN109448109B true CN109448109B (en) | 2023-06-20 |
Family
ID=65547073
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811230563.4A Active CN109448109B (en) | 2018-10-22 | 2018-10-22 | Three-dimensional reconstruction method of scanning electron microscope image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109448109B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110288701B (en) * | 2019-06-26 | 2023-01-24 | 图码思(成都)科技有限公司 | Three-dimensional reconstruction method based on depth focusing and terminal |
| CN110910506B (en) * | 2019-12-03 | 2021-02-09 | 江苏集萃华科智能装备科技有限公司 | Three-dimensional reconstruction method and device based on normal detection, detection device and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103791836A (en) * | 2014-01-28 | 2014-05-14 | 南京航空航天大学 | Numerically controlled tool cutting edge measuring method based on laser scanning confocal technology |
| CN106683068A (en) * | 2015-11-04 | 2017-05-17 | 北京文博远大数字技术有限公司 | Three-dimensional digital image acquisition method and equipment thereof |
| CN106959079A (en) * | 2017-03-27 | 2017-07-18 | 淮阴师范学院 | A kind of modified focuses on 3 D measuring method |
-
2018
- 2018-10-22 CN CN201811230563.4A patent/CN109448109B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103791836A (en) * | 2014-01-28 | 2014-05-14 | 南京航空航天大学 | Numerically controlled tool cutting edge measuring method based on laser scanning confocal technology |
| CN106683068A (en) * | 2015-11-04 | 2017-05-17 | 北京文博远大数字技术有限公司 | Three-dimensional digital image acquisition method and equipment thereof |
| CN106959079A (en) * | 2017-03-27 | 2017-07-18 | 淮阴师范学院 | A kind of modified focuses on 3 D measuring method |
Non-Patent Citations (3)
| Title |
|---|
| A hybrid 3D SEM reconstruction method optimized for complex geologic material surfaces;Shang Yan 等;《Mircron》;20170402;第26页-31页 * |
| 基于灰色关联度的聚焦形貌恢复;杨维 等;《计算机应用研究》;20150228;第32卷(第2期);第613页-618页 * |
| 摄影测量辅助地面激光点云的三维重建;刘冰洋 等;《郑州师范教育》;20150331;第4卷(第2期);第40页-44页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109448109A (en) | 2019-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3236418A2 (en) | Image processing apparatus, image processing method, and storage medium | |
| WO2018082085A1 (en) | Microscope image acquisition method based on sequence slice | |
| JP6385318B2 (en) | Transform 3D objects to segment objects in 3D medical images | |
| CN110189257B (en) | Point cloud acquisition method, device, system and storage medium | |
| CN109448109B (en) | Three-dimensional reconstruction method of scanning electron microscope image | |
| CN106570484B (en) | MIcrosope image acquisition method based on sequence section | |
| CN111523547B (en) | 3D semantic segmentation method and terminal | |
| CN108305268B (en) | Image segmentation method and device | |
| CN113822810A (en) | Method for positioning workpiece in three-dimensional space based on machine vision | |
| CN111127613B (en) | Image sequence three-dimensional reconstruction method and system based on scanning electron microscope | |
| JPWO2007069414A1 (en) | Method for analyzing cells and the like having a linear form, neuronal cell analysis method, and apparatus and program for executing those methods | |
| CN112633248A (en) | Deep learning all-in-focus microscopic image acquisition method | |
| Lee et al. | Optimizing image focus for 3D shape recovery through genetic algorithm | |
| CN116363181A (en) | Feature-based CT image and ultrasonic image liver registration method | |
| CN108955562B (en) | Digital extension method and system for microscopic depth of field of microscopic vision system | |
| CN108364274B (en) | Nondestructive clear reconstruction method of optical image under micro-nano scale | |
| CN116091317A (en) | Super-resolution method and system for secondary electron image of scanning electron microscope | |
| CN109084721B (en) | Method and apparatus for determining a topographical parameter of a target structure in a semiconductor device | |
| Ding et al. | Automatic 3D reconstruction of SEM images based on Nano-robotic manipulation and epipolar plane images | |
| US20230114624A1 (en) | Defect examination on a semiconductor specimen | |
| CN112330667B (en) | Morphology-based laser stripe center line extraction method | |
| CN110264433B (en) | Depth map interpolation method based on color segmentation guidance | |
| Sahay et al. | Shape extraction of low‐textured objects in video microscopy | |
| CN108805975B (en) | Microscopic 3D reconstruction method based on improved iterative shrinkage threshold algorithm | |
| CN111091569B (en) | Industrial CT image segmentation method with self-adaptive local parameters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 215100 South 3rd and 4th Floors of Huanxiu Lake Building, Xiangcheng High-speed Railway New Town, Suzhou City, Jiangsu Province Applicant after: JIANGSU JICUI MICRO-NANO AUTOMATION SYSTEM AND EQUIPMENT TECHNOLOGY RESEARCH INSTITUTE Co.,Ltd. Address before: 215100 F003 workstation, 3rd floor, 58 South Tiancheng Road, Xiangcheng High-speed Railway New Town, Suzhou City, Jiangsu Province Applicant before: JIANGSU JICUI MICRO-NANO AUTOMATION SYSTEM AND EQUIPMENT TECHNOLOGY RESEARCH INSTITUTE Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |