WO2023197785A1

WO2023197785A1 - Three-dimensional reconstruction method and apparatus for local orbital function

Info

Publication number: WO2023197785A1
Application number: PCT/CN2023/080253
Authority: WO
Inventors: 于荣; 毛梁泽; 程志英
Original assignee: 清华大学
Priority date: 2022-04-12
Filing date: 2023-03-08
Publication date: 2023-10-19
Also published as: CN114648611A; CN114648611B

Abstract

The present application discloses a three-dimensional reconstruction method and apparatus for a local orbital function. The method comprises: collecting image data of a sample at a plurality of tilt angles; scattering points at equal intervals in a real space and performing linear accumulation to obtain a calculated image at each tilt angle of the plurality of tilt angles; and further obtaining a gradient of a loss function with respect to a parameter to be optimized, optimizing said parameter according to the gradient, screening atoms satisfying a preset condition, recalculating a new loss function until a convergence condition is satisfied, and reconstructing three-dimensional space coordinates of a center of a local orbital function and the shape of the local orbital function in the real space to obtain a three-dimensional reconstruction result. Thus, the technical problems in the related art that three-dimensional coordinates of atoms can only be obtained from a reconstructed three-dimensional density matrix, and errors cannot be corrected, resulting in a relatively high hardware requirement in a reconstruction process and poor precision of reconstructed three-dimensional coordinates are solved.

Description

Three-dimensional reconstruction method and device of local orbital function

Cross-references to related applications

This application is based on the Chinese patent application with application number 202210381325.3 and the filing date is April 12, 2022, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated into this application as a reference.

Technical field

The present application relates to the field of three-dimensional imaging technology, and in particular to a three-dimensional reconstruction method and device of a local orbit function.

Background technique

Usually, the electron microscope photos of the sample contain rich information that is difficult to obtain intuitively. Applying a three-dimensional reconstruction algorithm to obtain the three-dimensional structural information of the sample from a series of electron microscope projection photos of the sample is essential for understanding the material composition at a fundamental level. It helps a lot with performance.

In recent years, with the development of data acquisition methods, iterative three-dimensional reconstruction algorithms and post-processing methods, AET (Atomic Electrical Tomography, atomic scale electron tomography) has become a powerful tool for the characterization of three-dimensional and four-dimensional atomic scale structures. It provides The ability to correlate the structure and properties of materials at the atomic level. AET in related technologies can determine three-dimensional atomic coordinates and element species with sub-angstrom accuracy, and reveal their atomic-scale time evolution in dynamic processes.

However, in the related technology, the AET algorithm obtains the three-dimensional coordinates of atoms through peak searching from the reconstructed three-dimensional density matrix, which has high hardware requirements, and during the peak searching process, cumbersome human intervention is required, which is difficult to avoid Errors caused by human intervention. At the same time, while directly reconstructing the three-dimensional coordinates of atoms, it is impossible to correct the sample drift and mechanical tilt error of the sample stage when collecting images. The accuracy is poor and needs to be improved.

Contents of the invention

This application provides a method and device for three-dimensional reconstruction of local orbital functions to solve the problem in related technologies that only the three-dimensional coordinates of atoms can be obtained from the reconstructed three-dimensional density matrix, and the errors cannot be corrected, resulting in reconstruction problems. The process has high hardware requirements and the reconstructed three-dimensional coordinates have poor accuracy.

The first embodiment of the present application provides a three-dimensional reconstruction method of a local orbit function, which includes the following steps: collecting image data of samples at multiple tilt angles; based on the image data, scattering is performed at equal intervals in real space. points and use linear accumulation to obtain the calculated image at each tilt angle of the multiple tilt angles; and calculate the loss function based on the calculated image at each tilt angle, and obtain the parameters of the loss function to be optimized gradient, and optimize the parameters to be optimized according to the gradient, screen out the atoms that meet the preset conditions, and recalculate the new loss function until the The convergence condition is to reconstruct the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function in the real space to obtain a three-dimensional reconstruction result.

Optionally, in one embodiment of the present application, collecting image data of the sample at the multiple tilt angles includes: acquiring initial image data of the sample at the multiple tilt angles; The image data is subjected to alignment and noise reduction processing, and the processed image is normalized to obtain the image data.

Optionally, in one embodiment of the present application, the calculation formula of the loss function is:

Among them, W represents the loss function, M represents the total number of tilt angles, j represents the serial number of the tilt angle, i represents the local orbit function serial number, P represents the side length of the image, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, f _j (u, v) represents the image calculated at the j-th angle, and b _j (u, v) represents the image obtained experimentally at the j-th angle.

Optionally, in one embodiment of the present application, filtering out atoms that meet preset conditions includes: after iteratively updating the parameters at each step, detecting the parameters of all local orbital functions before detecting any local When the parameters of the orbital function are less than the threshold obtained from all the parameters of the local orbital function, while deleting any of the local orbital functions, a local orbital function whose center distance is smaller than the preset pixel is obtained by establishing a binary tree. For the index, delete any local orbit function in the pair of local orbit functions; after each step iteratively updates the parameters and deletes the local orbit function, reduce the parameters of each local orbit function by a preset probability. is a preset multiple, and the protection time is set so that filtering operations and deletion operations are not allowed to be performed within the protection time.

Optionally, in one embodiment of the present application, the parameters to be optimized include the three-dimensional spatial coordinates of the center of each local orbit function, parameters describing its shape, three Euler angles corresponding to each corner, and each At least one of the drift of the sample under the angle and the mechanical tilt deviation of the sample stage.

The second embodiment of the present application provides a three-dimensional reconstruction device of a local orbit function, including: a collection module for collecting image data of samples at multiple tilt angles; an accumulation module for based on the image data , scatter points at equal intervals in the real space and use linear accumulation to obtain the calculated image at each tilt angle of the multiple tilt angles; and a reconstruction module used to calculate the image at each tilt angle according to Calculate the loss function, obtain the gradient of the loss function with respect to the parameters to be optimized, and optimize the parameters to be optimized according to the gradient, screen out atoms that meet the preset conditions, and recalculate the new loss function until the convergence conditions are met, The three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function are reconstructed in the real space to obtain a three-dimensional reconstruction result.

Optionally, in one embodiment of the present application, the acquisition module includes: an acquisition unit, used to acquire initial image data of the sample at the multiple tilt angles; and a noise reduction unit, used to perform the initial image data on the sample. The image data is subjected to alignment and noise reduction processing, and the processed image is normalized to obtain the image data.

Optionally, in one embodiment of the present application, the reconstruction module includes: a detection unit, used to detect the parameter deletion unit of all local orbit functions after the parameters are iteratively updated in each step, and a unit used to detect the parameter deletion unit of all local orbit functions after detecting When the parameters of any local orbit function are less than the threshold obtained from all the parameters of the local orbit function, while deleting the any local orbit function, a binary tree is established to obtain the center distance of the local orbit function less than the preset pixel. The local orbit function pair index is used to delete any local orbit function in the local orbit function pair; the protection unit is used to update each local orbit function after iteratively updating the parameters and deleting the local orbit function at each step. The parameters of the orbital function are all reduced to a preset multiple with a preset probability, and a protection time is set so that filtering operations and deletion operations are not allowed to be performed within the protection time.

A third embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes the program to implement The three-dimensional reconstruction method of local orbital function as described in the above embodiment.

A fourth embodiment of the present application provides a computer-readable storage medium that stores computer instructions, and the computer instructions are used to cause the computer to execute the local orbit function as described in the above embodiment. Three-dimensional reconstruction method.

The embodiment of the present application obtains calculated images at multiple tilt angles based on the collected image data of the sample at multiple tilt angles, and then calculates the loss function of the calculated image, and obtains the gradient of the optimized parameters, thereby obtaining the optimized parameters. , after repeated screening and calculation, until the loss function meets the convergence conditions, the three-dimensional reconstruction result is obtained, which can streamline the three-dimensional coordinate reconstruction process, not only reducing the requirements for hardware, but also reducing tedious human intervention and saving labor At the same time, during the iterative process, sample drift and mechanical tilt error of the sample stage can also be corrected, thereby improving the accuracy of three-dimensional coordinate reconstruction. This solves the problem in related technologies that the three-dimensional coordinates of atoms can only be obtained from the reconstructed three-dimensional density matrix, and the errors cannot be corrected, resulting in high hardware requirements for the reconstruction process and the accuracy of the reconstructed three-dimensional coordinates. Poor technical issues.

Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Description of the drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

Figure 1 is a flow chart of a three-dimensional reconstruction method of local orbital functions provided according to an embodiment of the present application;

Figure 2 is a flow chart of a three-dimensional reconstruction method of local orbital functions according to an embodiment of the present application;

Figure 3 is a schematic diagram of simulations at 25°, 0° and -25° of small particles composed of 10,000 atoms to be reconstructed according to an embodiment of the present application;

Figure 4 is a schematic diagram of scatter points composed of the initial input atomic coordinates of the three-dimensional reconstruction method of the local orbital function according to an embodiment of the present application;

Figure 5 is a schematic diagram of the calculation of the initial input local orbit function at tilt angles of 25°, 0° and -25° respectively according to the three-dimensional reconstruction method of the local orbit function according to an embodiment of the present application;

Figure 6 is a schematic polyline diagram of the value of the loss function during the iterative process of the three-dimensional reconstruction method of the local orbit function according to an embodiment of the present application;

Figure 7 is a schematic diagram of an atomic model obtained after convergence of the three-dimensional reconstruction method of local orbital functions according to an embodiment of the present application;

Figure 8 shows the difference between the calculated image and the experimental image during convergence of the three-dimensional reconstruction method of the local orbit function according to one embodiment of the present application;

Figure 9 is a histogram of the distance between the atomic coordinates and the real coordinates obtained by the convergence calculation of the three-dimensional reconstruction method of the local orbital function according to an embodiment of the present application;

Figure 10 is a schematic structural diagram of a three-dimensional reconstruction device of a local orbit function according to an embodiment of the present application;

FIG. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed ways

The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application, but should not be construed as limiting the present application.

The three-dimensional reconstruction method and device of the local orbit function according to the embodiment of the present application will be described below with reference to the accompanying drawings. Regarding the above background technology center, the related technologies mentioned can only obtain the three-dimensional coordinates of atoms from the reconstructed three-dimensional density matrix, and cannot correct the errors, resulting in high hardware requirements for the reconstruction process, and the reconstructed three-dimensional To solve the technical problem of poor coordinate accuracy, this application provides a three-dimensional reconstruction method of local orbit function. In this method, multiple tilt angles are obtained based on the collected image data of the sample at multiple tilt angles. Calculate the image under the calculation image, then calculate the loss function of the calculation image, obtain the gradient of the optimization parameters, and obtain the optimization parameters. After repeated screening and calculation, until the loss function The data satisfies the convergence conditions, and then the three-dimensional reconstruction results are obtained, which can streamline the three-dimensional coordinate reconstruction process, which not only reduces the requirements for hardware, but also reduces tedious human intervention and saves labor costs. At the same time, it can also be corrected during the iterative process. Sample drift and mechanical tilt error of the sample stage, thereby improving the accuracy of three-dimensional coordinate reconstruction. This solves the problem in related technologies that the three-dimensional coordinates of atoms can only be obtained from the reconstructed three-dimensional density matrix, and the errors cannot be corrected, resulting in high hardware requirements for the reconstruction process and the accuracy of the reconstructed three-dimensional coordinates. Poor technical issues.

Specifically, FIG. 1 is a schematic flowchart of a three-dimensional reconstruction method of a local orbital function provided by an embodiment of the present application.

As shown in Figure 1, the three-dimensional reconstruction method of the local orbit function includes the following steps:

In step S101, image data of the sample at multiple tilt angles is collected.

In the actual implementation process, the embodiment of the present application can capture HAADF (High-Angle Annular Dark-Field imaging) images of the sample at a series of different rotation angles, and then obtain the sample at multiple tilt angles. The image data is convenient for subsequent processing of image data and three-dimensional coordinate reconstruction.

Optionally, in one embodiment of the present application, collecting image data of samples at multiple tilt angles includes: acquiring initial image data of samples at multiple tilt angles; centering and combining the initial image data. axis and noise reduction processing, and normalized by the processed image to obtain image data.

Specifically, after the embodiment of the present application obtains the initial image data of the sample at multiple tilt angles through shooting, in order to ensure the accuracy of subsequent three-dimensional coordinate reconstruction, the initial image data can be subjected to centering and axis and noise reduction processing. While facilitating subsequent calculations, it avoids the impact of noise on three-dimensional coordinate reconstruction, and then through linear normalization, image data that can be used for subsequent calculations is obtained.

Among them, the specific process of linear normalization is as follows:

Among them, M represents the total number of tilt angles, j represents the serial number of the tilt angle, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, and b _j (u, v) represents the jth Images obtained from experiments at several angles.

In step S102, based on the image data, points are scattered at equal intervals in the real space and linear accumulation is used to obtain calculated images at each tilt angle of multiple tilt angles.

As a possible implementation method, the embodiment of the present application can first set up a discretized and limited number of local orbit functions in real space based on the image data obtained in the above steps as the initial input of the iterative process, where each The strength H of the local orbital function can be set to 1e ^-5 , the width can be set to 1.4, the sample drift at each angle (u _j , v _j ) and the offset of the Euler rotation angle Both can be set to 0.

Specifically, the embodiment of the present application can express the three-dimensional coordinates of the local orbit function center at different angles as:

in, is the Euler angle of the distribution corresponding to the j-th angle around the x-axis, y-axis, and z-axis, (x _i , y _i , z _i ) is the i-th local orbit function without rotation (all three Euler angles are is the central three-dimensional position coordinate when is 0), (u _ij , v _ij , w _ij ) is the central three-dimensional position coordinate of the i-th local orbit function at the j-th angle.

Further, this embodiment of the application can calculate the coordinates of the local orbit center at each angle:

Furthermore, embodiments of the present application can represent the local orbit function as a three-dimensional Gaussian function based on the above coordinates, and calculate the value of each local orbit function at each angle:

Among them, D _ij represents the value of the i-th local orbit function at the j-th angle in the real space (u, v, w), H _i and B _i are the parameters to be optimized, representing the i-th local orbit function respectively. The intensity and width of the orbit function, (u, v, w) are the three-dimensional position coordinates of the real space, (u _ij , v _ij , w _ij ) are the center three-dimensional position coordinates of the i-th local orbit function at the j-th angle , (u _j ,v _j ,w _j ) is the three-dimensional drift of the local orbit function center relative to the true position (u _ij ,v _ij ,w _ij ) at the jth angle.

Then through linear accumulation, the calculation image can be obtained in the embodiment of this application:

Among them, N represents the total number of local orbit functions, and D _ij represents the value of the i-th local orbit function at the j-th angle in the real space (u, v, w).

In step S103, the loss function is calculated based on the calculation image at each tilt angle, and the gradient of the loss function with respect to the parameters to be optimized is obtained, and the parameters to be optimized are optimized according to the gradient, atoms that meet the preset conditions are screened out, and new atoms are recalculated. The loss function until the convergence condition is met, the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function are reconstructed in real space, and the three-dimensional reconstruction result is obtained.

During the actual execution process, the embodiments of the present application can use the gradient optimization algorithm to solve the parameters, and reconstruct the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function in real space through multiple calculations and screenings. Obtain three-dimensional reconstruction results.

Specifically, embodiments of the present application can use the calculation images calculated in the above steps to calculate the loss function, and then obtain the gradient of the loss function with respect to the parameters to be optimized, and use the gradient optimization algorithm to optimize the parameters to be optimized, and then filter out the parameters that satisfy the preset The atom of the condition is calculated repeatedly until the convergence condition is met, thereby reconstructing the three-dimensional space coordinates of the center of the local orbital function and the shape of the local orbital function in real space, and obtaining the three-dimensional reconstruction result.

The embodiments of the present application can directly obtain the three-dimensional coordinates of the sample atoms, skipping the process of first obtaining the three-dimensional density matrix and then searching for peaks to obtain the three-dimensional coordinates. This not only reduces the requirements for hardware, but also eliminates cumbersome human intervention in the process of peak searching. This saves labor costs, and at the same time, during the iterative process, sample drift and mechanical tilt errors of the sample stage can be corrected, thereby improving the accuracy of three-dimensional coordinate reconstruction.

It should be noted that the preset conditions can be set by those skilled in the art according to actual conditions, and are not specifically limited here.

Optionally, in one embodiment of this application, the calculation formula of the loss function is:

Specifically, the embodiment of the present application can calculate the loss function through the calculation image in the above steps, and write the loss function as a function about the calculation image and the experimental image:

Among them, W represents the loss function, M represents the total number of tilt angles, j represents the serial number of the tilt angle, i represents the local orbit function serial number, P represents the side length of the image, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, f _j (u, v) represents the image calculated at the j-th angle, b _j (u, v) represents the experimental image at the j-th angle, m _j (u ,v) is the difference between the calculated image and the experimental image at the jth angle.

Optionally, in one embodiment of the present application, the parameters to be optimized include the three-dimensional spatial coordinates of the center of each local orbit function, parameters describing its shape, three Euler angles corresponding to each corner, and At least one of the drift of the sample and the mechanical tilt deviation of the sample stage.

After obtaining the loss function, the embodiment of the present application can calculate the loss function with respect to the three-dimensional coordinates (x _i , y _i , z _i ) of the center of the local orbit function, the intensity _Hi , the width Bi _, and the sample drift amount at each angle ( u _j ,v _j ), sample stage angle deviation gradient of equal parameters.

Further, this embodiment of the present application can use the gradient obtained by calculation to update the target parameters:

in, and is the learning rate of each parameter.

It should be noted that the gradient can be obtained using a software library with automatic derivation function or through analytical expressions.

Optionally, in one embodiment of the present application, screening out atoms that meet preset conditions includes: after iteratively updating the parameters in each step, detecting the parameters of all local orbital functions and detecting any local orbital function. When the parameters of are less than the threshold obtained from all local orbit function parameters, while deleting any local orbit function, establish a binary tree to obtain the local orbit function pair index whose center distance is less than the preset pixel, and delete the local orbit function Any local orbital function in the orbital function pair; after each step iteratively updates the parameters and deletes the local orbital function, the parameters of each local orbital function are reduced to a preset multiple with a preset probability, and set Protection time, so that filtering operations and deletion operations are not allowed during the protection time.

It can be understood that screening out atoms that meet preset conditions includes deleting redundant local orbital functions and screening local orbital functions.

Among them, the method of deleting redundant local orbital functions is: after updating the parameters iteratively at each step, check the parameters H of all local orbital functions. If the parameter H of a certain local orbital function is less than the parameter H of all local orbital functions. When the threshold value is obtained, the local orbit function is deleted. At the same time, by establishing a binary tree, the local orbit function pair index of the local orbit function center distance from the preset pixel is obtained, and any one of the local orbit function pairs is deleted. Orbital function.

It should be noted that the threshold can be set by those skilled in the art according to the actual situation, or can be set as a reference value, such as 0.01 times the maximum value of all local orbit function parameters H; the preset pixel can be set by those skilled in the art according to Set according to the actual situation, or set to a reference value, such as 2 pixels.

The method of screening local orbital functions is as follows: after iteratively updating the parameters and deleting the local orbital functions at each step, the preset probability of the parameter H of each local orbital function is reduced to a preset multiple, and a protection time is set. Local orbital functions cannot be filtered and deleted during the protection time.

It should be noted that the preset probability and preset multiple can be set by those skilled in the art according to the actual situation, or can be set as a reference value. For example, the parameter H of each local orbit function is reduced to 0.1 times with a probability of 0.02. ; The protection event can be set by those skilled in the art according to the actual situation, or can be set as a reference value, such as setting the protection time T=50.

The working principle of the three-dimensional reconstruction method of the local orbit function in the embodiment of the present application will be described in detail with a specific embodiment as shown in FIGS. 2 to 9 .

As shown in Figure 2, taking the reconstruction of a small particle composed of 10,000 atoms as an example, the embodiment of the present application includes the following steps:

Step S201: Collect image data. As shown in Figure 3, the embodiment of this application needs to reconstruct a small particle composed of 10,000 atoms, and obtain its simulation by tilting at angles of ±25°, ±20°, ±15°, ±5°, and 0°. Image, columns 1, 2, and 3 in Figure 3 correspond to images of 25°, 0°, and -25° respectively.

Step S202: Calculate the image. Embodiments of the present application can scatter points at equal intervals in real space based on image data and use linear accumulation to obtain calculated images at each tilt angle of multiple tilt angles. As shown in Figure 4, the initial scatter point set is a cylinder composed of equally spaced atoms. Referring to Figure 5, using the formula The calculated images at various angles at the initial iteration are obtained, among which the 1st, 2nd, and 3rd columns correspond to images of 25°, 0°, and -25° respectively.

Step S203: Calculate the loss function and iteratively update the target parameters. The embodiment of this application can calculate the loss function and the gradient of the target parameters, and use the following formula to iteratively update the target parameters:

in, and is the learning rate of each parameter.

Step S204: Delete and filter local orbital functions. Further, the embodiment of the present application can delete the local orbit function whose intensity H is less than the strongest local orbit function intensity Hmax*0.01, and at the same time reset the local orbit function with a probability of 0.02, so that the local orbit function intensity H decreases It is 0.1 times the original value and sets the protection period T=50.

By cyclically calculating the loss function, updating each target parameter, and deleting and filtering local orbital functions until the number of local orbital functions converges, three-dimensional coordinate reconstruction is achieved.

Among them, the line chart of the value of the loss function with the iterative process is shown in Figure 6; the schematic diagram of the atomic model obtained during final convergence is shown in Figure 7; the difference reference diagram between the calculated image and the experimental image is shown in Figure 8, where , the first column is the calculation diagram, the second column is the experimental diagram, the third column is the difference between the two, the first row is the data when the tilt angle is 25°, the second row is the data when the tilt angle is 20°; The distance histogram between the calculated atomic coordinates and the real coordinates is shown in Figure 9.

According to the three-dimensional reconstruction method of the local orbit function proposed in the embodiment of the present application, based on the collected image data of the sample at multiple tilt angles, calculated images at multiple tilt angles are obtained, and then the loss of the calculated image is calculated function to obtain the gradient of the optimization parameters, thereby obtaining the optimization parameters. After repeated screening and calculation, until the loss function meets the convergence conditions, the three-dimensional reconstruction result is obtained, which can streamline the three-dimensional coordinate reconstruction process, not only reducing the need for hardware requirements, it can also reduce tedious human intervention and save labor costs. At the same time, during the iteration process, it can also correct sample drift and mechanical tilt errors of the sample stage, thereby improving the accuracy of three-dimensional coordinate reconstruction. This solves the problem in related technologies that the three-dimensional coordinates of atoms can only be obtained from the reconstructed three-dimensional density matrix, and the errors cannot be corrected, resulting in high hardware requirements for the reconstruction process and the accuracy of the reconstructed three-dimensional coordinates. Poor technical issues.

Next, the three-dimensional reconstruction device of the local orbit function proposed according to the embodiment of the present application will be described with reference to the accompanying drawings.

Figure 10 is a block schematic diagram of a three-dimensional reconstruction device of a local orbit function according to an embodiment of the present application.

As shown in Figure 10, the three-dimensional reconstruction device 10 of the local orbit function includes: an acquisition module 100, an accumulation module 200 and a reconstruction module 300.

Specifically, the acquisition module 100 is used to acquire image data of samples at multiple tilt angles.

The accumulation module 200 is used to scatter points at equal intervals in the real space based on the image data and use linear accumulation to obtain calculated images at each tilt angle of multiple tilt angles.

The reconstruction module 300 is used to calculate the loss function based on the calculated image at each tilt angle, obtain the gradient of the loss function with respect to the parameters to be optimized, and optimize the parameters to be optimized based on the gradient, screen out atoms that meet the preset conditions, and re- count Calculate a new loss function until the convergence conditions are met, reconstruct the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function in real space, and obtain the three-dimensional reconstruction result.

Optionally, in one embodiment of the present application, the collection module 100 includes: an acquisition unit and a noise reduction unit.

Among them, the acquisition unit is used to acquire initial image data of the sample at multiple tilt angles.

The noise reduction unit is used to perform centering and noise reduction processing on the initial image data, and normalizes the processed image to obtain image data.

Optionally, in one embodiment of the present application, the reconstruction module 300 includes: a detection unit and a protection unit.

Among them, the detection unit is used to detect the parameter deletion unit of all local orbit functions after iteratively updating the parameters in each step, and is used to detect that the parameters of any local orbit function are smaller than those obtained from all the local orbit function parameters. At the threshold, while deleting any local orbit function, establish a binary tree to obtain the local orbit function pair index whose center distance is smaller than the preset pixel, and delete any local orbit function in the local orbit function pair.

The protection unit is used to reduce the parameters of each local orbit function to a preset multiple with a preset probability after iteratively updating the parameters and deleting the local orbit function at each step, and sets the protection time so that during the protection During this time, filtering operations and deletion operations are not allowed.

It should be noted that the foregoing explanation of the embodiment of the three-dimensional reconstruction method of the local orbit function also applies to the three-dimensional reconstruction device of the local orbit function in this embodiment, and will not be described again here.

According to the three-dimensional reconstruction device of the local orbit function proposed in the embodiment of the present application, based on the collected image data of the sample at multiple tilt angles, calculated images at multiple tilt angles are obtained, and then the loss of the calculated image is calculated function to obtain the gradient of the optimization parameters, thereby obtaining the optimization parameters. After repeated screening and calculation, until the loss function meets the convergence conditions, the three-dimensional reconstruction result is obtained, which can streamline the three-dimensional coordinate reconstruction process, not only reducing the need for hardware requirements, it can also reduce tedious human intervention and save labor costs. At the same time, during the iteration process, it can also correct sample drift and mechanical tilt errors of the sample stage, thereby improving the accuracy of three-dimensional coordinate reconstruction. This solves the problem in related technologies that the three-dimensional coordinates of atoms can only be obtained from the reconstructed three-dimensional density matrix, and the errors cannot be corrected, resulting in heavy errors. The construction process has high hardware requirements and the accuracy of the reconstructed three-dimensional coordinates is poor.

Figure 11 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include:

Memory 1101, processor 1102, and a computer program stored on memory 1101 and executable on processor 1102.

When the processor 1102 executes the program, it implements the three-dimensional reconstruction method of the local orbit function provided in the above embodiment.

Furthermore, electronic equipment also includes:

Communication interface 1103 is used for communication between the memory 1101 and the processor 1102.

Memory 1101 is used to store computer programs that can run on the processor 1102.

The memory 1101 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

If the memory 1101, the processor 1102 and the communication interface 1103 are implemented independently, the communication interface 1103, the memory 1101 and the processor 1102 can be connected to each other through a bus and complete communication with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in Figure 11, but it does not mean that there is only one bus or one type of bus.

Optionally, in terms of specific implementation, if the memory 1101, the processor 1102 and the communication interface 1103 are integrated on one chip, the memory 1101, the processor 1102 and the communication interface 1103 can communicate with each other through the internal interface.

The processor 1102 may be a central processing unit (Central Processing Unit, CPU for short), or an Application Specific Integrated Circuit (ASIC for short), or one or more processors configured to implement the embodiments of the present application. integrated circuit.

This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above three-dimensional reconstruction method of the local orbital function is implemented.

In the description of this specification, reference to the terms "one embodiment,""someembodiments,""anexample,""specificexamples," or "some examples" or the like means that specific features are described in connection with the embodiment or example. , structures, materials or features are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. In addition, to the extent not inconsistent with each other, the technical The skilled person may combine and combine the different embodiments or examples and features of different embodiments or examples described in this specification.

In addition, the terms “first” and “second” are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically limited.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments, or portions of code that include one or more executable instructions for implementing customized logical functions or steps of the process. , and the scope of the preferred embodiments of the present application includes additional implementations in which functions may be performed out of the order shown or discussed, including in a substantially simultaneous manner or in the reverse order, depending on the functionality involved, which shall It should be understood by those skilled in the technical field to which the embodiments of this application belong.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any computer-readable medium, For use by, or in combination with, instruction execution systems, devices or devices (such as computer-based systems, systems including processors or other systems that can fetch instructions from and execute instructions from the instruction execution system, device or device) or equipment. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or N wires (electronic device), portable computer disk cartridge (magnetic device), random access memory (RAM), Read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, and subsequently edited, interpreted, or otherwise suitable as necessary. process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present application can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if it is implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: discrete logic gate circuits with logic functions for implementing data signals; Logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

Those of ordinary skill in the art can understand that all or part of the steps involved in implementing the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When executed, the program includes one of the steps of the method embodiment or a combination thereof.

In addition, each functional unit in various embodiments of the present application can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software function modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

The storage media mentioned above can be read-only memory, magnetic disks or optical disks, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above-mentioned embodiments are illustrative and cannot be understood as limitations of the present application. Those of ordinary skill in the art can make modifications to the above-mentioned embodiments within the scope of the present application. The embodiments are subject to changes, modifications, substitutions and variations.

Claims

A three-dimensional reconstruction method of local orbital function, which is characterized by including the following steps:

Collect image data of samples at multiple tilt angles;

Based on the image data, scatter points at equal intervals in real space and use linear accumulation to obtain calculated images at each tilt angle of the multiple tilt angles; and

Calculate the loss function based on the calculated image at each tilt angle, obtain the gradient of the loss function with respect to the parameters to be optimized, optimize the parameters to be optimized based on the gradient, screen out atoms that meet the preset conditions, and recalculate The new loss function is used until the convergence conditions are met, and the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function are reconstructed in the real space to obtain a three-dimensional reconstruction result.
The method according to claim 1, characterized in that collecting image data of the sample at multiple tilt angles includes:

Obtain initial image data of the sample at the multiple tilt angles;

The initial image data is subjected to centering and noise reduction processing, and the processed image is normalized to obtain the image data.
The method according to claim 1, characterized in that the calculation formula of the loss function is:

Among them, W represents the loss function, M represents the total number of tilt angles, j represents the serial number of the tilt angle, i represents the local orbit function serial number, P represents the side length of the image, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, f j (u, v) represents the image calculated at the j-th angle, and b j (u, v) represents the image obtained experimentally at the j-th angle.
The method of claim 1, wherein screening out atoms that meet preset conditions includes:

After the parameters are updated iteratively at each step, the parameters of all local orbital functions are detected.

When it is detected that the parameters of any local orbit function are less than the threshold value obtained from all the parameters of the local orbit function, while deleting the any local orbit function, the center distance of the local orbit function is less than the preset value by establishing a binary tree. Set the local orbit function pair index of the pixel, and delete any local orbit function in the local orbit function pair;

After each step iteratively updates the parameters and deletes the local orbit function, the parameters of each local orbit function are reduced to a preset multiple with a preset probability, and a protection time is set so that within the protection time , filtering operations and deletion operations are not allowed.
The method according to any one of claims 1 to 4, characterized in that the parameters to be optimized include the three-dimensional spatial coordinates of the center of each local orbit function, parameters describing its shape, and three European parameters corresponding to each corner. At least one of the pull angle, the drift of the sample at each angle, and the mechanical tilt deviation of the sample stage.
A three-dimensional reconstruction device of local orbital function, which is characterized by including:

Acquisition module, used to collect image data of samples at multiple tilt angles;

An accumulation module, configured to scatter points at equal intervals in real space based on the image data and use linear accumulation to obtain the calculated image at each tilt angle of the plurality of tilt angles; and

A reconstruction module, configured to calculate a loss function based on the calculated image at each tilt angle, obtain the gradient of the loss function with respect to the parameters to be optimized, and optimize the parameters to be optimized according to the gradient, and filter out the parameters that satisfy the preset Condition atoms, recalculate the new loss function until the convergence condition is met, reconstruct the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function in the real space, and obtain the three-dimensional reconstruction result.
The device according to claim 6, characterized in that the collection module includes:

An acquisition unit, configured to acquire initial image data of the sample at the multiple tilt angles;

A noise reduction unit is used to perform centering and noise reduction processing on the initial image data, and normalize the processed image to obtain the image data.
The device according to claim 6, characterized in that the calculation formula of the loss function is:

Among them, W represents the loss function, M represents the total number of tilt angles, j represents the serial number of the tilt angle, i represents the local orbit function serial number, P represents the side length of the image, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, f j (u, v) represents the image calculated at the j-th angle, and b j (u, v) represents the image obtained experimentally at the j-th angle.
The device according to claim 6, characterized in that the reconstruction module includes:

The detection unit is used to detect the parameters of all local orbital functions after the parameters are updated iteratively at each step.

A deletion unit configured to delete any local orbit function and obtain the local orbit by establishing a binary tree when it is detected that the parameters of any local orbit function are less than the threshold obtained from all the local orbit function parameters. If the function center distance is smaller than the index of the local orbit function pair of the preset pixel, delete any local orbit function in the local orbit function pair;

A protection unit, used to reduce the parameters of each local orbit function to a preset multiple with a preset probability after iteratively updating the parameters and deleting the local orbit function at each step, and setting a protection time, so that Filtering operations and deletion operations are not allowed during the protection time.
The device according to any one of claims 6 to 9, characterized in that the parameters to be optimized include the three-dimensional spatial coordinates of the center of each local orbit function, parameters describing its shape, and three European parameters corresponding to each corner. At least one of the pull angle, the drift of the sample at each angle, and the mechanical tilt deviation of the sample stage.
An electronic device, characterized in that it includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the method of claim 1 -5 Ren The three-dimensional reconstruction method of the local orbital function described in one item.
A computer-readable storage medium with a computer program stored thereon, characterized in that the program is executed by a processor to implement the three-dimensional reconstruction of the local orbital function as claimed in any one of claims 1-5. method.