CN114648611A - Three-dimensional reconstruction method and device of local orbit function - Google Patents

Three-dimensional reconstruction method and device of local orbit function Download PDF

Info

Publication number
CN114648611A
CN114648611A CN202210381325.3A CN202210381325A CN114648611A CN 114648611 A CN114648611 A CN 114648611A CN 202210381325 A CN202210381325 A CN 202210381325A CN 114648611 A CN114648611 A CN 114648611A
Authority
CN
China
Prior art keywords
function
local
image
dimensional
parameters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210381325.3A
Other languages
Chinese (zh)
Other versions
CN114648611B (en
Inventor
于荣
毛梁泽
程志英
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN202210381325.3A priority Critical patent/CN114648611B/en
Publication of CN114648611A publication Critical patent/CN114648611A/en
Priority to PCT/CN2023/080253 priority patent/WO2023197785A1/en
Application granted granted Critical
Publication of CN114648611B publication Critical patent/CN114648611B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Theoretical Computer Science (AREA)
  • Pure & Applied Mathematics (AREA)
  • Software Systems (AREA)
  • Computational Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Algebra (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Processing (AREA)

Abstract

The application discloses a three-dimensional reconstruction method and a three-dimensional reconstruction device for a local orbit function, wherein the method comprises the following steps: acquiring image data of samples under a plurality of tilting angles, scattering points at equal intervals in real space and utilizing linear accumulation to obtain a calculation image under each tilting angle of the plurality of tilting angles, further acquiring the gradient of a loss function relative to a parameter to be optimized, screening out atoms meeting preset conditions according to the parameter to be optimized in the gradient optimization, recalculating a new loss function until a convergence condition is met, reconstructing a three-dimensional space coordinate of the center of a local orbit function and the shape of the local orbit function in real space, and obtaining a three-dimensional reconstruction result. Therefore, the technical problems that in the related technology, the three-dimensional coordinates of atoms can only be acquired from the reconstructed three-dimensional density matrix, errors cannot be corrected, the requirement of the reconstruction process on hardware is high, and the accuracy of the reconstructed three-dimensional coordinates is poor are solved.

Description

Three-dimensional reconstruction method and device of local orbit function
Technical Field
The present application relates to the field of three-dimensional imaging technologies, and in particular, to a method and an apparatus for three-dimensional reconstruction of a local orbit function.
Background
In general, the electron microscope photographs of samples contain information which is rich in samples and difficult to visually acquire, and the three-dimensional structure information of the samples acquired from a series of electron microscope projection photographs of the samples by applying a three-dimensional reconstruction algorithm is very helpful for understanding the relationship between material components and performance at a fundamental level.
In recent years, with the development of data acquisition methods, iterative three-dimensional reconstruction algorithms and post-processing methods, AET (Atomic Electrical Tomography) has become a powerful tool for three-dimensional and four-dimensional Atomic scale structure characterization, which provides the ability to correlate material structure and properties at the Atomic level, and AET in the related art can determine three-dimensional Atomic coordinates and element species with sub-angstrom accuracy and reveal their Atomic scale time evolution in a dynamic process.
However, in the related art, the AET algorithm obtains the three-dimensional coordinates of the atoms from the reconstructed three-dimensional density matrix by peak searching, the requirement on hardware is high, and in the peak searching process, tedious manual intervention is required, which is difficult to avoid errors caused by the manual intervention, and meanwhile, when the three-dimensional coordinates of the atoms are directly reconstructed, sample drift and mechanical tilt errors of a sample stage existing when images are acquired cannot be corrected, so that the accuracy is poor and needs to be improved.
Disclosure of Invention
The application provides a three-dimensional reconstruction method and a three-dimensional reconstruction device for a local orbit function, which are used for solving the technical problems that in the related technology, the three-dimensional coordinates of atoms can only be obtained from a reconstructed three-dimensional density matrix, errors cannot be corrected, the requirement of a reconstruction process on hardware is high, and the accuracy of the reconstructed three-dimensional coordinates is poor.
An embodiment of a first aspect of the present application provides a three-dimensional reconstruction method for a local orbit function, including the following steps: acquiring image data of a sample under a plurality of tilting angles; based on the image data, scattering points at equal intervals in real space and utilizing linear accumulation to obtain a calculation image under each tilting angle of the plurality of tilting angles; and calculating a loss function according to the calculation image at each tilting angle, acquiring the gradient of the loss function about the parameter to be optimized, optimizing the parameter to be optimized according to the gradient, screening out atoms meeting preset conditions, recalculating a new loss function until meeting convergence conditions, and reconstructing the three-dimensional space coordinate of the center of the local orbit function and the shape of the local orbit function in the real space to obtain a three-dimensional reconstruction result.
Optionally, in an embodiment of the present application, acquiring image data of the sample at the plurality of tilt angles includes: acquiring initial image data of the sample at the plurality of tilting angles; and carrying out centering and denoising processing on the initial image data, and normalizing the processed image to obtain the image data.
Optionally, in an embodiment of the present application, the calculation formula of the loss function is:
Figure BDA0003591922700000021
wherein W represents a loss function, M represents the total number of tilting angles, j represents the serial number of the tilting angles, i represents the serial number of the local orbit function, P represents the side length of the image, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, f represents the linear coordinate of each pixel in the image, andj(u, v) represents the image calculated at the j-th angle, bj(u, v) represents the image obtained from the experiment at the j-th angle.
Optionally, in an embodiment of the present application, the screening out atoms that satisfy a preset condition includes: after the parameters are updated in each step of iteration, detecting the parameters of all the local orbit functions, deleting any local orbit function when detecting that the parameter of any local orbit function is smaller than the threshold value obtained by all the local orbit function parameters, obtaining a local orbit function pair index of which the central distance of the local orbit function is smaller than a preset pixel by establishing a binary tree, and deleting any local orbit function in the local orbit function pair; after the parameters are updated iteratively and the local area orbit functions are deleted, the parameters of each local area orbit function are reduced to preset multiples according to preset probability, and protection time is set, so that the screening operation and the deletion operation are not allowed to be executed within the protection time.
Optionally, in an embodiment of the present application, the parameter to be optimized includes at least one of a three-dimensional space coordinate of a center of each local orbital function, a parameter describing a shape thereof, three euler angles corresponding to each rotation angle, a drift of the sample at each angle, and a mechanical tilt deviation of the sample stage.
The embodiment of the second aspect of the present application provides a three-dimensional reconstruction apparatus of a local orbit function, including: the acquisition module is used for acquiring image data of the samples at a plurality of tilting angles; the accumulation module is used for scattering points at equal intervals in real space based on the image data and obtaining a calculation image under each tilting angle of the plurality of tilting angles by utilizing linear accumulation; and the reconstruction module is used for calculating a loss function according to the calculation image at each tilting angle, acquiring the gradient of the loss function relative to the parameter to be optimized, optimizing the parameter to be optimized according to the gradient, screening out atoms meeting preset conditions, recalculating a new loss function until convergence conditions are met, reconstructing the three-dimensional space coordinate of the center of the local orbit function and the shape of the local orbit function in the real space, and obtaining a three-dimensional reconstruction result.
Optionally, in an embodiment of the present application, the acquisition module includes: an acquisition unit configured to acquire initial image data of the sample at the plurality of tilt angles; and the noise reduction unit is used for carrying out centering and noise reduction on the initial image data and normalizing the processed image to obtain the image data.
Optionally, in an embodiment of the present application, the calculation formula of the loss function is:
Figure BDA0003591922700000022
wherein W represents a loss function, M represents the total number of tilting angles, j represents the serial number of the tilting angles, i represents the serial number of the local orbit function, P represents the side length of the image, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, f represents the linear coordinate of each pixel in the image, andj(u, v) represents the image calculated at the j-th angle, bj(u, v) generationTable j angle experimental images.
Optionally, in an embodiment of the present application, the reconstruction module includes: the detection unit is used for detecting a parameter deleting unit of all the local orbit functions after the parameters are updated in each step in an iterative manner, and is used for deleting any local orbit function when the parameters of the local orbit function are detected to be smaller than the threshold value obtained by all the local orbit function parameters, obtaining a local orbit function pair index of which the central distance of the local orbit function is smaller than a preset pixel by establishing a binary tree, and deleting any local orbit function in the local orbit function pair; and the protection unit is used for reducing the parameters of each local track function to preset multiples according to preset probability after the parameters are updated iteratively and the local track functions are deleted, and setting protection time so that the screening operation and the deletion operation are not allowed to be executed within the protection time.
Optionally, in an embodiment of the present application, the parameter to be optimized includes at least one of a three-dimensional space coordinate of a center of each local orbital function, a parameter describing a shape thereof, three euler angles corresponding to each rotation angle, a drift of the sample at each angle, and a mechanical tilt deviation of the sample stage.
An embodiment of a third aspect of the present application provides an electronic device, including: a memory, a processor and a computer program stored on the memory and operable on the processor, the processor executing the program to implement the three-dimensional reconstruction method of the local orbit function as described in the above embodiments.
A fourth aspect of the present application provides a computer-readable storage medium, which stores computer instructions for causing the computer to execute the three-dimensional reconstruction method of the local area orbit function according to the foregoing embodiment.
The embodiment of the application is based on the image data of the collected samples at a plurality of tilting angles, the calculation images at a plurality of tilting angles are obtained, further, the loss function of the calculation images is calculated, the gradient of the optimized parameters is obtained, the optimized parameters are obtained, after repeated screening and calculation, the convergence condition is met until the loss function, further, the three-dimensional reconstruction result is obtained, the reconstruction process of the three-dimensional coordinate can be simplified, the requirements for hardware are reduced, tedious manual intervention can be reduced, the labor cost is saved, meanwhile, in the iteration process, the mechanical tilting errors of the sample drift and the sample stage can be corrected, and further, the accuracy of three-dimensional coordinate reconstruction is improved. Therefore, the technical problems that in the related technology, the three-dimensional coordinates of atoms can only be acquired from the reconstructed three-dimensional density matrix, errors cannot be corrected, the requirement of the reconstruction process on hardware is high, and the accuracy of the reconstructed three-dimensional coordinates is poor are solved.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a flowchart of a three-dimensional reconstruction method of a local orbit function according to an embodiment of the present application;
FIG. 2 is a flow chart of a method for three-dimensional reconstruction of a local orbit function according to one embodiment of the present application;
FIG. 3 is a schematic diagram of a simulation of 10000 atoms of small particles at 25, 0, and-25 to be reconstructed according to one embodiment of the present application;
FIG. 4 is a schematic diagram of a scatter plot formed by atomic coordinates of an initial input of a three-dimensional reconstruction method of a local orbit function according to an embodiment of the present application;
fig. 5 is a schematic diagram illustrating calculation of the local orbit function at tilting angles of 25 °, 0 ° and-25 ° at the initial input of the method for three-dimensional reconstruction of the local orbit function according to an embodiment of the present application;
FIG. 6 is a broken-line diagram illustrating values of a loss function during an iteration of a method for three-dimensional reconstruction of a local orbit function according to an embodiment of the present application;
FIG. 7 is a schematic diagram of an atomic model obtained after convergence of a three-dimensional reconstruction method of a local orbit function according to an embodiment of the present application;
FIG. 8 is a difference between a computed image and an experimental image at convergence of a three-dimensional reconstruction method of a local orbit function according to an embodiment of the present application;
FIG. 9 is a histogram of distances between atomic coordinates and real coordinates obtained from a convergence calculation of a three-dimensional reconstruction method of a local orbit function according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of a three-dimensional reconstruction apparatus for a local orbit function according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
The following describes a three-dimensional reconstruction method and apparatus for a local orbit function according to an embodiment of the present application with reference to the drawings. Aiming at the technical problems that the related technology mentioned in the center of the background technology can only obtain the three-dimensional coordinates of atoms from the reconstructed three-dimensional density matrix, errors cannot be corrected, the requirement of the reconstruction process on hardware is high, and the accuracy of the reconstructed three-dimensional coordinates is poor, the application provides the three-dimensional reconstruction method of the local orbit function, in the method, the computed images under a plurality of tilting angles are obtained based on the acquired image data of the samples under a plurality of tilting angles, the loss function of the computed images is further computed, the gradient of the optimized parameters is obtained, the optimized parameters are obtained, after repeated screening and computation, the loss function meets the convergence condition, the three-dimensional reconstruction result is further obtained, the reconstruction process of the three-dimensional coordinates can be simplified, the requirement on the hardware is reduced, and the complicated human intervention can be reduced, the labor cost is saved, and meanwhile, in the iteration process, the mechanical tilting error of the sample drift and the sample stage can be corrected, so that the accuracy of three-dimensional coordinate reconstruction is improved. Therefore, the technical problems that in the related technology, the three-dimensional coordinates of atoms can only be acquired from the reconstructed three-dimensional density matrix, errors cannot be corrected, the requirement of the reconstruction process on hardware is high, and the accuracy of the reconstructed three-dimensional coordinates is poor are solved.
Specifically, fig. 1 is a schematic flowchart of a three-dimensional reconstruction method of a local orbit function according to an embodiment of the present application.
As shown in fig. 1, the three-dimensional reconstruction method of the local orbit function includes the following steps:
in step S101, image data of the sample at a plurality of tilt angles is acquired.
In an actual implementation process, the embodiment of the application can shoot the HAADF (High-Angle Annular Dark-Field image) images of the sample under a series of different rotation angles, so that the image data of the sample under a plurality of tilting angles can be obtained, the image data can be conveniently processed subsequently, and the three-dimensional coordinate reconstruction can be performed.
Optionally, in an embodiment of the present application, acquiring image data of the sample at a plurality of tilt angles includes: acquiring initial image data of a sample at a plurality of tilting angles; and carrying out centering and denoising processing on the initial image data, and normalizing the processed image to obtain image data.
Specifically, after the initial image data of the samples at a plurality of tilting angles are obtained through shooting, in order to ensure the accuracy of subsequent three-dimensional coordinate reconstruction, the method and the device can perform neutralization axis and noise reduction processing on the initial image data, facilitate subsequent calculation, avoid the influence of noise on the three-dimensional coordinate reconstruction, and obtain image data which can be used for the subsequent calculation through linear normalization.
The specific process of linear normalization is as follows:
Figure BDA0003591922700000051
wherein M represents a total number of tilting angles, j represents a serial number of tilting angles, u represents an abscissa of each pixel in the image, v represents an ordinate of each pixel in the image, b represents a total number of tilting angles, andj(u, v) represents the image obtained experimentally at the j-th angle.
In step S102, based on the image data, the calculated image for each of the plurality of tilt angles is obtained by performing point scattering at equal intervals in real space and by linear accumulation.
As a possible implementation manner, the embodiment of the present application may first set a discretized, limited number of local orbit functions in a real space with a certain rule as initial inputs of an iterative process based on the image data obtained in the above steps, where the intensity H of each local orbit function may be set to 1e-5The width can be set to 1.4, the amount of sample drift (u) at each anglej,vj) Offset from the Euler angle
Figure BDA0003591922700000052
May be set to 0.
Specifically, the three-dimensional coordinates of the center of the local orbit function at different angles can be expressed as:
Figure BDA0003591922700000061
wherein psij、θj、
Figure BDA0003591922700000062
Euler angles around the x-axis, y-axis, and z-axis for the distribution corresponding to the jth angle, (x)i,yi,zi) For the central three-dimensional position coordinates of the ith local orbit function when it is not rotated (all three Euler angles are 0), (u)ij,vij,wij) The central three-dimensional position coordinate of the ith local orbit function at the jth angle is obtained.
Further, the embodiment of the present application may calculate coordinates of the center of the local area orbit at each angle:
Figure BDA0003591922700000063
furthermore, in the embodiment of the present application, the local orbit function may be expressed as a three-dimensional gaussian function according to the coordinates, and a value of each local orbit function at each angle is calculated:
Figure BDA0003591922700000064
wherein D isijRepresents the value of the ith local orbital function at the j-th angle at the real space (u, v, w) position, HiAnd BiRespectively representing the intensity and the width of the ith local orbit function for the parameters to be optimized, (u, v, w) are real space three-dimensional position coordinates, (uij,vij,wij) For the central three-dimensional position coordinate of the ith local orbit function at the jth angle, (u)j,vj,wj) For the local orbital function center at angle j relative to the true position (u)ij,vij,wij) Is shifted in three-dimensional directions.
Through linear accumulation, the embodiment of the application can obtain a calculation image:
Figure BDA0003591922700000065
wherein N represents the total number of the local orbital functions, DijRepresenting the value of the ith local orbit function at the real space (u, v, w) position at the jth angle.
In step S103, a loss function is calculated according to the calculation image at each tilt angle, a gradient of the loss function with respect to the parameter to be optimized is obtained, the parameter to be optimized is optimized according to the gradient, atoms satisfying a preset condition are screened out, a new loss function is recalculated until a convergence condition is satisfied, a three-dimensional space coordinate of the center of the local orbit function and a shape of the local orbit function are reconstructed in a real space, and a three-dimensional reconstruction result is obtained.
In the actual execution process, the embodiment of the application can utilize the optimization algorithm of the gradient to solve the parameters, and reconstruct the three-dimensional space coordinates of the center of the local orbit function and the shape of the local orbit function in the real space through multiple times of calculation and screening to obtain a three-dimensional reconstruction result.
Specifically, the embodiment of the present application may calculate a loss function by using the calculation image obtained through calculation in the above step, further obtain a gradient of the loss function with respect to a parameter to be optimized, optimize the parameter to be optimized by using an optimization algorithm of the gradient, further screen out an atom that satisfies a preset condition, and repeatedly calculate the loss function until a convergence condition is satisfied, thereby reconstructing a three-dimensional space coordinate of a center of the local orbit function and a shape of the local orbit function in a real space, and obtaining a three-dimensional reconstruction result.
The method and the device can directly obtain the three-dimensional coordinates of the sample atoms, skip the process of obtaining the three-dimensional density matrix and then searching the peak to obtain the three-dimensional coordinates, not only reduce the requirements on hardware, but also remove the fussy human intervention in the peak searching process, save labor cost, and simultaneously can correct the mechanical tilting error of sample drift and a sample stage in the iteration process, thereby improving the accuracy of three-dimensional coordinate reconstruction.
It should be noted that the preset condition can be set by those skilled in the art according to practical situations, and is not limited specifically herein.
Optionally, in an embodiment of the present application, the calculation formula of the loss function is:
Figure BDA0003591922700000071
wherein W represents a loss function, M represents the total number of tilting angles, j represents the serial number of the tilting angles, i represents the serial number of the local orbit function, P represents the side length of the image, u represents the abscissa of each pixel in the image, and v represents each pixel in the imageOrdinate of individual pixel, fj(u, v) represents the image calculated at the j-th angle, bj(u, v) represents the image obtained from the experiment at the j-th angle.
Specifically, the embodiment of the present application may calculate the loss function through the calculation image in the above step, and write the loss function as a function with respect to the calculation image and the experimental image:
Figure BDA0003591922700000072
wherein W represents a loss function, M represents the total number of tilting angles, j represents the serial number of the tilting angles, i represents the serial number of the local orbit function, P represents the side length of the image, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, f represents the linear coordinate of each pixel in the image, andj(u, v) represents the image calculated at the j-th angle, bj(u, v) represents the image obtained experimentally at the j-th angle, mj(u, v) is the difference between the calculated image and the experimental image at the j-th angle.
Optionally, in an embodiment of the present application, the parameter to be optimized includes a three-dimensional space coordinate of a center of each local orbital function, a parameter describing a shape thereof, three euler angles corresponding to each rotation angle, at least one of a drift of the sample at each angle, and a mechanical tilt deviation of the sample stage.
After obtaining the loss function, the embodiment of the application may find the three-dimensional coordinate (x) of the loss function with respect to the center of the local orbit functioni,yi,zi) Strength HiWidth BiAmount of sample drift at each angle (u)j,uj) Angle deviation psi of sample stagej、θj、
Figure BDA0003591922700000073
Isoparametric gradients.
Further, the embodiment of the present application may update the target parameter by using the calculated gradient:
Figure BDA0003591922700000074
Figure BDA0003591922700000081
Figure BDA0003591922700000082
Figure BDA0003591922700000083
Figure BDA0003591922700000084
Figure BDA0003591922700000085
Figure BDA0003591922700000086
Figure BDA0003591922700000087
Figure BDA0003591922700000088
Figure BDA0003591922700000089
wherein the content of the first and second substances,
Figure BDA00035919227000000810
and
Figure BDA00035919227000000811
is the learning rate of each parameter.
It should be noted that the gradient may be obtained by using a software library with an automatic derivation function, or by analyzing an expression.
Optionally, in an embodiment of the present application, the screening out the atoms that satisfy the preset condition includes: after the parameters are updated in each step of iteration, detecting the parameters of all the local orbit functions, deleting any local orbit function when detecting that the parameter of any local orbit function is smaller than the threshold value obtained by all the local orbit function parameters, obtaining a local orbit function pair index of which the central distance of the local orbit function is smaller than a preset pixel by establishing a binary tree, and deleting any local orbit function in the local orbit function pair; after the parameters are updated and the local area orbit functions are deleted in each step in an iterative mode, the parameters of each local area orbit function are reduced to preset multiples according to preset probability, and protection time is set, so that screening operation and deleting operation are not allowed to be executed within the protection time.
It is understood that the screening of atoms satisfying the predetermined condition includes deleting the redundant local orbit function and screening the local orbit function.
The method for deleting the redundant local area track function comprises the following steps: after the parameters are updated in each step, checking the parameters H of all the local orbit functions, deleting the local orbit function if the parameter H of a certain local orbit function is smaller than a threshold value obtained by all the local orbit function parameters, obtaining a local orbit function pair index of the central distance of the local orbit function from a preset pixel by a method of establishing a binary tree, and deleting any one local orbit function in the local orbit function pair.
It should be noted that the threshold may be set by those skilled in the art according to actual situations, and may also be set as a reference value, such as 0.01 times of the maximum value of all local orbit function parameters H; the preset pixels can be set by those skilled in the art according to actual conditions, and can also be set as a reference value, such as 2 pixels.
The method for screening the local area orbit function comprises the following steps: after the parameters are updated iteratively and the local orbit functions are deleted in each step, the preset probability of the parameters H of each local orbit function is reduced to a preset multiple, and the protection time is set, so that the local orbit functions can not be screened and deleted within the protection time.
It should be noted that the preset probability and the preset multiple may be set by those skilled in the art according to actual situations, or may be set as reference values, for example, the parameter H of each local area orbit function is reduced to 0.1 times with a probability of 0.02; the protection event may be set by those skilled in the art according to actual conditions, and may also be set as a reference value, for example, setting the protection time T to 50.
The working principle of the three-dimensional reconstruction method of the local orbit function according to the embodiment of the present application is described in detail with reference to fig. 2 to fig. 9.
As shown in fig. 2, taking reconstruction of a small particle composed of 10000 atoms as an example, the embodiment of the present application includes the following steps:
step S201: image data is collected. As shown in fig. 3, the present example requires reconstruction of 10000 atomic small particles, and simulated images thereof are obtained by tilting at angles of ± 25 °, ± 20 °, ± 15 °, ± 5 °, and 0 °, and the 1 st, 2 nd, and 3 rd columns in fig. 3 correspond to 25 °, 0 °, and-25 ° images, respectively.
Step S202: an image is calculated. According to the embodiment of the application, the scattering points are carried out at equal intervals in real space based on the image data, and the calculation images of a plurality of tilting angles under each tilting angle are obtained by linear accumulation. As shown in FIG. 4, the initial set of scattering points is a cylinder of equally spaced atoms and is shown with reference to FIG. 5 using the formula
Figure BDA0003591922700000091
And obtaining the calculation images under each angle at the initial iteration, wherein the 1 st column, the 2 nd column and the 3 rd column respectively correspond to the images of 25 degrees, 0 degrees and-25 degrees.
Step S203: and calculating a loss function, and performing iterative updating on the target parameters. The embodiment of the application can calculate the lossFunction(s)
Figure BDA0003591922700000092
And the gradient of the target parameter, and the target parameter is updated iteratively by using the following formula:
Figure BDA0003591922700000093
Figure BDA0003591922700000094
Figure BDA0003591922700000095
Figure BDA0003591922700000096
Figure BDA0003591922700000097
Figure BDA0003591922700000098
Figure BDA0003591922700000099
Figure BDA00035919227000000910
Figure BDA0003591922700000101
Figure BDA0003591922700000102
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003591922700000103
and
Figure BDA0003591922700000104
is the learning rate of each parameter.
Step S204: and deleting and screening the local area track function. Further, in the embodiment of the present application, the local area track function with the intensity H smaller than the intensity Hmax of the strongest local area track function by 0.01 may be deleted, and the local area track function may be reset with a probability of 0.02, so that the intensity H of the local area track function is reduced to 0.1 times of the original intensity H, and the protection period T is set to 50.
And (4) calculating a loss function circularly, updating each target parameter, deleting and screening the operation of the local orbit function until the number of the local orbit functions is converged, and realizing the reconstruction of the three-dimensional coordinate.
Wherein the line graph of the value of the loss function with the iterative process is shown in fig. 6; a schematic diagram of the atomic model obtained at final convergence is shown in FIG. 7; the difference between the calculated image and the experimental image is shown in fig. 8, in which the 1 st column is a calculation chart, the 2 nd column is an experimental chart, the 3 rd column is a difference therebetween, the 1 st row is data at a tilting angle of 25 °, and the 2 nd row is data at a tilting angle of 20 °; the calculated distance histogram of the atomic coordinates and the real coordinates is shown in fig. 9.
According to the three-dimensional reconstruction method of the local orbit function, the calculation images under the plurality of tilting angles are obtained based on the acquired image data of the samples under the plurality of tilting angles, then the loss function of the calculation images is calculated, the gradient of the optimization parameter is obtained, after repeated screening and calculation, until the loss function meets the convergence condition, the three-dimensional reconstruction result is obtained, the reconstruction process of the three-dimensional coordinate can be simplified, the requirements on hardware are lowered, tedious human intervention can be reduced, labor cost is saved, meanwhile, in the iteration process, the sample drift and the mechanical tilting error of a sample stage can be corrected, and the accuracy of three-dimensional coordinate reconstruction is improved. Therefore, the technical problems that in the related technology, the three-dimensional coordinates of atoms can only be acquired from the reconstructed three-dimensional density matrix, errors cannot be corrected, the requirement of the reconstruction process on hardware is high, and the accuracy of the reconstructed three-dimensional coordinates is poor are solved.
Next, a three-dimensional reconstruction apparatus of a local orbit function according to an embodiment of the present application is described with reference to the drawings.
Fig. 10 is a block diagram illustrating a three-dimensional reconstruction apparatus of a local orbit function according to an embodiment of the present application.
As shown in fig. 10, the three-dimensional reconstruction apparatus 10 for a local orbit function includes: an acquisition module 100, an accumulation module 200 and a reconstruction module 300.
Specifically, the acquisition module 100 is configured to acquire image data of a sample at a plurality of tilting angles.
And an accumulation module 200, configured to perform point scattering at equal intervals in real space based on the image data and obtain a calculation image at each of the plurality of tilting angles by using linear accumulation.
The reconstruction module 300 is configured to calculate a loss function according to the calculation image at each tilt angle, obtain a gradient of the loss function with respect to a parameter to be optimized, optimize the parameter to be optimized according to the gradient, screen out atoms meeting a preset condition, recalculate a new loss function until a convergence condition is met, reconstruct a three-dimensional space coordinate of the center of the local orbit function and a shape of the local orbit function in a real space, and obtain a three-dimensional reconstruction result.
Optionally, in an embodiment of the present application, the acquisition module 100 includes: the device comprises an acquisition unit and a noise reduction unit.
The device comprises an acquisition unit, a display unit and a control unit, wherein the acquisition unit is used for acquiring initial image data of a sample under a plurality of tilting angles.
And the denoising unit is used for carrying out centering and denoising processing on the initial image data, and normalizing the processed image to obtain the image data.
Optionally, in an embodiment of the present application, the calculation formula of the loss function is:
Figure BDA0003591922700000111
wherein W represents a loss function, M represents the total number of tilting angles, j represents the serial number of the tilting angles, i represents the serial number of the local orbit function, P represents the side length of the image, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, f represents the linear coordinate of each pixel in the image, andj(u, v) represents the image calculated at the j-th angle, bj(u, v) represents the image obtained from the experiment at the j-th angle.
Optionally, in an embodiment of the present application, the reconstruction module 300 includes: a detection unit and a protection unit.
The detection unit is used for detecting a parameter deleting unit of all the local area track functions after the parameters are updated in each step in an iterative manner, and is used for deleting any local area track function when the parameters of any local area track function are detected to be smaller than a threshold value obtained by all the local area track function parameters, obtaining a local area track function pair index of which the central distance of the local area track function is smaller than a preset pixel by establishing a binary tree, and deleting any local area track function in the local area track function pair.
And the protection unit is used for reducing the parameters of each local area orbit function to preset multiples according to preset probability after the parameters are updated iteratively and the local area orbit function is deleted at each step, and setting protection time, so that the screening operation and the deletion operation are not allowed to be executed in the protection time.
Optionally, in an embodiment of the present application, the parameter to be optimized includes a three-dimensional space coordinate of a center of each local orbital function, a parameter describing a shape thereof, three euler angles corresponding to each rotation angle, at least one of a drift of the sample at each angle, and a mechanical tilt deviation of the sample stage.
It should be noted that the foregoing explanation of the embodiment of the three-dimensional reconstruction method for a local area orbit function is also applicable to the three-dimensional reconstruction apparatus for a local area orbit function in this embodiment, and details are not repeated here.
According to the three-dimensional reconstruction device of the local orbit function, based on the acquired image data of the samples at a plurality of tilting angles, calculation images at a plurality of tilting angles are obtained, then the loss function of the calculation images is calculated, the gradient of the optimization parameter is obtained, after repeated screening and calculation, until the loss function meets the convergence condition, a three-dimensional reconstruction result is obtained, the reconstruction process of a three-dimensional coordinate can be simplified, the requirements on hardware are reduced, tedious human intervention can be reduced, labor cost is saved, meanwhile, in the iteration process, the mechanical tilting errors of the sample drift and a sample stage can be corrected, and the accuracy of three-dimensional coordinate reconstruction is improved. Therefore, the technical problems that in the related technology, the three-dimensional coordinates of atoms can only be acquired from the reconstructed three-dimensional density matrix, errors cannot be corrected, the requirement of the reconstruction process on hardware is high, and the accuracy of the reconstructed three-dimensional coordinates is poor are solved.
Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device may include:
a memory 1101, a processor 1102, and a computer program stored on the memory 1101 and executable on the processor 1102.
The processor 1102, when executing the program, implements the three-dimensional reconstruction method of the local orbit function provided in the above-described embodiments.
Further, the electronic device further includes:
a communication interface 1103 for communicating between the memory 1101 and the processor 1102.
A memory 1101 for storing computer programs that are executable on the processor 1102.
The memory 1101 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
If the memory 1101, the processor 1102 and the communication interface 1103 are implemented independently, the communication interface 1103, the memory 1101 and the processor 1102 may be connected to each other through a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 11, but this is not intended to represent only one bus or type of bus.
Alternatively, in specific implementation, if the memory 1101, the processor 1102 and the communication interface 1103 are integrated on one chip, the memory 1101, the processor 1102 and the communication interface 1103 may complete communication with each other through an internal interface.
The processor 1102 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present Application.
The present embodiment also provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the three-dimensional reconstruction method of the local orbit function as above.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In the description of the present application, "N" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more N executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of implementing the embodiments of the present application.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or N wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer-readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are exemplary and should not be construed as limiting the present application and that changes, modifications, substitutions and alterations in the above embodiments may be made by those of ordinary skill in the art within the scope of the present application.

Claims (12)

1. A three-dimensional reconstruction method of a local orbit function is characterized by comprising the following steps:
acquiring image data of a sample under a plurality of tilting angles;
based on the image data, scattering points at equal intervals in real space and utilizing linear accumulation to obtain a calculation image under each tilting angle of the plurality of tilting angles; and
calculating a loss function according to the calculation image at each tilting angle, obtaining the gradient of the loss function about the parameter to be optimized, optimizing the parameter to be optimized according to the gradient, screening out atoms meeting preset conditions, recalculating a new loss function until meeting convergence conditions, reconstructing the three-dimensional space coordinate of the center of the local orbit function and the shape of the local orbit function in the real space, and obtaining a three-dimensional reconstruction result.
2. The method of claim 1, wherein acquiring image data of the sample at the plurality of tilt angles comprises:
acquiring initial image data of the sample under the plurality of tilting angles;
and carrying out centering and denoising processing on the initial image data, and normalizing the processed image to obtain the image data.
3. The method of claim 1, wherein the loss function is calculated by:
Figure FDA0003591922690000011
wherein W represents a loss function, M represents a total number of tilt angles, j represents a serial number of tilt angles, i represents a local track function serial number, and P represents an imageU represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, fj(u, v) represents the image calculated at the j-th angle, bj(u, v) represents the image obtained from the experiment at the j-th angle.
4. The method according to claim 1, wherein the screening out atoms satisfying a preset condition comprises:
after the parameters are updated in each step of iteration, the parameters of all the local area orbit functions are detected
When detecting that the parameter of any local orbit function is smaller than the threshold value obtained by all the local orbit function parameters, deleting any local orbit function, obtaining a local orbit function pair index of which the central distance of the local orbit function is smaller than a preset pixel by establishing a binary tree, and deleting any local orbit function in the local orbit function pair;
after the parameters are updated iteratively and the local area track functions are deleted, the parameters of each local area track function are reduced to preset multiples according to preset probability, and protection time is set, so that the screening operation and the deletion operation are not allowed to be executed within the protection time.
5. The method according to any one of claims 1 to 4, wherein the parameters to be optimized comprise at least one of three-dimensional space coordinates of the center of each local orbital function, parameters describing the shape thereof, three Euler angles for each rotation angle, the drift of the sample at each angle, and the mechanical tilt deviation of the sample stage.
6. An apparatus for three-dimensional reconstruction of a local orbit function, comprising:
the acquisition module is used for acquiring image data of the samples under a plurality of tilting angles;
the accumulation module is used for scattering points at equal intervals in real space based on the image data and obtaining a calculation image under each tilting angle of the plurality of tilting angles by utilizing linear accumulation; and
and the reconstruction module is used for calculating a loss function according to the calculation image at each tilting angle, acquiring the gradient of the loss function about the parameter to be optimized, screening out atoms meeting preset conditions according to the parameter to be optimized according to the gradient optimization, recalculating a new loss function until convergence conditions are met, and reconstructing a three-dimensional space coordinate of the center of the local orbit function and the shape of the local orbit function in the real space to obtain a three-dimensional reconstruction result.
7. The apparatus of claim 6, wherein the acquisition module comprises:
an acquisition unit configured to acquire initial image data of the sample at the plurality of tilt angles;
and the noise reduction unit is used for carrying out centering and noise reduction on the initial image data and normalizing the processed image to obtain the image data.
8. The apparatus of claim 6, wherein the loss function is calculated by:
Figure FDA0003591922690000021
wherein W represents a loss function, M represents the total number of tilting angles, j represents the serial number of the tilting angles, i represents the serial number of the local orbit function, P represents the side length of the image, u represents the abscissa of each pixel in the image, v represents the ordinate of each pixel in the image, f represents the linear coordinate of each pixel in the image, andj(u, v) represents the image calculated at the j-th angle, bj(u, v) represents the image obtained from the experiment at the j-th angle.
9. The apparatus of claim 6, wherein the reconstruction module comprises:
a detection unit for detecting the parameters of all the local orbit functions after the parameters are updated in each step
A deleting unit, configured to delete any local orbit function when detecting that a parameter of the local orbit function is smaller than a threshold obtained by all the local orbit function parameters, obtain, through establishing a binary tree, a local orbit function pair index whose local orbit function center distance is smaller than a preset pixel, and delete any local orbit function in the local orbit function pair;
and the protection unit is used for reducing the parameters of each local area orbit function to preset multiples according to preset probability after the parameters are updated iteratively and the local area orbit function is deleted at each step, and setting protection time, so that the screening operation and the deletion operation are not allowed to be executed within the protection time.
10. The apparatus according to any one of claims 6 to 9, wherein the parameters to be optimized comprise at least one of three-dimensional space coordinates of the center of each local orbital function, parameters describing the shape thereof, three euler angles for each rotation angle, drift of the sample at each angle, and mechanical tilt deviation of the sample stage.
11. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, the processor executing the program to implement the method of three-dimensional reconstruction of local area trajectory functions as claimed in any one of claims 1 to 5.
12. A computer-readable storage medium, on which a computer program is stored, which program is executable by a processor for implementing the method for three-dimensional reconstruction of a local area trajectory function as claimed in any one of claims 1 to 5.
CN202210381325.3A 2022-04-12 2022-04-12 Three-dimensional reconstruction method and device for local orbit function Active CN114648611B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202210381325.3A CN114648611B (en) 2022-04-12 2022-04-12 Three-dimensional reconstruction method and device for local orbit function
PCT/CN2023/080253 WO2023197785A1 (en) 2022-04-12 2023-03-08 Three-dimensional reconstruction method and apparatus for local orbital function

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210381325.3A CN114648611B (en) 2022-04-12 2022-04-12 Three-dimensional reconstruction method and device for local orbit function

Publications (2)

Publication Number Publication Date
CN114648611A true CN114648611A (en) 2022-06-21
CN114648611B CN114648611B (en) 2023-07-18

Family

ID=81997728

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210381325.3A Active CN114648611B (en) 2022-04-12 2022-04-12 Three-dimensional reconstruction method and device for local orbit function

Country Status (2)

Country Link
CN (1) CN114648611B (en)
WO (1) WO2023197785A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023197785A1 (en) * 2022-04-12 2023-10-19 清华大学 Three-dimensional reconstruction method and apparatus for local orbital function
CN117392316A (en) * 2023-10-13 2024-01-12 清华大学 Three-dimensional reconstruction method and device based on series of under-focus images

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014171988A2 (en) * 2013-01-29 2014-10-23 Andrew Robert Korb Methods for analyzing and compressing multiple images
US20160203382A1 (en) * 2012-03-22 2016-07-14 The Charles Stark Draper Laboratory, Inc. Compressive sensing with local geometric features
US20160328253A1 (en) * 2015-05-05 2016-11-10 Kyndi, Inc. Quanton representation for emulating quantum-like computation on classical processors
WO2019246397A1 (en) * 2018-06-21 2019-12-26 The University Of Chicago A fully fourier space spherical convolutional neural network based on clebsch-gordan transforms
US10607164B1 (en) * 2019-07-28 2020-03-31 Zichu Wang Architects space programming process that determines new classroom sizes and count for educational institutions
CN111179339A (en) * 2019-12-13 2020-05-19 深圳市瑞立视多媒体科技有限公司 Coordinate positioning method, device and equipment based on triangulation and storage medium
WO2020098686A1 (en) * 2018-11-16 2020-05-22 广州市百果园信息技术有限公司 Face detection model training method and apparatus, and face key point detection method and apparatus
US20210104023A1 (en) * 2020-05-18 2021-04-08 Shanghai United Imaging Healthcare Co., Ltd. Systems and methods for image reconstruction
WO2021077720A1 (en) * 2019-10-25 2021-04-29 深圳奥比中光科技有限公司 Method, apparatus, and system for acquiring three-dimensional model of object, and electronic device
CN113034681A (en) * 2021-04-07 2021-06-25 清华大学 Three-dimensional reconstruction method and device for spatial plane relation constraint
EP3866112A2 (en) * 2020-11-30 2021-08-18 Beijing Baidu Netcom Science And Technology Co. Ltd. Method, apparatus, device, storage medium and program for three-dimensional reconstruction
WO2021179485A1 (en) * 2020-03-11 2021-09-16 平安科技（深圳）有限公司 Image rectification processing method and apparatus, storage medium, and computer device
WO2021245274A1 (en) * 2020-06-06 2021-12-09 Querbes Olivier Taking an optical impression of a patient's dental arch
EP3926408A1 (en) * 2020-06-18 2021-12-22 Konica Minolta, Inc. Image forming method and image forming system
CN114186191A (en) * 2021-11-19 2022-03-15 合肥联宝信息技术有限公司 Method, device and equipment for calculating coordinate transformation matrix and readable storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10049446B2 (en) * 2015-12-18 2018-08-14 Carestream Health, Inc. Accelerated statistical iterative reconstruction
CN111208513B (en) * 2020-01-15 2023-03-31 西安电子科技大学 Space target ISAR image sequence energy back projection and three-dimensional reconstruction method
CN111862316B (en) * 2020-07-28 2024-01-05 杭州深瞳科技有限公司 Three-dimensional reconstruction method of dense direct RGBD (Red Green blue-white) of tight coupling of IMU (inertial measurement Unit) based on optimization
CN113720865B (en) * 2021-08-06 2022-09-02 清华大学 Electronic lamination imaging method and device for automatically correcting tape axis deviation of sample
CN114648611B (en) * 2022-04-12 2023-07-18 清华大学 Three-dimensional reconstruction method and device for local orbit function

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160203382A1 (en) * 2012-03-22 2016-07-14 The Charles Stark Draper Laboratory, Inc. Compressive sensing with local geometric features
WO2014171988A2 (en) * 2013-01-29 2014-10-23 Andrew Robert Korb Methods for analyzing and compressing multiple images
US20160328253A1 (en) * 2015-05-05 2016-11-10 Kyndi, Inc. Quanton representation for emulating quantum-like computation on classical processors
WO2019246397A1 (en) * 2018-06-21 2019-12-26 The University Of Chicago A fully fourier space spherical convolutional neural network based on clebsch-gordan transforms
WO2020098686A1 (en) * 2018-11-16 2020-05-22 广州市百果园信息技术有限公司 Face detection model training method and apparatus, and face key point detection method and apparatus
US10607164B1 (en) * 2019-07-28 2020-03-31 Zichu Wang Architects space programming process that determines new classroom sizes and count for educational institutions
WO2021077720A1 (en) * 2019-10-25 2021-04-29 深圳奥比中光科技有限公司 Method, apparatus, and system for acquiring three-dimensional model of object, and electronic device
CN111179339A (en) * 2019-12-13 2020-05-19 深圳市瑞立视多媒体科技有限公司 Coordinate positioning method, device and equipment based on triangulation and storage medium
WO2021179485A1 (en) * 2020-03-11 2021-09-16 平安科技（深圳）有限公司 Image rectification processing method and apparatus, storage medium, and computer device
US20210104023A1 (en) * 2020-05-18 2021-04-08 Shanghai United Imaging Healthcare Co., Ltd. Systems and methods for image reconstruction
WO2021245274A1 (en) * 2020-06-06 2021-12-09 Querbes Olivier Taking an optical impression of a patient's dental arch
EP3926408A1 (en) * 2020-06-18 2021-12-22 Konica Minolta, Inc. Image forming method and image forming system
EP3866112A2 (en) * 2020-11-30 2021-08-18 Beijing Baidu Netcom Science And Technology Co. Ltd. Method, apparatus, device, storage medium and program for three-dimensional reconstruction
CN113034681A (en) * 2021-04-07 2021-06-25 清华大学 Three-dimensional reconstruction method and device for spatial plane relation constraint
CN114186191A (en) * 2021-11-19 2022-03-15 合肥联宝信息技术有限公司 Method, device and equipment for calculating coordinate transformation matrix and readable storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DANNY KARMON等: "Risk Minimization in Structured Prediction using Orbit loss", ARXIV, pages 1 - 16 *
叶佳,胡晋生,朱静,程志英: "利用轴电子通道统计法测定杂质原子的占位", 中国科学E辑, no. 02, pages 14 - 21 *
张淑萍;吴文;万毅;: "基于多阶段生成对抗网络的单幅图像阴影去除方法", 计算机应用, no. 08, pages 214 - 221 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023197785A1 (en) * 2022-04-12 2023-10-19 清华大学 Three-dimensional reconstruction method and apparatus for local orbital function
CN117392316A (en) * 2023-10-13 2024-01-12 清华大学 Three-dimensional reconstruction method and device based on series of under-focus images

Also Published As

Publication number Publication date
CN114648611B (en) 2023-07-18
WO2023197785A1 (en) 2023-10-19

Similar Documents

Publication Publication Date Title
CN114648611B (en) Three-dimensional reconstruction method and device for local orbit function
CN112669463B (en) Method for reconstructing curved surface of three-dimensional point cloud, computer device and computer-readable storage medium
CN111768411B (en) Coronary centerline extraction method, device, computer equipment and storage medium
CN113223078B (en) Mark point matching method, device, computer equipment and storage medium
CN109300139B (en) Lane line detection method and device
EP2591431B1 (en) Computed tomography method, computer program, computing device and computed tomography system
CN110728675A (en) Pulmonary nodule analysis device, model training method, device and analysis equipment
CN113159103A (en) Image matching method, image matching device, electronic equipment and storage medium
JP4175536B2 (en) Boundary data inside / outside judgment method and program
Honti et al. Automation of cylinder segmentation from point cloud data
CN115731527A (en) Road boundary detection method, apparatus, computer device and storage medium
CN116468103A (en) Training method, application method and system for lung nodule benign and malignant recognition model
US20220189806A1 (en) Estimating heights of defects in a wafer
CN117274132A (en) Multi-scale self-encoder generation method, electronic device and storage medium
CN114966576A (en) Radar external reference calibration method and device based on prior map and computer equipment
Kruszynski et al. An interactive visualization system for quantifying coral structures.
CN117455936B (en) Point cloud data processing method and device and electronic equipment
CN116719896B (en) POI data mining method and device, computer equipment and storage medium
CN116959637B (en) Three-dimensional reconstruction method and device based on depth-dependent electron beam and computer equipment
US20230401691A1 (en) Image defect detection method, electronic device and readable storage medium
CN116071745B (en) Method and device for processing electron microscope density map target recognition model
CN112884820B (en) Image initial registration and neural network training method, device and equipment
CN111161242B (en) Lung nodule HU value determination method, device, storage medium and computer equipment
RU2673774C2 (en) Method for assessing structural changes in sample of material as result of exposure to sample
CN117635840A (en) Method and device for three-dimensional reconstruction of local orbit function based on scanning diffraction pattern

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant