CN117392316B - Three-dimensional reconstruction method and device based on series of under-focus images - Google Patents
Three-dimensional reconstruction method and device based on series of under-focus images Download PDFInfo
- Publication number
- CN117392316B CN117392316B CN202311331045.2A CN202311331045A CN117392316B CN 117392316 B CN117392316 B CN 117392316B CN 202311331045 A CN202311331045 A CN 202311331045A CN 117392316 B CN117392316 B CN 117392316B
- Authority
- CN
- China
- Prior art keywords
- images
- under
- dimensional
- dimensional reconstruction
- focus
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims description 13
- 238000009826 distribution Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 7
- 238000003860 storage Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 abstract description 8
- 230000006870 function Effects 0.000 description 12
- 238000003384 imaging method Methods 0.000 description 11
- 239000002245 particle Substances 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000004088 simulation Methods 0.000 description 7
- 239000000463 material Substances 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 239000011148 porous material Substances 0.000 description 3
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical group [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000003325 tomography Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 229910010293 ceramic material Inorganic materials 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010894 electron beam technology Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 239000006261 foam material Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012067 mathematical method Methods 0.000 description 1
- 229910001092 metal group alloy Inorganic materials 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009828 non-uniform distribution Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 229920000642 polymer Polymers 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000002424 x-ray crystallography Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The application relates to the technical field of computer image processing, in particular to a three-dimensional reconstruction method and device based on series of under-focus images, wherein the method comprises the following steps: collecting a plurality of two-dimensional images of an object with different under-focus values at a plurality of tilting angles; based on a plurality of two-dimensional images, performing three-dimensional reconstruction on the object by using a preset three-dimensional reconstruction algorithm to obtain a three-dimensional reconstruction result. Therefore, the technical problems that in the related art, the operation of enabling all parts of an object under each angle to be in a positive focus condition is complex and difficult to realize, meanwhile, the complete information of a large-size object is difficult to obtain, the computing resource and time requirements are obviously increased and the like are solved.
Description
Technical Field
The application relates to the technical field of computer image processing, in particular to a three-dimensional reconstruction method and device based on series of under-focus images.
Background
In the related art, in order to obtain three-dimensional coordinates of atoms constituting an object, a tomography technology can be used for performing three-dimensional reconstruction of atomic resolution on the object, wherein the current tomography technology is used for realizing three-dimensional reconstruction of the object by shooting two-dimensional projection images of the object under different angles, thereby realizing high-resolution imaging of atoms in the object, having the characteristics of high precision and high resolution, and being convenient for researching the microstructure and properties of the substance.
However, in the related art, in order to ensure that each portion of the object under each angle is in the normal focus condition, accurate adjustment is required for the positions of the light source, the detector and the object, the conditions are not easy to meet and the complexity of the operation is increased, meanwhile, for large-sized objects, due to factors such as the example size, the detector size and the experimental setting, it is difficult to acquire enough angles to cover the whole object, the full-scale information cannot be ensured, and the required computing resources and time are also increased significantly, thereby limiting the capability of acquiring high-quality images rapidly.
Disclosure of Invention
The application provides a three-dimensional reconstruction method and device based on series under-focus images, which are used for solving the technical problems that in the related art, the operation of enabling all parts of an object under each angle to be in a normal focus condition is complex and difficult to realize, meanwhile, the complete information of a large-size object is difficult to obtain, the computing resource and time requirements are obviously increased, and the like.
An embodiment of a first aspect of the present application provides a three-dimensional reconstruction method based on a series of under-focus images, including the steps of: collecting a plurality of two-dimensional images of an object with different under-focus values at a plurality of tilting angles; and based on the plurality of two-dimensional images, carrying out three-dimensional reconstruction on the object by using a preset three-dimensional reconstruction algorithm to obtain a three-dimensional reconstruction result.
Alternatively, in one embodiment of the present application, the series of under-focus values of the plurality of images may be distributed in a manner uniformly distributed around the positive focus value 0 or non-uniformly distributed around the positive focus value 0.
Alternatively, in one embodiment of the present application, the plurality of images may be of the type, but are not limited to, high angle annular dark field images.
Optionally, in one embodiment of the present application, after acquiring the plurality of two-dimensional images of the object with different under-focus values at the plurality of tilt angles, the method further includes: and preprocessing an image sequence formed by the plurality of two-dimensional images to obtain a plurality of two-dimensional images meeting preset three-dimensional reconstruction conditions.
An embodiment of the second aspect of the present application provides a three-dimensional reconstruction device based on a series of under-focus images, including: the acquisition module is used for acquiring a plurality of two-dimensional images of the object with different under-focus values under a plurality of inclination angles; and the reconstruction module is used for carrying out three-dimensional reconstruction on the object by utilizing a preset three-dimensional reconstruction algorithm based on the plurality of two-dimensional images to obtain a three-dimensional reconstruction result.
Alternatively, in one embodiment of the present application, the series of under-focus values of the plurality of images may be distributed in a manner uniformly distributed around the positive focus value 0 or non-uniformly distributed around the positive focus value 0.
Alternatively, in one embodiment of the present application, the plurality of images may be of the type, but are not limited to, high angle annular dark field images.
Optionally, in one embodiment of the present application, further includes: the preprocessing module is used for preprocessing an image sequence formed by the plurality of two-dimensional images to obtain a plurality of two-dimensional images meeting preset three-dimensional reconstruction conditions.
An embodiment of a third aspect of the present application provides an electronic device, including: the system comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the three-dimensional reconstruction method based on the series of under-focus images.
A fourth aspect embodiment of the present application provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the above three-dimensional reconstruction method based on a series of under-focus images.
According to the embodiment of the application, the object can be reconstructed by acquiring a plurality of two-dimensional images of the object with different under-focus values under a plurality of tilting angles and utilizing a preset three-dimensional reconstruction algorithm, so that a high-quality three-dimensional reconstruction result is obtained, the operation complexity is reduced, accurate object structure information is provided, and the reconstruction accuracy is improved. Therefore, the technical problems that in the related art, the operation of enabling all parts of an object under each angle to be in a positive focus condition is complex and difficult to realize, meanwhile, the complete information of a large-size object is difficult to obtain, the computing resource and time requirements are obviously increased and the like are solved.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a flow chart of a three-dimensional reconstruction method based on a series of under-focus images provided in accordance with an embodiment of the present application;
FIG. 2 is a flow chart of a three-dimensional reconstruction method based on a series of under-focused images according to one embodiment of the application;
FIG. 3 is a schematic view of images of different defocus values at different tilt angles according to one embodiment of the present application;
FIG. 4 is a schematic diagram of a coordinate distance deviation from a real atom for a result atom according to one embodiment of the present application;
FIG. 5 is a schematic image diagram of different under-focus values at different tilt angles according to an embodiment of the present application;
FIG. 6 is a comparison of the tilt angle distribution of a conventional three-dimensional reconstruction method according to one embodiment of the present application and a three-dimensional reconstruction method according to an embodiment of the present application;
Fig. 7 is a schematic structural diagram of a three-dimensional reconstruction device based on a series of under-focus images according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.
The following describes a three-dimensional reconstruction method and device based on a series of under-focus images according to an embodiment of the present application with reference to the accompanying drawings. Aiming at the technical problems that in the related art mentioned in the background art, the operation of enabling all parts of an object under each angle to be in a positive focus condition is complex and difficult to realize, meanwhile, the complete information of the large-size object is difficult to obtain, the calculation resource and the time requirement are obviously increased, and the like, the application provides a three-dimensional reconstruction method based on a series of under-focus images. Therefore, the technical problems that in the related art, the operation of enabling all parts of an object under each angle to be in a positive focus condition is complex and difficult to realize, meanwhile, the complete information of a large-size object is difficult to obtain, the computing resource and time requirements are obviously increased and the like are solved.
Specifically, fig. 1 is a schematic flow chart of a three-dimensional reconstruction method based on a series of under-focus images according to an embodiment of the present application.
As shown in fig. 1, the three-dimensional reconstruction method based on the series of under-focus images comprises the following steps:
In step S101, a plurality of two-dimensional images of an object having different under-focus values at a plurality of tilt angles are acquired.
In some embodiments, the embodiments of the present application may collect, in an imaging device in actual use, a plurality of two-dimensional images of an object having different under-focus values at a plurality of tilt angles by adjusting positions of a detector and the object, so as to facilitate subsequent processing and analysis of the images, thereby obtaining a more accurate three-dimensional reconstruction result and providing more comprehensive and detailed object structure information.
In the actual implementation process, referring to fig. 2, the embodiment of the present application first acquires a plurality of two-dimensional images of an object with different under-focus values at a plurality of tilt angles, including the following steps:
step S201: tilting the object to an angle;
Step S202: shooting two-dimensional images corresponding to different under-focus values of an object at the tilting angle;
Step S203: repeating the steps S201-S202 to obtain two-dimensional images corresponding to different under-focus values of the object under different angles;
The number of the images under focus at the same angle may be any integer greater than 1, and is not particularly limited herein.
Thus, with reference to fig. 3, a schematic image of different under-focus values at different tilt angles is obtained.
Wherein a, b and c are multi-slice simulation images when the under-focus of the model particles is-5 nm,0nm and 5nm respectively when the tilting angle is 70 degrees; d. e and f are multi-slice simulation images of model particles respectively at minus 5nm,0nm and 5nm when the tilting angle is 0 degrees; g. h and i are multi-slice simulation images of model particles respectively at minus 5nm,0nm and 5nm under focus when the tilting angle is minus 70 degrees.
Specifically, different tilting angles have different advantages and effects in three-dimensional reconstruction, wherein when the tilting angle is 70 degrees, the three-dimensional reconstruction can be performed by obtaining more comprehensive information through a plurality of images with different projection angles, and the 70-degree tilting angle can provide more information to restore the real shape and structure of an object; when the tilting angle is 0 degrees, the image projection will obtain a cross-sectional view of the object, and although complete three-dimensional information cannot be obtained, the projection image with the tilting angle of 0 degrees still plays an important role in obtaining morphological information of the object; at tilt angles of 70 ° and-70 °, images obtained by two opposing tilt angle projections may provide stronger depth information, thereby reconstructing more accurately the three-dimensional shape and structure of the object.
Different information can be obtained through different tilting angles, a more comprehensive and more accurate three-dimensional reconstruction result can be obtained when the tilting angle is larger, and more complete and accurate object structure information can be obtained by comprehensively considering images of different tilting angles, so that further research and analysis are facilitated.
Alternatively, in one embodiment of the present application, the series of under-focus values for the plurality of images may be distributed in a manner that may be uniformly distributed around the plus focus value 0 or non-uniformly distributed around the plus focus value 0.
It is understood that in the embodiment of the present application, the series of under-focus values of the plurality of images may be uniformly distributed around the positive focus value 0 or may be unevenly distributed around the positive focus value 0.
Specifically, under the condition of uniform distribution, the under-focus value of the image at each angle is relatively uniformly distributed around the positive focus value 0, so that the whole model can be better covered, and balanced projection information at each angle is provided, thereby being beneficial to reducing the artifact and improving the accuracy and stability of reconstruction in the reconstruction process.
However, in the case of non-uniform distribution, the under-focus value of the image at each angle may be asymmetric or in a specific distribution manner around the positive focus value 0, and may be designed according to the shape, density distribution or region of interest of a specific object, and by adjusting the under-focus value distribution according to the characteristics of the object, the details of the specific region may be better highlighted or weakened, thereby meeting specific imaging requirements.
In summary, through the series of images which are uniformly or unevenly distributed and under-focused, the incompleteness of each angle can be compensated, more comprehensive and accurate object structure information is provided, and specific imaging requirements in different fields are met.
Alternatively, in one embodiment of the present application, the plurality of images may be of the type, but are not limited to, high angle annular dark field images.
That is, annular dark field imaging is a very special imaging technique that uses high angle projection and high sensitivity detectors to capture scattered light signals around an object, where the detectors record weak signals generated by the scattered light of the object, so that detailed information of the surface and internal minute structures of the object can be obtained from the generated signals.
In embodiments of the present application, annular dark field imaging can provide more scattered light information, including details of object edges, textures, microstructures, etc., while exhibiting good sensitivity to objects with low contrast and low absorptivity, such that the embodiments of the present application can be used to advantage in determining internal details of objects with multi-material and complex structures, for example, high angle annular dark field imaging can be used to study porous materials with micro-or nano-scale, such as porous metal alloys, ceramic materials, or polymer foam materials, and annular dark field imaging can be used to very clearly observe pore structures, pore distribution, and connectivity within the material, thereby enhancing the resolution of fine details, and helping to optimize the physical and chemical properties of the material.
Optionally, in one embodiment of the present application, after acquiring a plurality of two-dimensional images of the object with different under-focus values at a plurality of tilt angles, the method further includes: preprocessing an image sequence formed by a plurality of two-dimensional images to obtain a plurality of two-dimensional images meeting preset three-dimensional reconstruction conditions.
In the actual execution process, the embodiment of the application performs preprocessing on an image sequence formed by a plurality of two-dimensional images after acquiring a plurality of two-dimensional images of an object with different under-focus values under a plurality of tilting angles so as to meet the preset three-dimensional reconstruction condition.
Specifically, centering the image, which can be understood as adjusting the image to a centered position to ensure accurate rotation and alignment; performing axis combination processing can be understood as performing translation and rotation operations on each image according to the rotation angle and the information of the scanning axis, so that the images are aligned under the same reference coordinate system, and thus artifacts caused by rotation and translation are reduced; noise reduction is carried out, noise in an image is reduced or eliminated by applying a proper filter or a denoising algorithm, the image quality is improved, and the definition and detail display of a reconstruction result are further improved; and carrying out normalization processing, and carrying out gray value normalization on the images to ensure that each image has a similar brightness range and ensure data consistency in the three-dimensional reconstruction process. By preprocessing the image sequence, the image data can be optimized, the preset three-dimensional reconstruction condition is met, and the data suitable for subsequent three-dimensional reconstruction is provided, so that the quality and accuracy of the three-dimensional reconstruction result are improved.
The preset three-dimensional reconstruction conditions can be determined according to requirements of practical application requirements, image quality, required reconstruction space size, resolution, projection geometric characteristics of an image sequence and the like. The specific values and adjustment modes will vary according to the specific imaging system and application scenario, and the specific settings are set by those skilled in the art, which are not particularly limited herein.
In step S102, based on the plurality of two-dimensional images, a three-dimensional reconstruction is performed on the object by using a preset three-dimensional reconstruction algorithm, so as to obtain a three-dimensional reconstruction result.
In some embodiments, the embodiments of the present application may process and analyze a plurality of two-dimensional images by using a preset three-dimensional reconstruction algorithm to recover three-dimensional structure and morphological information of an object, for example, three-dimensional coordinates and atomic types of atoms that form the object, where the preset three-dimensional reconstruction algorithm generally uses fuzzy information of projection to perform interpolation, back projection, filtering, and so on by using a mathematical method, so as to obtain three-dimensional density distribution of the object.
Specifically, back-projecting projection data of a plurality of two-dimensional images to generate candidate three-dimensional voxels or projection data, filtering the projection data by applying a filtering algorithm to compensate for artifacts or noise introduced in the projection process, interpolating incomplete projection data in space by an interpolation method to fill missing information, and reconstructing the interpolated data according to a defined reconstruction space to obtain three-dimensional structure information of an object.
By performing three-dimensional reconstruction based on a plurality of two-dimensional images, depth information of an object can be acquired from different angles and view angles, more comprehensive and accurate structural display is provided, and in practical application, the technology of X-ray crystallography, electron microscope imaging and the like can be used, so that images with higher resolution can be provided to reveal structural and composition information of an object at an atomic level.
A three-dimensional reconstruction method based on a series of under-focus images according to an embodiment of the present application will be described in detail with reference to fig. 2, 3, 4, 5, and 6.
It can be understood that in the embodiment of the present application, the small particles composed of 200000 silicon atoms are required to be reconstructed, the three-dimensional coordinates of the silicon atoms are randomly distributed in space, and the atomic distance is between 2.32 a and 2.53 a, and the diameter of the particles is 19.03nm. Multi-slice simulated images (slice thickness: 2 a; electron beam sampling: 0.1 a; sample sampling: 0.1 a; detector inner and outer half angles: 30mrad, 195mrad; accelerated electron energy: 300keV; convergence half angle: 25 mrad) were calculated for 24 different tilt angles, respectively-5 nm,0nm,5nm, by tilting the model particles from-70 ° to 70 °. And (3) carrying out three-dimensional reconstruction on the multi-slice simulation image with the edge length of 612 and the pixel size of 0.34A and different under-focus values of the obtained model particles under 24 tilt angles.
For example, in connection with fig. 2, an embodiment of the present application includes the following steps:
step S201: tilting the object to an angle;
Step S202: shooting two-dimensional images corresponding to different under-focus values of an object at the tilting angle;
Step S203: repeating the steps S201-S202 to obtain two-dimensional images corresponding to different under-focus values of the object under different angles;
it will be appreciated that the number of different under-focused images at the same angle may be any integer greater than 1, and is not particularly limited herein.
Thus, with reference to fig. 3, a schematic image of different under-focus values at different tilt angles is obtained.
Wherein a, b and c are multi-slice simulation images when the under-focus of the model particles is-5 nm,0nm and 5nm respectively when the tilting angle is 70 degrees; d. e and f are multi-slice simulation images of model particles respectively at minus 5nm,0nm and 5nm when the tilting angle is 0 degrees; g. h and i are multi-slice simulation images of model particles respectively at minus 5nm,0nm and 5nm under focus when the tilting angle is minus 70 degrees.
Step S204: preprocessing an image sequence;
Specifically, pretreatment includes centering, axis combination, noise reduction, normalization, and the like.
Step S205: three-dimensional reconstruction is carried out on the image sequence by applying a three-dimensional reconstruction algorithm, and structural information of an object is obtained;
The structural information of the object includes three-dimensional coordinates of atoms, atomic types and the like of the constituent object.
Referring to fig. 4, when the total number of acquired images is the same, the three-dimensional reconstruction method based on series under-focus and the conventional three-dimensional reconstruction concept are compared, at this time, the number of under-focus images acquired under each tilting angle is 3, the number of tilting angles is also reduced to 1/3 of the conventional three-dimensional reconstruction concept, and a is a tilting angle distribution schematic diagram of the conventional three-dimensional reconstruction concept; b is a tilting angle distribution schematic diagram of a three-dimensional reconstruction method based on series under-focus.
In combination with the illustration of fig. 5, during the three-dimensional reconstruction, the value of the loss function can be plotted as a line graph of the value of the loss function along with the iteration number of the reconstruction algorithm by calculating the loss function and recording the change of the value of the loss function along with the iteration number, specifically, the loss function value gradually decreases along with the increase of the iteration number, which indicates that the reconstruction result gradually approaches the target, and indicates that the algorithm is approaching toward a better solution. The line graph takes the iteration times as an abscissa and the value of the loss function as an ordinate to show the change of the loss function in the iteration process. According to the trend and change on the line graph, visual representation of the optimization performance of the reconstruction algorithm can be provided, decision making of parameter adjustment, algorithm improvement or other optimization strategies can be carried out, and researchers can be helped to evaluate and improve the three-dimensional reconstruction result.
In combination with the distance deviation histogram of the atomic coordinates in the three-dimensional reconstruction and the atomic coordinates in the real model shown in fig. 6, the deviation distribution condition in the reconstruction result of the atomic coordinates can be directly observed, the main deviation type, the size and the distribution range of the deviation in the reconstruction result are determined, specifically, the histogram presents a peak in the center concentration, the width is smaller, and the atomic coordinates of the reconstruction result are very close to the atomic coordinates in the real model, and the accuracy is higher.
According to the three-dimensional reconstruction method based on the series of under-focus images, which is provided by the embodiment of the application, a plurality of two-dimensional images of the object with different under-focus values under a plurality of inclination angles can be acquired, and the object is reconstructed by utilizing a preset three-dimensional reconstruction algorithm, so that a high-quality three-dimensional reconstruction result is obtained, the operation complexity is reduced, accurate object structure information is provided, and the reconstruction precision is improved. Therefore, the technical problems that in the related art, the operation of enabling all parts of an object under each angle to be in a positive focus condition is complex and difficult to realize, meanwhile, the complete information of a large-size object is difficult to obtain, the computing resource and time requirements are obviously increased and the like are solved.
A three-dimensional reconstruction device based on a series of under-focus images according to an embodiment of the present application will be described next with reference to the accompanying drawings.
Fig. 7 is a block schematic diagram of a three-dimensional reconstruction apparatus based on a series of under-focused images according to an embodiment of the present application.
As shown in fig. 7, the three-dimensional reconstruction device 10 based on a series of under-focus images includes: an acquisition module 100 and a reconstruction module 200.
Specifically, the acquisition module 100 is configured to acquire a plurality of two-dimensional images of an object with different under-focus values at a plurality of tilt angles.
The reconstruction module 200 is configured to perform three-dimensional reconstruction on the object by using a preset three-dimensional reconstruction algorithm based on the plurality of two-dimensional images, so as to obtain a three-dimensional reconstruction result.
Alternatively, in one embodiment of the application, the series of under-focus values for the plurality of images are distributed in a manner that is uniformly distributed about the normal focus value 0 or non-uniformly distributed about the normal focus value 0
Alternatively, in one embodiment of the application, the plurality of images are of the type high angle annular dark field image.
Optionally, in one embodiment of the present application, further includes: and a preprocessing module.
The preprocessing module is used for preprocessing an image sequence formed by a plurality of two-dimensional images to obtain a plurality of two-dimensional images meeting preset three-dimensional reconstruction conditions.
It should be noted that the foregoing explanation of the embodiment of the three-dimensional reconstruction method based on the series of under-focus images is also applicable to the three-dimensional reconstruction device based on the series of under-focus images of this embodiment, and will not be repeated here.
According to the three-dimensional reconstruction device based on the series of under-focus images, provided by the embodiment of the application, the object can be reconstructed by collecting a plurality of two-dimensional images of the object with different under-focus values under a plurality of inclination angles and utilizing a preset three-dimensional reconstruction algorithm, so that a high-quality three-dimensional reconstruction result is obtained, the operation complexity is reduced, accurate object structure information is provided, and the reconstruction precision is improved. Therefore, the technical problems that in the related art, the operation of enabling all parts of an object under each angle to be in a positive focus condition is complex and difficult to realize, meanwhile, the complete information of a large-size object is difficult to obtain, the computing resource and time requirements are obviously increased and the like are solved.
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device may include:
A memory 801, a processor 802, and a computer program stored on the memory 801 and executable on the processor 802.
The processor 802 implements the three-dimensional reconstruction method based on the series of under-focus images provided in the above-described embodiment when executing a program.
Further, the electronic device further includes:
a communication interface 803 for communication between the memory 801 and the processor 802.
A memory 801 for storing a computer program executable on the processor 802.
The memory 801 may include high-speed RAM memory or may further include non-volatile memory (non-volatile memory), such as at least one magnetic disk memory.
If the memory 801, the processor 802, and the communication interface 803 are implemented independently, the communication interface 803, the memory 801, and the processor 802 may be connected to each other through a bus and perform communication with each other. The bus may be an industry standard architecture (Industry Standard Architecture, abbreviated ISA) bus, an external device interconnect (PERIPHERAL COMPONENT, abbreviated PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 8, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 801, the processor 802, and the communication interface 803 are integrated on a chip, the memory 801, the processor 802, and the communication interface 803 may communicate with each other through internal interfaces.
The processor 802 may be a central processing unit (Central Processing Unit, abbreviated as CPU), or an Application SPECIFIC INTEGRATED Circuit, abbreviated as ASIC, or one or more integrated circuits configured to implement embodiments of the present application.
The embodiment of the application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the three-dimensional reconstruction method based on a series of under-focus images as above.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, "N" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order from that shown or discussed, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or N wires, a portable computer cartridge (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (4)
1. The three-dimensional reconstruction method based on the series of under-focus images is characterized by comprising the following steps of:
Collecting a plurality of two-dimensional images of an object with different under-focus values corresponding to each tilting angle in a plurality of tilting angles;
Based on the plurality of two-dimensional images, performing three-dimensional reconstruction on the object by using a preset three-dimensional reconstruction algorithm to obtain a three-dimensional reconstruction result;
The distribution mode of the series of under-focus values of the two-dimensional images is that the series of under-focus values are uniformly distributed around a positive focus value 0 in a preset nanometer level or are unevenly distributed around the positive focus value 0 in the preset nanometer level;
The types of the two-dimensional images are high-angle annular dark field images;
After acquiring the plurality of two-dimensional images of the object at each of the plurality of tilt angles with different under-focus values, further comprising: and preprocessing an image sequence formed by the plurality of two-dimensional images to obtain a plurality of two-dimensional images meeting preset three-dimensional reconstruction conditions.
2. A three-dimensional reconstruction device based on a series of under-focused images, comprising:
The acquisition module is used for acquiring a plurality of two-dimensional images with different under-focus values, corresponding to each tilting angle in the plurality of tilting angles, of the object;
the reconstruction module is used for carrying out three-dimensional reconstruction on the object by utilizing a preset three-dimensional reconstruction algorithm based on the plurality of two-dimensional images to obtain a three-dimensional reconstruction result;
The distribution mode of the series of under-focus values of the two-dimensional images is that the series of under-focus values are uniformly distributed around a positive focus value 0 in a preset nanometer level or are unevenly distributed around the positive focus value 0 in the preset nanometer level;
The types of the two-dimensional images are high-angle annular dark field images;
the three-dimensional reconstruction device based on the series of under-focus images further comprises: the preprocessing module is used for preprocessing an image sequence formed by the plurality of two-dimensional images to obtain a plurality of two-dimensional images meeting preset three-dimensional reconstruction conditions.
3. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the series under-focus image based three-dimensional reconstruction method as defined in claim 1.
4. A computer-readable storage medium having stored thereon a computer program, characterized in that the program is executed by a processor for implementing a three-dimensional reconstruction method based on a series of under-focus images as claimed in claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311331045.2A CN117392316B (en) | 2023-10-13 | 2023-10-13 | Three-dimensional reconstruction method and device based on series of under-focus images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311331045.2A CN117392316B (en) | 2023-10-13 | 2023-10-13 | Three-dimensional reconstruction method and device based on series of under-focus images |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117392316A CN117392316A (en) | 2024-01-12 |
| CN117392316B true CN117392316B (en) | 2024-06-18 |
Family
ID=89438460
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311331045.2A Active CN117392316B (en) | 2023-10-13 | 2023-10-13 | Three-dimensional reconstruction method and device based on series of under-focus images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117392316B (en) |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110420026B (en) * | 2019-07-15 | 2020-05-19 | 中国科学院自动化研究所 | Magnetic particle imaging three-dimensional reconstruction method, system and device based on FFL |
| CN110874864B (en) * | 2019-10-25 | 2022-01-14 | 奥比中光科技集团股份有限公司 | Method, device, electronic equipment and system for obtaining three-dimensional model of object |
| CN110763161B (en) * | 2019-11-22 | 2024-04-09 | 安徽大学 | Three-dimensional reconstruction data acquisition system based on intensity transmission equation |
| CN111429506A (en) * | 2020-03-30 | 2020-07-17 | 长沙微成像电子科技有限公司 | Volume fraction obtaining method, device and system based on three-dimensional reconstruction and storage medium |
| WO2022016412A1 (en) * | 2020-07-22 | 2022-01-27 | 深圳市汇顶科技股份有限公司 | Depth information image acquisition apparatus and electronic device |
| CN114494242A (en) * | 2022-02-21 | 2022-05-13 | 平安科技（深圳）有限公司 | Time series data detection method, device, equipment and computer storage medium |
| CN114648611B (en) * | 2022-04-12 | 2023-07-18 | 清华大学 | Three-dimensional reconstruction method and device for local orbit function |
-
2023
- 2023-10-13 CN CN202311331045.2A patent/CN117392316B/en active Active
Non-Patent Citations (1)
| Title |
|---|
| 成像参数对ADF STEM三维重构成像精确性的影响;明文全，陈江华，沈若涵，何玉涛，陈志逵;电子显微学报;20191009;第38卷(第5期);502-511 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117392316A (en) | 2024-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dahmen et al. | Feature adaptive sampling for scanning electron microscopy | |
| Salzer et al. | Quantitative comparison of segmentation algorithms for FIB‐SEM images of porous media | |
| Fernandez | Computational methods for materials characterization by electron tomography | |
| Turoňová et al. | On geometric artifacts in cryo electron tomography | |
| Hwang et al. | Towards the low-dose characterization of beam sensitive nanostructures via implementation of sparse image acquisition in scanning transmission electron microscopy | |
| Körner et al. | Increasing throughput in x-ray computed tomography measurement of surface topography using sinogram interpolation | |
| Trampert et al. | Exemplar-based inpainting as a solution to the missing wedge problem in electron tomography | |
| Monaco et al. | A comparison between holographic and near-field ptychographic X-ray tomography for solid oxide cell materials | |
| CN117392316B (en) | Three-dimensional reconstruction method and device based on series of under-focus images | |
| Bouhaouel et al. | Task-specific acquisition trajectories optimized using observer models | |
| Okariz et al. | A methodology for finding the optimal iteration number of the SIRT algorithm for quantitative electron tomography | |
| US7702180B2 (en) | Imaging method and device for the computer-assisted evaluation of computer-tomographic measurements by means of direct iterative reconstruction | |
| Okamoto et al. | Patch-based artifact reduction for three-dimensional volume projection data of sparse-view micro-computed tomography | |
| EP2972254B1 (en) | Extended field iterative reconstruction technique (efirt) for correlated noise removal | |
| CN113052929A (en) | Linear scanning CL reconstruction method based on projection visual angle weighting | |
| CN117491400B (en) | Scanning transmission diffraction method and device for eliminating heat diffusion scattering and inelastic scattering | |
| Hall et al. | Electron tomography methods for C. elegans | |
| US11645792B2 (en) | Edge phase effects removal using wavelet correction and particle classification using combined absorption and phase contrast | |
| Zhang et al. | REST: A method for restoring signals and revealing individual macromolecule states in cryo-ET | |
| Nakano et al. | Parameter optimization for 3D-reconstruction from XFEL diffraction patterns based on Fourier slice matching | |
| Andrew et al. | Iterative reconstruction for increased imaging throughput in lab-based 3D XRM | |
| US20220207256A1 (en) | Charged particle microscope systems and clustering processes for high dynamic range sample analysis | |
| Kumar | Computational Methods for Nanoscale X-ray Computed Tomography Image Analysis of Fuel Cell and Battery Materials | |
| Moebel et al. | A monte carlo framework for denoising and missing wedge reconstruction in cryo-electron tomography | |
| Roldan | Reconstruction of porous structures from FIB-SEM data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |