WO2024183239A1 - 基于fpga的三维重建装置、方法、系统及存储介质 - Google Patents

基于fpga的三维重建装置、方法、系统及存储介质 Download PDF

Info

Publication number
WO2024183239A1
WO2024183239A1 PCT/CN2023/113481 CN2023113481W WO2024183239A1 WO 2024183239 A1 WO2024183239 A1 WO 2024183239A1 CN 2023113481 W CN2023113481 W CN 2023113481W WO 2024183239 A1 WO2024183239 A1 WO 2024183239A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional
image
reconstruction
images
intensity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/113481
Other languages
English (en)
French (fr)
Inventor
张旺
秦文健
曾光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Publication of WO2024183239A1 publication Critical patent/WO2024183239A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining

Definitions

  • the present application relates to the field of three-dimensional reconstruction technology. Specifically, the present application relates to a three-dimensional reconstruction device, method, system and storage medium based on FPGA.
  • Three-dimensional reconstruction is a hot topic in medical imaging technology.
  • there are two main ways to achieve three-dimensional reconstruction of objects One is to stitch together a large number of two-dimensional images of the object to obtain a three-dimensional image.
  • the other is to perform three-dimensional frequency domain to spatial domain, spatial domain to frequency domain and other transformations based on the three-dimensional information of the object to obtain a three-dimensional image.
  • the current three-dimensional reconstruction system mainly uses high-performance computers or GPUs to achieve three-dimensional reconstruction of objects.
  • high-performance computers or GPUs there are many disadvantages in achieving three-dimensional reconstruction through high-performance computers or GPUs.
  • the cost of high-performance computers or GPUs is high.
  • high-performance computers or GPUs are large in size and are not easy to embed into devices that are easy to transplant.
  • the embodiments of the present application provide a 3D reconstruction device, method, system and storage medium based on FPGA, which can solve the problems of high cost and large volume of 3D reconstruction system in related technologies.
  • the technical solution is as follows:
  • a 3D reconstruction device based on FPGA includes: an image acquisition module, used to acquire at least one two-dimensional image of a target to be reconstructed; wherein each of the two-dimensional images corresponds to a projection angle of the target to be reconstructed; an image transformation module, used to transform the two-dimensional images of each of the projection angles in parallel to obtain an intensity image corresponding to each projection angle; and a three-dimensional reconstruction module, used to perform three-dimensional reconstruction in parallel based on the intensity images of each projection angle to obtain a three-dimensional image of the target to be reconstructed.
  • a 3D reconstruction method based on FPGA includes: acquiring at least one two-dimensional image of a target to be reconstructed; wherein each of the two-dimensional images corresponds to a projection angle of the target to be reconstructed; transforming the two-dimensional images of each projection angle in parallel to obtain an intensity image corresponding to each projection angle; and performing three-dimensional reconstruction in parallel based on the intensity images of each projection angle to obtain a three-dimensional image of the target to be reconstructed.
  • a 3D reconstruction system based on FPGA includes the 3D reconstruction device based on FPGA as described above.
  • a storage medium stores a computer program thereon, and when the computer program is executed by a processor, the FPGA-based three-dimensional reconstruction method as described above is implemented.
  • a computer program product includes a computer program, the computer program is stored in a storage medium, a processor of a computer device reads the computer program from the storage medium, and the processor The processor executes the computer program so that the computer device implements the FPGA-based three-dimensional reconstruction method as described above when executing.
  • the FPGA unit in the three-dimensional reconstruction system is used to perform parallel reconstruction to obtain a three-dimensional image of the target to be reconstructed; on the one hand, the parallel acceleration processing mechanism based on the FPGA unit reduces the power consumption of the three-dimensional reconstruction and ensures the speed of the three-dimensional reconstruction; on the other hand, the FPGA unit has low cost and small size, and is easy to embed into a device that is easy to transplant, effectively solving the problems of high cost and large size of the three-dimensional reconstruction system in related technologies.
  • FIG1 is a block diagram of a 3D reconstruction system based on FPGA according to the present application.
  • FIG2 is a flow chart of a 3D reconstruction method based on FPGA according to an exemplary embodiment
  • FIG3 is a schematic diagram of a specific implementation of three-dimensional reconstruction involved in the embodiment corresponding to FIG2 ;
  • FIG4 is a schematic diagram of obtaining an intensity image using two-dimensional images according to the embodiment corresponding to FIG2 ;
  • FIG5 is a flow chart of step 330 in one embodiment of the embodiment corresponding to FIG2;
  • FIG6 is a flow chart of step 350 in one embodiment of the embodiment corresponding to FIG2 ;
  • FIG. 7 is a schematic diagram of a specific implementation of cross-sectional strength data involved in the embodiment corresponding to FIG. 7 ;
  • FIGS. 8 to 9 are schematic diagrams showing a specific implementation of a 3D reconstruction method based on FPGA
  • FIG10 is a structural block diagram of a 3D reconstruction device based on FPGA according to an exemplary embodiment
  • Fig. 11 is a structural block diagram of an electronic device according to an exemplary embodiment.
  • Three-dimensional reconstruction technology can not only fully display the internal microscopic structure of the object being measured, but also reflect the relationship between the internal details of the object being measured, and can provide information far greater than two-dimensional images. Therefore, three-dimensional reconstruction technology is a current research hotspot.
  • a classic method of three-dimensional reconstruction is tomographic reconstruction. Tomography refers to obtaining information about the object being measured layer by layer and then reconstructing it in three dimensions through an algorithm. In microscopic imaging, the classic method of obtaining object information layer by layer is holographic imaging.
  • Holographic imaging is an advance of microscopic three-dimensional tomographic reconstruction, which uses multi-angle holograms to reconstruct the image of a three-dimensional object.
  • the holographic tomographic three-dimensional reconstruction algorithm may be divided into two types: tomographic reconstruction that considers the diffraction effect and tomographic reconstruction that does not consider the diffraction effect.
  • the tomography that takes into account the diffraction of the sample believes that light not only propagates along a straight line in the sample, but also diffracts when the light wave passes through the object.
  • This algorithm is based on the Fourier diffraction projection theory and can reconstruct a three-dimensional image with a relatively high resolution.
  • the related technology still has the defects of high cost and large size of the three-dimensional reconstruction system.
  • the FPGA-based three-dimensional reconstruction device provided in the present application can effectively reduce the volume of the three-dimensional reconstruction system while ensuring the three-dimensional reconstruction speed, reduce the cost of the three-dimensional reconstruction system, and solve the problems of high cost and large volume of the three-dimensional reconstruction device in the related art.
  • the FPGA-based three-dimensional reconstruction device can be deployed in the three-dimensional reconstruction system, and can also be deployed in an electronic device, for example, the electronic device can be a desktop computer, a laptop computer, a server configured with a von Neumann architecture, or a mobile device carrying a single-chip microcomputer, etc.
  • a FPGA-based three-dimensional reconstruction method is also provided, and accordingly, the FPGA-based three-dimensional reconstruction method is suitable for a FPGA-based three-dimensional reconstruction device.
  • the 3D reconstruction system 100 includes a field programmable gate array FPGA module 110; wherein the FPGA module 110 may include multiple transformation units 111, multiple reconstruction units 113 and a memory bar 115, an AXI bus 117 and a high-speed interface 119.
  • the FPGA module 110 is used to obtain at least one two-dimensional image of the target to be reconstructed; wherein each two-dimensional image corresponds to a projection angle of the target to be reconstructed; and the two-dimensional images of each projection angle are transformed in parallel to obtain Corresponding to the intensity image of each projection angle; performing three-dimensional reconstruction in parallel based on the intensity image of each projection angle to obtain a three-dimensional image of the target to be reconstructed.
  • the FPGA-based 3D reconstruction device may be deployed in the FPGA module 110 .
  • Each transformation unit 111 is used to perform Fourier forward transformation on the two-dimensional image at each projection angle to obtain each two-dimensional image after Fourier forward transformation; and perform inverse Fourier transformation based on each two-dimensional image after Fourier forward transformation to obtain an intensity image at each projection angle.
  • Each reconstruction unit 113 is used to divide the target to be reconstructed into several sections; based on the intensity images of each section and each projection angle, obtain the section intensity data corresponding to each section; wherein the section intensity data includes the intensity images of each projection angle corresponding to a section; perform filtered back-projection on each section intensity data in parallel to obtain the slice image corresponding to each section; perform three-dimensional reconstruction based on each slice image to obtain a three-dimensional image of the target to be reconstructed.
  • the reconstruction unit 113 is further configured to perform filtered back-projection in parallel on intensity images of each projection angle corresponding to a section based on one section intensity data among the section intensity data to obtain a slice image corresponding to the section intensity data.
  • the three-dimensional reconstruction system may also include an image scanning module, which is used to scan the target to be reconstructed based on different projection angles to obtain a two-dimensional image corresponding to each projection angle.
  • an image scanning module which is used to scan the target to be reconstructed based on different projection angles to obtain a two-dimensional image corresponding to each projection angle.
  • the process of three-dimensional reconstruction is explained.
  • the two-dimensional image of the target to be reconstructed is transmitted to the FPGA module 110 through the high-speed interface 119, and the two-dimensional image of each projection angle enters a transformation unit 111, and is transformed in parallel in each transformation unit 111.
  • the number of transformation units 111 of the FPGA module 110 depends on the speed of two-dimensional image generation and the speed of processing of the transformation unit 111, and the number can be adaptively adjusted to ensure maximum transformation efficiency.
  • the transformation unit may also include a first transformation unit and a second transformation unit, wherein the first transformation unit is used to perform Fourier forward transform on the two-dimensional image to obtain a two-dimensional image that has completed the Fourier forward transform; and the second transformation unit is used to perform Fourier inverse transform based on the two-dimensional image that has completed the Fourier forward transform to obtain an intensity image.
  • each intensity image generated by the transformation unit 111 is stored in the memory bar 115 via the AXI bus 117 within the FPGA module 110.
  • each reconstruction unit 113 performs reconstruction in parallel based on each intensity image to obtain a three-dimensional image of the target to be reconstructed.
  • the reconstruction unit 113 may include a data indexing unit, a filtering unit and an interpolation back-projection unit.
  • the indexing module is used to divide the target to be reconstructed into several sections to obtain section intensity data corresponding to each section; the filtering unit is used to filter the section intensity data to remove noise; the interpolation back-projection unit is used to filter and back-project the section intensity data to obtain slice images corresponding to each section, and obtain a three-dimensional image of the target to be reconstructed based on the slice images corresponding to each section.
  • An embodiment of the present application provides a 3D reconstruction method based on FPGA. The method is applicable to the 3D reconstruction system 100 in the principle block diagram shown in FIG. 1 .
  • each step of the method is a three-dimensional reconstruction system.
  • the present invention is described with reference to an example, but is not intended to be a specific limitation thereto.
  • the method may include the following steps:
  • Step 310 Acquire at least one two-dimensional image of the object to be reconstructed.
  • the two-dimensional image may be derived from a two-dimensional image obtained by real-time scanning of the target to be reconstructed, or may be a two-dimensional image pre-stored in an electronic device for a historical time period. Then, for the electronic device, after scanning and obtaining the two-dimensional image, the two-dimensional image may be processed in real time, or may be pre-stored and processed, for example, the two-dimensional image may be processed when the occupancy of the electronic device is low, or the two-dimensional image may be processed according to the instructions of the staff. Therefore, the three-dimensional reconstruction in this embodiment may be for the two-dimensional image acquired in real time, or for the two-dimensional image acquired in a historical time period, which is not specifically limited here.
  • Each two-dimensional image corresponds to a projection angle of a target to be reconstructed, and the target to be reconstructed may refer to an object that needs to be three-dimensionally reconstructed.
  • the image scanning module can be used to scan the target to be reconstructed based on different projection angles to obtain two-dimensional images corresponding to each projection angle.
  • the projection angle refers to the angle at which imaging technologies such as X-rays, electron beams or laser beams illuminate the target to be reconstructed from different directions during the 3D reconstruction process.
  • imaging technologies such as X-rays, electron beams or laser beams illuminate the target to be reconstructed from different directions during the 3D reconstruction process.
  • the 2D image at each projection angle can be regarded as the 2D projection of the target to be reconstructed in that direction, which is important information for reconstructing the 3D image.
  • the projection angle When determining the projection angle, multiple factors need to be considered, such as the shape, size, and imaging requirements of the object to be reconstructed. Generally speaking, selecting more projection angles will improve the accuracy and quality of the 3D image, but it will also increase the time and cost of 3D reconstruction. Therefore, it is necessary to balance time and quality and select the most appropriate projection angle. For example, selecting 30 to 200 projection angles can achieve better reconstruction results.
  • the projection angle may be changed by rotating the object to be reconstructed, or by rotating the light, which is not limited here.
  • FIG3 shows a schematic diagram of a specific implementation of three-dimensional reconstruction.
  • the light and its coherent light are irradiated on the target to be reconstructed, and a two-dimensional image is recorded on the CCD or CMOS sensor.
  • the projection angle is 0°.
  • the light or the target to be reconstructed is rotated around the Z axis so that the light produces a projection angle theta relative to the target to be reconstructed, and the two-dimensional image at this theta projection angle is continuously obtained.
  • the two-dimensional image is rotated in sequence so that theta is 1°, 2°, 3°, ..., 179°. In this way, the two-dimensional image of the target to be reconstructed at 180 angles can be obtained.
  • Step 330 transform the two-dimensional images at each projection angle in parallel to obtain an intensity image corresponding to each projection angle.
  • Figure 4 shows the principle diagram of using each two-dimensional image to obtain an intensity image.
  • the object plane coordinates be x0, y0
  • the light wave field be U0 (x0, y0)
  • the diffraction screen coordinates after diffraction at a certain spatial distance d are x, y
  • the diffraction field is U (x, y).
  • Reference Fourier transform and inverse transform symbols The angular spectrum diffraction integral can be expressed as equation (1) and equation (2):
  • fx, fy are frequency domain coordinates
  • j is an imaginary unit
  • k 2 ⁇ / ⁇
  • ⁇ is the wavelength of light.
  • j is an imaginary unit
  • k 1/ ⁇
  • ⁇ is the wavelength of light
  • the Fourier transform can be implemented by the transformation unit in the FPGA unit. Specifically, the following steps are included: performing a Fourier transform on the two-dimensional image at each projection angle to obtain each two-dimensional image that has completed the Fourier transform; performing an inverse Fourier transform based on each two-dimensional image that has completed the Fourier transform to obtain an intensity image at each projection angle.
  • Step 331 Select a set number of two-dimensional images from the two-dimensional images at each of the projection angles for parallel transformation.
  • Step 333 If the parallel transformation is completed for each of the two-dimensional images, a set number of two-dimensional images are selected from the remaining two-dimensional images for parallel transformation until the parallel transformation of the two-dimensional images at each of the projection angles is completed.
  • multiple transform units can be used in parallel in appropriate amounts according to the speed of the 2D image input and the computing resources of the FPGA unit. For example, if the first 2D image has not been processed in the first transform unit and the second 2D image is input, then an idle second transform unit can be used to process the second 2D image.
  • the set number can be determined according to the number of transformation units in the three-dimensional system to achieve maximum parallel transformation efficiency.
  • the set number is equal to the number of transformation units, which is not limited here.
  • the delay in two-dimensional image processing can be reduced, the speed of two-dimensional image processing can be accelerated, and thus the efficiency of three-dimensional reconstruction can be improved.
  • Step 350 performing three-dimensional reconstruction in parallel based on the intensity images at each projection angle to obtain a three-dimensional image of the target to be reconstructed.
  • the three-dimensional reconstruction can be performed using the reconstruction units in the three-dimensional system, as shown in FIG6 , including the following steps:
  • Step 351 dividing the target to be reconstructed into a number of cross sections.
  • Step 353 based on the intensity images of each cross section and each projection angle, obtain the cross-sectional intensity data corresponding to each cross section.
  • Step 355 performing filtered back-projection on the cross-sectional intensity data in parallel to obtain slice images corresponding to each cross section.
  • Step 357 performing three-dimensional reconstruction based on each slice image to obtain a three-dimensional image of the target to be reconstructed.
  • the cross-sectional intensity data includes intensity images at various projection angles corresponding to a cross section.
  • a circle represents a cross section, and the light is scanned around the cross section.
  • the intensity image at each projection angle can obtain a one-dimensional intensity image.
  • the intensity images at multiple angles are arranged according to the angle to form a cross-sectional intensity data.
  • a computer algorithm such as a model fitting algorithm, a voxelization algorithm, a viewpoint-based voxelization algorithm, or a point cloud-based algorithm, which is not limited here.
  • the FPGA unit includes multiple reconstruction units
  • multiple cross-sectional intensity data can be transformed in parallel through multiple reconstruction units.
  • Multiple reconstruction units can be used in parallel in appropriate amounts according to the speed of cross-sectional intensity data input and the computing resources of the FPGA unit. For example, if the first cross-sectional intensity data has not been processed in the first reconstruction unit and the second cross-sectional intensity data is input, then a second reconstruction unit in an idle state can be used to process the second cross-sectional intensity data.
  • the cross-sectional intensity data include intensity images at various projection angles corresponding to a cross-section
  • the intensity images at various projection angles corresponding to a cross-section are filtered back-projected in parallel to obtain a slice image corresponding to the cross-sectional intensity data.
  • a section intensity data includes intensity images corresponding to each projection angle in a section
  • the reconstruction unit simultaneously reconstructs the intensity images corresponding to each projection angle in parallel, then the speed of obtaining the corresponding slice image from the section intensity data can be greatly accelerated.
  • multiple cross-sectional intensity data are processed in parallel by multiple reconstruction units, thereby realizing frame-parallel accelerated processing of multiple cross-sectional intensity data and improving the efficiency of three-dimensional reconstruction; on the other hand, one reconstruction unit performs filtered back-projection on one cross-sectional intensity data in multiple rows in parallel, thereby realizing row-parallel accelerated processing of cross-sectional intensity data, thereby improving the processing speed and processing capacity of each reconstruction unit.
  • the FPGA unit in the three-dimensional reconstruction system is used to perform parallel reconstruction to obtain a three-dimensional image of the target to be reconstructed; on the one hand, the parallel acceleration processing mechanism based on the FPGA unit reduces the power consumption of the three-dimensional reconstruction and ensures the speed of the three-dimensional reconstruction; on the other hand, the FPGA unit has low cost and small size, and is easy to embed into a device that is easy to transplant, effectively solving the problems of high cost and large size of the three-dimensional reconstruction system in the related technology.
  • FIG8 is a schematic diagram of a specific implementation of a 3D reconstruction method based on FPGA, and FIG9 is a flow chart of the 3D reconstruction method.
  • FIG8 is now described in conjunction with FIG9 :
  • step 701 a two-dimensional image of each projection angle of the target to be detected is acquired by using an image scanning unit.
  • step 703 the transform units in the FPGA unit are used to perform Fourier transform on the two-dimensional images in parallel to obtain intensity images.
  • each reconstruction unit in the FPGA unit is used to perform row and frame dual parallel 3D reconstruction based on each intensity image to obtain a 3D image of the target to be reconstructed.
  • the 3D reconstruction system of this scheme realizes the 3D reconstruction of the reconstruction target based on the idea of dual parallel acceleration of rows and frames of the FPGA unit; on the one hand, it can achieve a 3D reconstruction speed similar to that of a GPU, and on the other hand, the 3D system has a small size, low cost, and low power consumption, thus solving the defects of high cost and large size of the 3D reconstruction system in the related technology.
  • the following is an embodiment of the device of the present application, which can be used to execute the FPGA-based 3D reconstruction method involved in the present application.
  • the FPGA-based 3D reconstruction method involved in the present application please refer to the method embodiment of the FPGA-based 3D reconstruction method involved in the present application.
  • a 3D reconstruction device 900 based on FPGA including but not limited to: an image acquisition module 910 , an image transformation module 930 , and a 3D reconstruction module 950 .
  • the image acquisition module 910 is used to acquire at least one two-dimensional image of the target to be reconstructed; wherein each two-dimensional image corresponds to a projection angle of the target to be reconstructed.
  • the image transformation module 930 is used to transform the two-dimensional images at each projection angle in parallel to obtain an intensity image corresponding to each projection angle.
  • the three-dimensional reconstruction module 950 is used to perform three-dimensional reconstruction in parallel based on the intensity images at each projection angle to obtain a three-dimensional image of the target to be reconstructed.
  • the image transformation module 930 also includes multiple transformation units, each transformation unit is used to perform Fourier transform on the two-dimensional image at each projection angle to obtain each two-dimensional image that has completed the Fourier transform; and perform inverse Fourier transform based on each two-dimensional image that has completed the Fourier transform to obtain an intensity image at each projection angle.
  • the three-dimensional reconstruction module 950 also includes multiple reconstruction units, each reconstruction unit is used to divide the target to be reconstructed into a number of sections; based on the intensity images of each section and each projection angle, the section intensity data corresponding to each section is obtained; wherein the section intensity data includes the intensity image of each projection angle corresponding to a section; each section intensity data is filtered back-projected in parallel to obtain a slice image corresponding to each section; three-dimensional reconstruction is performed based on each slice image to obtain a three-dimensional image of the target to be reconstructed.
  • the reconstruction unit is further used to perform filtered back-projection in parallel on intensity images of each projection angle corresponding to a section based on one section intensity data among the section intensity data to obtain a slice image corresponding to the section intensity data.
  • the FPGA-based 3D reconstruction device and the FPGA-based 3D reconstruction system provided in the above embodiments perform FPGA-based 3D reconstruction
  • only the division of the above-mentioned functional modules is used as an example.
  • the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the FPGA-based 3D reconstruction device and the FPGA-based 3D reconstruction system will be divided into different functional modules to complete all or part of the functions described above.
  • the embodiments of the FPGA-based 3D reconstruction device, FPGA-based 3D reconstruction system and FPGA-based 3D reconstruction method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiments and will not be repeated here.
  • the electronic device 4000 may include: a desktop computer, a laptop computer, a server, a mobile device carrying a single-chip microcomputer, etc., which is suitable for the three-dimensional reconstruction system 100 in the implementation environment of Figure 1.
  • the electronic device 4000 includes at least one processor 4001 , at least one communication bus 4002 , and at least one memory 4003 .
  • the processor 4001 and the memory 4003 are connected, such as through a communication bus 4002.
  • the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and/or data reception.
  • the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.
  • Processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
  • the communication bus 4002 may include a path for transmitting information between the above components.
  • the communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc.
  • the communication bus 4002 may be divided into an address bus, a data bus, a control bus, etc.
  • FIG. 11 only uses a thick line, but it does not mean that there is only one bus or one type of bus.
  • the memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or a portable Any other media that carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these.
  • ROM Read Only Memory
  • RAM Random Access Memory
  • EEPROM Electrically Erasable Programmable Read Only Memory
  • CD-ROM Compact Disc Read Only Memory
  • optical disk storage including compact optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.
  • magnetic disk storage medium or other magnetic storage device or
  • the memory 4003 stores a computer program
  • the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002 .
  • a storage medium is provided in an embodiment of the present application, on which a computer program is stored.
  • the computer program is executed by a processor, the FPGA-based three-dimensional reconstruction method in the above-mentioned embodiments is implemented.
  • a computer program product includes a computer program, and the computer program is stored in a storage medium.
  • a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the computer device executes the FPGA-based three-dimensional reconstruction method in the above-mentioned embodiments.
  • this solution uses the FPGA unit in the 3D reconstruction system for parallel reconstruction to obtain a 3D image of the target to be reconstructed.
  • the parallel acceleration processing mechanism based on the FPGA unit reduces the power consumption of the 3D reconstruction and can achieve a 3D reconstruction speed similar to that of a GPU.
  • the FPGA unit has low cost, small size, and low energy consumption, and is easy to embed into easily transplantable devices, such as microscopes, effectively solving the problems of high cost and large size of the 3D reconstruction system in the related technologies.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)

Abstract

本申请实施例提供了一种基于FPGA的三维重建装置、方法、系统及存储介质，涉及三维重建技术领域。其中，该装置包括：图像获取模块，用于获取待重建目标的至少一张二维图像；其中，各所述二维图像分别对应于所述待重建目标的一个投影角度；图像变换模块，用于对各所述投影角度的所述二维图像并行进行变换，得到对应于各投影角度的强度图像；三维重建模块，用于基于各投影角度的强度图像并行进行三维重建，得到所述待重建目标的三维图像。本申请实施例解决了相关技术中三维重建系统成本高、体积大的问题。

Description

基于FPGA的三维重建装置、方法、系统及存储介质 技术领域
本申请涉及三维重建技术领域，具体而言，本申请涉及一种基于FPGA的三维重建装置、方法、系统及存储介质。
背景技术
三维重建是医学成像技术中的热点，目前，主要有两种方式实现对物体的三维重建，一是基于物体大量的二维图片进行拼接，得到三维图片，二是基于物体的三维信息，进行三维的频域到空域、空域到频域等变换，以得到三维图像。
然而，由于三维图像所包含的信息量大，无论利用何种方式实现对物体的三维重建，都会产生巨大的计算开销，因此，当前的三维重建系统，主要利用高性能计算机或者GPU实现对物体的三维重建，但是，通过高性能计算机或者GPU实现三维重建，存在许多缺点，一方面，高性能计算机或者GPU的成本高，另一方面，高性能计算机或者GPU的体积大，不便于嵌入到方便移植的设备。
发明内容
本申请各实施例提供了一种基于FPGA的三维重建装置、方法、系统及存储介质，可以解决相关技术中存在的三维重建系统成本高、体积大的问题。所述技术方案如下：
根据本申请实施例的一个方面，一种基于FPGA的三维重建装置，所述装置包括：图像获取模块，用于获取待重建目标的至少一张二维图像；其中，各所述二维图像分别对应于所述待重建目标的一个投影角度；图像变换模块，用于对各所述投影角度的所述二维图像并行进行变换，得到对应于各投影角度的强度图像；三维重建模块，用于基于各投影角度的强度图像并行进行三维重建，得到所述待重建目标的三维图像。
根据本申请实施例的一个方面，一种基于FPGA的三维重建方法，所述方法包括：获取待重建目标的至少一张二维图像；其中，各所述二维图像分别对应于所述待重建目标的一个投影角度；对各所述投影角度的所述二维图像并行进行变换，得到对应于各投影角度的强度图像；基于各投影角度的强度图像并行进行三维重建，得到所述待重建目标的三维图像。
根据本申请实施例的一个方面，一种基于FPGA的三维重建系统，所述系统包括如上所述的基于FPGA的三维重建装置。
根据本申请实施例的一个方面，一种存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现如上所述的基于FPGA的三维重建方法。
根据本申请实施例的一个方面，一种计算机程序产品，计算机程序产品包括计算机程序，计算机程序存储在存储介质中，计算机设备的处理器从存储介质读取计算机程序，处 理器执行计算机程序，使得计算机设备执行时实现如上所述的基于FPGA的三维重建方法。
本申请提供的技术方案带来的有益效果是：
在上述技术方案中，基于待重建目标的各二维图像，利用三维重建系统中的FPGA单元进行并行重建，得到待重建目标的三维图像；一方面，基于FPGA单元的并行加速处理机制，减少了三维重建的功耗，保证了三维重建的速度，另一方面，FPGA单元的成本低、体积小，便于嵌入到方便移植的设备，有效地解决相关技术中存在的三维重建系统成本高、体积大的问题。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对本申请实施例描述中所需要使用的附图作简单地介绍。
图1是根据本申请所涉及一种基于FPGA的三维重建系统的结构框图；
图2是根据一示例性实施例示出的一种基于FPGA的三维重建方法的流程图；
图3是图2对应实施例所涉及的三维重建的具体实现示意图；
图4是图2对应实施例所涉及的利用各二维图像得到强度图像的原理图；
图5是图2对应实施例中步骤330在一个实施例的流程图；
图6是图2对应实施例中步骤350在一个实施例的流程图；
图7是图7对应实施例所涉及的一个截面强度数据的具体实现示意图；
图8至图9是一种基于FPGA的三维重建方法的具体实现示意图；
图10是根据一示例性实施例示出的一种基于FPGA的三维重建装置的结构框图；
图11是根据一示例性实施例示出的一种电子设备的结构框图。
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本申请，而不能解释为对本申请的限制。
本技术领域技术人员可以理解，除非特意声明，这里使用的单数形式“一”、“一个”、“所述”和“该”也可包括复数形式。应该进一步理解的是，本申请的说明书中使用的措辞“包括”是指存在所述特征、整数、步骤、操作、元件和/或组件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。应该理解，当我们称元件被“连接”或“耦接”到另一元件时，它可以直接连接或耦接到其他元件，或者也可以存在中间元件。此外，这里使用的“连接”或“耦接”可以包括无线连接或无线耦接。这里使用的措辞“和/或”包括一个或更多个相关联的列出项的全部或任一单元和全部组合。
细胞、组织显微成像是现代生物医学领域一项重要的成像技术，为医生、科研人员对医学和生命科学的研究提供了巨大辅助和有力证据。但是传统的二维成像方法已经不足够满足日益发展的科研和医学诊断要求，三维重建技术早已顺势发展开来。三维重建技术不仅可以全方位展示被测物体内部微观结构，还能体现被测物体内部细节之间的关系，可以提供远远大于二维图像的信息，因此三维重建技术是当前的一个研究热点。三维重建的一种经典方法是层析重建。层析法是指按层获取被测物的信息然后通过算法进行三维重建。显微成像中，按层获取物体信息的经典方法是全息成像。
全息成像是显微三维层析重建的提前，利用多角度的全息图重建三维物体的像。全息层析三维重建算法根据是否考虑衍射效应可能分为考虑衍射效应的层析重建和不考虑衍射的层析重建两种。考虑到样品衍射的层析认为光在样品中除了沿着直线传播，而且光波穿过物体时还发生衍射现象，这种算法是基于傅里叶衍射投影理论，能重建分辨率比较高的三维图像，但是它的算法理论比较复杂，基于会利用到三维傅里叶变换等，有傅里叶映射算法、滤波反传播算法等；不考虑样品的层析认为光直线穿过样品，并且没有衍射效应，这种算法基于傅里叶中心切片定理，主要算法包括滤波反投影、直接反投影、迭代重建算法等。
现有的全息层析三维重建技术大都是基于高性能计算机或者GPU实现的，在高性能计算机上利用CPU实现三维重建的方法对计算机性能要求高，而且速度慢，在GPU上实现能够加速计算，但是GPU成本高，而且功耗大，不便于嵌入到方便移植的设备中，例如，不适用于嵌入式便携显微镜。
由上可知，相关技术中仍存在三维重建系统成本高、体积大的缺陷。
为此，本申请提供的基于FPGA的三维重建装置，能够在保证三维重建速度的同时，有效地缩小三维重建系统的体积，减少三维重建系统的成本，解决相关技术中存在的三维重建装置成本高、体积大的问题。该基于FPGA的三维重建装置可部署于三维重建系统中，还可部署于配置于电子设备，例如，该电子设备可以是配置冯诺依曼体系结构的台式电脑、笔记本电脑、服务器，还可以是携带单片机的移动设备等等。进一步地，还提供了一种基于FPGA的三维重建方法，相应地，该基于FPGA的三维重建方法适用于基于FPGA的三维重建装置。
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
请参见图1，图1示出了一种基于FPGA的三维重建系统的原理框图，如图1所示，三维重建系统100包括现场可编程逻辑门阵列FPGA模块110；其中，FPGA模块110可以包括多个变换单元111、多个重建单元113和内存条115、AXI总线117及高速接口119。
其中，FPGA模块110，用于获取待重建目标的至少一张二维图像；其中，各二维图像分别对应于待重建目标的一个投影角度；对各投影角度的二维图像并行进行变换，得到 对应于各投影角度的强度图像；基于各投影角度的强度图像并行进行三维重建，得到待重建目标的三维图像。
在一个可能的实现方式，基于FPGA的三维重建装置可部署于FPGA模块110中。
各变换单元111，用于对各投影角度的二维图像进行傅里叶正变换，得到完成傅里叶正变换的各二维图像；基于完成傅里叶正变换的各二维图像进行傅里叶逆变换，得到各投影角度的强度图像。
各重建单元113，用于将待重建目标划分为若干个截面；基于各截面以及各投影角度的强度图像，得到对应于各截面的截面强度数据；其中，截面强度数据包括一个截面对应的各投影角度的强度图像；分别对各截面强度数据并行进行滤波反投影，得到各截面对应的切片图像；基于各切片图像进行三维重建，得到待重建目标的三维图像。
在一个实施例中，重建单元113，还用于基于各截面强度数据中的一个截面强度数据，对一个截面对应的各投影角度的强度图像并行进行滤波反投影，得到对应于截面强度数据的切片图像。
进一步地，在一个实施例中，三维重建系统还可以包括图像扫描模块，用于基于不同投影角度扫描待重建目标，得到各投影角度对应的二维图像。
现结合图1的三维重建系统，对三维重建的流程进行说明，具体地，如图1所示，待重建目标的二维图像通过高速接口119传入FPGA模块110，每个投影角度的二维图像进入一个变换单元111，在各变换单元111中并行进行变换，其中，FPGA模块110的变换单元111的数量取决于二维图像生成的速度、以及变换单元111处理的速度，可以对其数量进行适应性调整，保证最大化变换效率。
在一个实施例中，变换单元还可以包括一个第一变换单元和一个第二变换单元，第一变换单元用于对二维图像进行傅里叶正变换，得到完成傅里叶正变换的二维图像；第二变换单元用于基于完成傅里叶正变换的二维图像进行傅里叶逆变换，得到强度图像。
随后，变换单元111生成的各强度图像通过FPGA模块110片内AXI总线117存于内存条115中，当全部投影角度的强度图像存储完毕，各重建单元113基于各强度图像并行进行重建，以得到待重建目标的三维图像。
在一个实施例中，重建单元113可以包括数据索引单元、滤波单元和插值反投影单元，索引模块用于将待重建目标划分为若干个截面，以得到各截面对应的截面强度数据；滤波单元用于对各截面强度数据进行滤波，以去除噪音；插值反投影单元用于对各截面强度数据进行滤波反投影，以得到各截面对应的切片图像，并根据各截面对应的切片图像，得到待重建目标的三维图像。
请参阅图2，本申请实施例提供了一种基于FPGA的三维重建方法，该方法适用于图1所示出原理框图中的三维重建系统100。
在下述方法实施例中，为了便于描述，以该方法各步骤的执行主体为三维重建系统为 例进行说明，但是并非对此构成具体限定。
如图2所示，该方法可以包括以下步骤：
步骤310，获取待重建目标的至少一张二维图像。
关于二维图像，可以来源于对待重建目标实时扫描得到的二维图像，也可以是预先存储于电子设备的一历史时间段的二维图像。那么，对于电子设备而言，在扫描得到二维图像之后，可以实时处理二维图像，还可以预先存储了再处理，例如，在电子设备的占用低的时候处理二维图像，或者，根据工作人员的指示处理二维图像。由此，本实施例中的三维重建可以针对实时获取到的二维图像，也可以针对历史时间段获取到的二维图像，在此并未进行具体限定。
其中，各二维图像分别对应于待重建目标的一个投影角度，待重建目标可以是指需要进行三维重建的物体。可以利用图像扫描模块，基于不同投影角度扫描待重建目标，得到各投影角度对应的二维图像。
需要说明的是，投影角度是指在三维重建过程中，X射线、电子束或激光束等成像技术从不同方向照射待重建目标时的角度。在三维重建过程中，为了获得足够的二维图像，需要从不同方向照射待重建目标，收集多个投影角度的二维图像。每个投影角度的二维图像可以看做是待重建目标在该方向上的二维投影，是重建三维图像的重要信息。
在确定投影角度时，需要考虑多个因素，例如待重建目标的形状、尺寸和成像要求等。一般而言，选择更多的投影角度会提高三维图像的精度和质量，但同时也会增加三维重建的时间和成本。因此，需要在时间和质量之间进行权衡，选择最合适的投影角度。例如，选择30到200个投影角度可以获得较好的重建效果。
关于改变投影角度，可以通过旋转待重建目标改变投影角度，也可以旋转光线改变投影角度，在此不作限定。
图3中示出了一个三维重建的具体实现示意图，如图3所示，光线和其相干光照射在待重建目标上，在CCD或CMOS传感器上记录到一个二维图像，此时投影角度为0°，绕Z轴旋转光线或者待重建目标，使得光线相对待重建目标产生一个投影角度theta，继续获得此theta投影角度下的二维图像，依次旋转，使theta为1°，2°，3°，……，179°，这样就可以获得待重建目标在180个角度的二维图像。
步骤330，对各投影角度的二维图像并行进行变换，得到对应于各投影角度的强度图像。
关于对二维图像进行变换，可以通过角谱法实现，图4中示出了利用各二维图像得到强度图像的原理图，如图4所示，在直角坐标系中,令物平面坐标为x0,y0,光波场为U0(x0,y0),经某一空间距离d衍射后的衍射屏坐标为x,y,衍射场为U(x,y)。引用傅里叶变换及逆变换符号角谱衍射积分可以写为式(1)和式(2)：

其中，式中fx，fy为频域坐标，j是虚数单位，k＝2π/λ，λ为光波长。根据上式反向求解U0(x0,y0)即为强度图像，如式(3)：
其中，式中j是虚数单位，k＝1/λ，λ为光波长。
由上可知，角谱法中应用了一次二维傅里叶正变换和一次二维傅里叶逆变换。
在一个实施例中，可以通过FPGA单元中的变换单元实现傅里叶变换，具体地，包括以下步骤：对各投影角度的二维图像进行傅里叶正变换，得到完成傅里叶正变换的各二维图像；基于完成傅里叶正变换的各二维图像进行傅里叶逆变换，得到各投影角度的强度图像。
进一步说明的是，由于FPGA单元中有多个变换单元，通过多个变换单元，可以并行变换多个二维图像，具体地，在一个实施例中，如图5所示，包括以下步骤：
步骤331，由各所述投影角度的二维图像中选取设定数量的二维图像进行并行变换。
步骤333，若各所述二维图像完成所述并行变换，则由其余的二维图像中选取设定数量的二维图像进行并行变换，直至完成对各所述投影角度的二维图像的并行变换。
首先说明的是，可以根据二维图像输入的速度、以及FPGA单元的计算资源适量并行使用多个变换单元。例如，若第一张二维图像在第一变换单元中还未处理结束，又输入了第二张二维图像，那么就可以使用一个处于空闲状态的第二变换单元来处理第二张二维图像。
基于此，设定数量可以根据三维系统中变换单元的数量决定，以达到最大并行变换效率，例如，设定数量等于变换单元的数量，在此不作限定。
通过上述实施例，利用FPGA单元中多个变换单元并行计算，可以减少二维图像处理时的延时，加快二维图像处理的速度，从而提高三维重建的效率。
步骤350，基于各投影角度的强度图像并行进行三维重建，得到待重建目标的三维图像。
其中，可以利用三维系统中的各重建单元进行三维重建，如图6所示，包括以下步骤：
步骤351，将待重建目标划分为若干个截面。
步骤353，基于各截面以及各投影角度的强度图像，得到对应于各截面的截面强度数据。
步骤355，分别对截面强度数据并行进行滤波反投影，得到各截面对应的切片图像。
步骤357，基于各切片图像进行三维重建，得到待重建目标的三维图像。
其中，截面强度数据包括一个截面对应的各投影角度的强度图像。
关于截面强度数据，回请参见图3，如图3所示，z＝0时待重建目标的截面，可以对应各个投影角度的强度图像，从而组成一个180个投影角度的强度图像，进而，该180个投影角度的强度图像就是截面强度数据。
举例来说，如图7所示，圆代表一个截面，光线绕着截面扫描，每个投影角度的强度图像可以得到一条一维的强度图像，多个角度的强度图像按角度排列，就组成了一个截面强度数据。
进而，可以根据滤波反投影法，由截面强度数据重建得到待重建目标的z＝0的切片图像。因此，遍历所有的z就可以重建出各截面对应的切片图像，将所有切片图像组合在一起进行三维重建，则可以得到待重建目标的三维图像。
关于将所有切片图像组合在一起进行三维重建，可以使用计算机算法实现，例如模型拟合算法、体素化算法、基于视角的体素化算法或基于点云的算法，在此不作限定。
进一步地，由于FPGA单元包括多个重建单元，因此，通过多个重建单元，可以并行变换多个截面强度数据。可以根据截面强度数据输入的速度、以及FPGA单元的计算资源适量并行使用多个重建单元。例如，若第一张截面强度数据在第一重建单元中还未处理结束，又输入了第二张截面强度数据，那么就可以使用一个处于空闲状态的第二重建单元来处理第二张截面强度数据。
当然，由于截面强度数据包括一个截面对应的各投影角度的强度图像，那么，利用FPGA单元中重建单元的并行计算特点，在一个可能的实现方式，基于各截面强度数据中的一个截面强度数据，对一个截面对应的各投影角度的强度图像并行进行滤波反投影，得到对应于截面强度数据的切片图像。
可以理解，由于一个截面强度数据包括一个截面中各投影角度对应的强度图像，那么，若重建单元同时对各投影角度对应的强度图像进行并行重建，那么，能够大大加快由该截面强度数据得到对应的切片图像的速度。
通过上述过程，一方面，通过多个重建单元并行处理多个截面强度数据，实现了对多个截面强度数据的帧并行加速处理，加快了三维重建的效率；另一方面，通过一个重建单元多行并行地对一个截面强度数据进行滤波反投影，实现了对截面强度数据的行并行加速处理，从而提升了每个重建单元的处理速度和处理能力。
综上所述，在上述实施例的结合下，基于待重建目标的各二维图像，利用三维重建系统中的FPGA单元进行并行重建，得到待重建目标的三维图像；一方面，基于FPGA单元的并行加速处理机制，减少了三维重建的功耗，保证了三维重建的速度，另一方面，FPGA单元的成本低、体积小，便于嵌入到方便移植的设备，有效地解决相关技术中存在的三维重建系统成本高、体积大的问题。
图8是一种基于FPGA的三维重建方法的具体实现示意图，图9为该三维重建方法的流程图，现结合图9对图8进行说明：
通过步骤701，利用图像扫描单元获取待检测目标各投影角度的二维图像。
通过步骤703，利用FPGA单元中的各变换单元，对各二维图像并行进行傅里叶变换，得到各强度图像。
通过步骤705，利用FPGA单元中的各重建单元，基于各强度图像进行行、帧双并行三维重建，得到待重建目标的三维图像。
由上可知，本方案的三维重建系统，基于FPGA单元的行、帧双并行加速的思路，实现对待重建目标的三维重建；一方面，可以达到类比GPU的三维重建速度，另一方面，该三维系统的体积小、成本低、功耗少，从而解决了相关技术中存在的三维重建系统成本高、体积大的缺陷。
下述为本申请装置实施例，可以用于执行本申请所涉及的基于FPGA的三维重建方法。对于本申请装置实施例中未披露的细节，请参照本申请所涉及的基于FPGA的三维重建方法的方法实施例。
请参阅图10，本申请实施例中提供了一种基于FPGA的三维重建装置900，包括但不限于：图像获取模块910、图像变换模块930、以及三维重建模块950。
其中，图像获取模块910，用于获取待重建目标的至少一张二维图像；其中，各二维图像分别对应于待重建目标的一个投影角度。
图像变换模块930，用于对各投影角度的二维图像并行进行变换，得到对应于各投影角度的强度图像。
三维重建模块950，用于基于各投影角度的强度图像并行进行三维重建，得到待重建目标的三维图像。
在一个实施例中，图像变换模块930还包括多个变换单元，各变换单元，用于对各投影角度的二维图像进行傅里叶正变换，得到完成傅里叶正变换的各二维图像；基于完成傅里叶正变换的各二维图像进行傅里叶逆变换，得到各投影角度的强度图像。
在一个实施例中，三维重建模块950还包括多个重建单元，各重建单元，用于将待重建目标划分为若干个截面；基于各截面以及各投影角度的强度图像，得到对应于各截面的截面强度数据；其中，截面强度数据包括一个截面对应的各投影角度的强度图像；分别对各截面强度数据并行进行滤波反投影，得到各截面对应的切片图像；基于各切片图像进行三维重建，得到待重建目标的三维图像。
在一个实施例中，重建单元，还用于基于各截面强度数据中的一个截面强度数据，对一个截面对应的各投影角度的强度图像并行进行滤波反投影，得到对应于截面强度数据的切片图像。
需要说明的是，上述实施例所提供的基于FPGA的三维重建装置和基于FPGA的三维重建系统在进行基于FPGA的三维重建时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即基于FPGA的三维重建装置、以及基于FPGA的三维重建系统的内部结构将划分为不同的功能模块，以完成以上描述的全部或者部分功能。
另外，上述实施例所提供的基于FPGA的三维重建装置、基于FPGA的三维重建系统与基于FPGA的三维重建方法的实施例属于同一构思，其中各个模块执行操作的具体方式已经在方法实施例中进行了详细描述，此处不再赘述。
请参阅图11，本申请实施例中提供了一种电子设备4000，该电子设备4000可以包括：台式电脑、笔记本电脑、服务器、携带单片机的移动设备等，适用于图1实施环境中的三维重建系统100。
在图11中，该电子设备4000包括至少一个处理器4001、至少一条通信总线4002以及至少一个存储器4003。
其中，处理器4001和存储器4003相连，如通过通信总线4002相连。可选地，电子设备4000还可以包括收发器4004，收发器4004可以用于该电子设备与其他电子设备之间的数据交互，如数据的发送和/或数据的接收等。需要说明的是，实际应用中收发器4004不限于一个，该电子设备4000的结构并不构成对本申请实施例的限定。
处理器4001可以是CPU(Central Processing Unit，中央处理器)，通用处理器，DSP(Digital Signal Processor，数据信号处理器)，ASIC(Application Specific Integrated Circuit，专用集成电路)，FPGA(Field Programmable Gate Array，现场可编程门阵列)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器4001也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等。
通信总线4002可包括一通路，在上述组件之间传送信息。通信总线4002可以是PCI(Peripheral Component Interconnect，外设部件互连标准)总线或EISA(Extended Industry Standard Architecture，扩展工业标准结构)总线等。通信总线4002可以分为地址总线、数据总线、控制总线等。为便于表示，图11中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
存储器4003可以是ROM(Read Only Memory，只读存储器)或可存储静态信息和指令的其他类型的静态存储设备，RAM(Random Access Memory，随机存取存储器)或者可存储信息和指令的其他类型的动态存储设备，也可以是EEPROM(Electrically Erasable Programmable Read Only Memory，电可擦可编程只读存储器)、CD-ROM(Compact Disc Read Only Memory，只读光盘)或其他光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其他磁存储设备、或者能够用于携 带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。
存储器4003上存储有计算机程序，处理器4001通过通信总线4002读取存储器4003中存储的计算机程序。
该计算机程序被处理器4001执行时实现上述各实施例中的基于FPGA的三维重建方法。
此外，本申请实施例中提供了一种存储介质，该存储介质上存储有计算机程序，该计算机程序被处理器执行时实现上述各实施例中的基于FPGA的三维重建方法。
本申请实施例中提供了一种计算机程序产品，该计算机程序产品包括计算机程序，该计算机程序存储在存储介质中。计算机设备的处理器从存储介质读取该计算机程序，处理器执行该计算机程序，使得该计算机设备执行上述各实施例中的基于FPGA的三维重建方法。
与相关技术相比，本方案利用三维重建系统中的FPGA单元进行并行重建，得到待重建目标的三维图像；一方面，基于FPGA单元的并行加速处理机制，减少了三维重建的功耗，可以达到类比GPU的三维重建速度，另一方面，FPGA单元的成本低、体积小、能耗小，便于嵌入到方便移植的设备，例如，显微镜，有效地解决相关技术中存在的三维重建系统成本高、体积大的问题。
应该理解的是，虽然附图的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，其可以以其他的顺序执行。而且，附图的流程图中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，其执行顺序也不必然是依次进行，而是可以与其他步骤或者其他步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
以上所述仅是本申请的部分实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本申请的保护范围。

Claims (8)

  1. 一种基于FPGA的三维重建装置，其特征在于，应用于FPGA，所述FPGA包括三维重建装置、总线单元和存储单元，所述装置包括：
    图像获取模块，用于获取待重建目标的至少一张二维图像；其中，各所述二维图像分别对应于所述待重建目标的一个投影角度；
    图像变换模块，所述图像变换模块包括多个变换单元；各所述变换单元用于对各所述投影角度的所述二维图像并行进行变换，得到对应于各投影角度的强度图像，并通过所述总线单元将各所述强度图像存储于所述存储单元中；
    三维重建模块，所述三维重建模块包括多个重建单元；各所述重建单元，用于获取所述存储单元中的各所述强度图像，将所述待重建目标划分为若干个截面；基于各所述截面以及各所述投影角度的强度图像，得到对应于各所述截面的截面强度数据；其中，所述截面强度数据包括一个所述截面对应的各所述投影角度的强度图像；分别对各所述截面强度数据并行进行滤波反投影，得到各所述截面对应的切片图像；基于各所述切片图像进行三维重建，得到所述待重建目标的三维图像。
  2. 如权利要求1所述的装置，其特征在于，所述图像变换模块包括多个变换单元；
    各所述变换单元，用于对各所述投影角度的二维图像进行傅里叶正变换，得到完成所述傅里叶正变换的各二维图像；基于完成所述傅里叶正变换的各所述二维图像进行傅里叶逆变换，得到各所述投影角度的强度图像。
  3. 如权利要求1所述的装置，其特征在于，所述重建单元，还用于基于各所述截面强度数据中的一个截面强度数据，对一个所述截面对应的各投影角度的强度图像并行进行滤波反投影，得到对应于所述截面强度数据的切片图像。
  4. 一种基于FPGA的三维重建方法，其特征在于，应用于FPGA模块，所述FPGA包括三维重建装置、总线单元和存储单元，所述方法包括：
    获取待重建目标的至少一张二维图像；其中，各所述二维图像分别对应于所述待重建目标的一个投影角度；
    对各所述投影角度的所述二维图像并行进行变换，得到对应于各投影角度的强度图像；并通过所述总线单元将各所述强度图像存储于所述存储单元中；
    获取所述存储单元中的各所述强度图像，将所述待重建目标划分为若干个截面；
    基于各所述截面以及各所述投影角度的强度图像，得到对应于各所述截面的截面强度数据，其中，所述截面强度数据包括一个所述截面对应的各投影角度的强度图像；
    分别对所述截面强度数据并行进行滤波反投影，得到各所述截面对应的切片图像；基于各所述切片图像进行三维重建，得到所述待重建目标的三维图像。
  5. 如权利要求4所述的方法，其特征在于，所述对各所述投影角度的所述二维图像并行进行变换，得到对应于各投影角度的强度图像，包括：
    由各所述投影角度的二维图像中选取设定数量的二维图像进行并行变换；
    若各所述二维图像完成所述并行变换，则由其余的二维图像中选取设定数量的二维图像进行并行变换，直至完成对各所述投影角度的二维图像的并行变换。
  6. 一种基于FPGA的三维重建系统，其特征在于，所述系统包括FPGA，所述FPGA包括如权利要求1至3任一项所述的三维重建装置。
  7. 如权利要求6所述的系统，其特征在于，所述系统还包括图像扫描模块；
    所述图像扫描模块，用于基于不同投影角度扫描待重建目标，得到各投影角度对应的二维图像。
  8. 一种存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求4至5中任一项所述的基于FPGA的三维重建方法。
PCT/CN2023/113481 2023-03-06 2023-08-17 基于fpga的三维重建装置、方法、系统及存储介质 Ceased WO2024183239A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310204311.9 2023-03-06
CN202310204311.9A CN116051754B (zh) 2023-03-06 2023-03-06 基于fpga的三维重建装置、方法、系统及存储介质

Publications (1)

Publication Number Publication Date
WO2024183239A1 true WO2024183239A1 (zh) 2024-09-12

Family

ID=86133278

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/CN2023/113481 Ceased WO2024183239A1 (zh) 2023-03-06 2023-08-17 基于fpga的三维重建装置、方法、系统及存储介质
PCT/CN2023/133577 Ceased WO2024183342A1 (zh) 2023-03-06 2023-11-23 基于fpga的三维重建装置、方法、系统及存储介质

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/133577 Ceased WO2024183342A1 (zh) 2023-03-06 2023-11-23 基于fpga的三维重建装置、方法、系统及存储介质

Country Status (2)

Country Link
CN (1) CN116051754B (zh)
WO (2) WO2024183239A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116051754B (zh) * 2023-03-06 2023-09-22 中国科学院深圳先进技术研究院 基于fpga的三维重建装置、方法、系统及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040264634A1 (en) * 2003-06-25 2004-12-30 General Electric Company Fourier based method, apparatus, and medium for optimal reconstruction in digital tomosynthesis
CN106248340A (zh) * 2016-07-08 2016-12-21 西南科技大学 一种基于三维超声成像技术的风洞模型3d冰形在线测量方法
CN115393403A (zh) * 2022-09-14 2022-11-25 浙江理工大学 融合多组双目视觉系统的管径测量方法、装置及设备
CN115644811A (zh) * 2022-10-20 2023-01-31 电子科技大学 一种基于光学相干断层成像系统的快速的投影图像重建方法
CN116051754A (zh) * 2023-03-06 2023-05-02 中国科学院深圳先进技术研究院 基于fpga的三维重建装置、方法、系统及存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2324832A1 (en) * 2000-10-31 2002-04-30 Roger Lecomte Real-time image reconstruction for computed tomography systems
CN103366389A (zh) * 2013-04-27 2013-10-23 中国人民解放军北京军区总医院 Ct图像重建方法
CN105527306B (zh) * 2016-01-15 2018-05-08 北京工业大学 一种基于面阵式探测器太赫兹层析三维成像的方法
US20180018757A1 (en) * 2016-07-13 2018-01-18 Kenji Suzuki Transforming projection data in tomography by means of machine learning
US11326870B2 (en) * 2018-01-29 2022-05-10 Duke University Systems and methods for enhanced resolution imaging based on multiple cross-sectional images acquired at different angles
CN111080779A (zh) * 2019-12-23 2020-04-28 南阳理工学院 一种基于fpga的实时三维重建系统及其方法
CN111462266B (zh) * 2020-03-20 2023-08-04 北京东软医疗设备有限公司 图像重建方法、装置、ct设备及ct系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040264634A1 (en) * 2003-06-25 2004-12-30 General Electric Company Fourier based method, apparatus, and medium for optimal reconstruction in digital tomosynthesis
CN106248340A (zh) * 2016-07-08 2016-12-21 西南科技大学 一种基于三维超声成像技术的风洞模型3d冰形在线测量方法
CN115393403A (zh) * 2022-09-14 2022-11-25 浙江理工大学 融合多组双目视觉系统的管径测量方法、装置及设备
CN115644811A (zh) * 2022-10-20 2023-01-31 电子科技大学 一种基于光学相干断层成像系统的快速的投影图像重建方法
CN116051754A (zh) * 2023-03-06 2023-05-02 中国科学院深圳先进技术研究院 基于fpga的三维重建装置、方法、系统及存储介质

Also Published As

Publication number Publication date
CN116051754A (zh) 2023-05-02
CN116051754B (zh) 2023-09-22
WO2024183342A1 (zh) 2024-09-12

Similar Documents

Publication Publication Date Title
JP6259531B2 (ja) 画像フィルタリング方法及びｃｔシステム
Yu et al. Scalable and accurate multi-GPU-based image reconstruction of large-scale ptychography data
Birk et al. A comprehensive comparison of GPU-and FPGA-based acceleration of reflection image reconstruction for 3D ultrasound computer tomography
KR101978728B1 (ko) 스마트 기기를 이용한 딥 러닝 기반 의료용 초음파 영상 분류 방법 및 장치
CN108937853A (zh) 一种多端口并行处理的三维光声显微成像方法及成像系统
WO2024183342A1 (zh) 基于fpga的三维重建装置、方法、系统及存储介质
Hahne et al. Rf-ulm: Ultrasound localization microscopy learned from radio-frequency wavefronts
Ruhlandt et al. Three-dimensional propagation in near-field tomographic X-ray phase retrieval
Yang et al. High frame rate 3-D ultrasound imaging using separable beamforming
CN112288847B (zh) 基于快速傅里叶变换的光场三维重建方法
WO2024183383A1 (zh) 一种全息层析三维显微镜
Heigl et al. High-speed reconstruction for C-arm computed tomography
Romero et al. Using GPUs for beamforming acceleration on SAFT imaging
CN114967397B (zh) 一种无透镜全息三维成像构建方法及装置
Tao et al. Sparse-view phase-contrast and attenuation-based CT reconstruction utilizing model-driven deep learning
CN118314237A (zh) 基于cuda架构的工业锥束ct三维重建方法及系统
Ahmad et al. Undecimated discrete wavelets transform-based image fusion and de-noising in FPGA
KR102591860B1 (ko) 누락된 원뿔 아티팩트를 제거하기 위한 비지도 학습 기반 뉴럴 네트워크를 이용한 단층촬영영상 처리 방법 및 그 장치
Wen et al. GPU-based volume reconstruction for freehand 3D ultrasound imaging
Shkarin et al. GPU-optimized direct Fourier method for on-line tomography
Hansen et al. Synthetic aperture beamformation using the GPU
Romanenko et al. A three-dimensional deconvolution algorithm using graphic processors
González-Ruiz et al. FlowDenoising: Structure-preserving denoising in 3D electron microscopy (3DEM)
CN121254283B (zh) 一种基于行列寻址超声阵列的超快三维波数域成像方法
JP2005326332A (ja) 画像情報処理装置および方法、並びにプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23925943

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23925943

Country of ref document: EP

Kind code of ref document: A1