CN114998150B - Three-dimensional reconstruction method and device for ultrasonic image - Google Patents
Three-dimensional reconstruction method and device for ultrasonic image Download PDFInfo
- Publication number
- CN114998150B CN114998150B CN202210681854.5A CN202210681854A CN114998150B CN 114998150 B CN114998150 B CN 114998150B CN 202210681854 A CN202210681854 A CN 202210681854A CN 114998150 B CN114998150 B CN 114998150B
- Authority
- CN
- China
- Prior art keywords
- interpolation
- space
- tetrahedron
- reconstruction
- lattice points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000013507 mapping Methods 0.000 claims abstract description 25
- 238000004364 calculation method Methods 0.000 claims abstract description 17
- 230000004927 fusion Effects 0.000 claims abstract description 7
- 238000002604 ultrasonography Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 9
- 238000013135 deep learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
- Image Processing (AREA)
Abstract
The three-dimensional reconstruction method and the device of the ultrasonic image can reduce the influence of parameter imbalance caused by uneven acquisition density, have lower reconstruction error, increase the application range, have independence in the interpolation process of different voxels, can perform parallel calculation, and greatly improve the reconstruction efficiency. The method comprises the following steps: (1) spatial mapping: taking pixels in a two-dimensional image obtained from an ultrasonic acquisition system as plane point clouds with only central values, and obtaining point clouds in a corresponding space coordinate system through space mapping; (2) tetrahedral subdivision: dividing a space under a new coordinate system into a plurality of tetrahedrons consisting of point clouds as vertexes; (3) tetrahedral interpolation: interpolation is carried out on lattice points in tetrahedrons, tetrahedron vertexes are obtained through pixel mapping, interpolation calculation is carried out on the lattice points at positions corresponding to the voxels, the interpolation method adopts fusion of volume weight interpolation and space second-order polynomial interpolation, parallel calculation is not affected by interpolation of each tetrahedron, and three-dimensional volume data after reconstruction is obtained.
Description
Technical Field
The invention relates to the technical field of medical image processing, in particular to a three-dimensional reconstruction method of an ultrasonic image and a three-dimensional reconstruction device of the ultrasonic image.
Background
The ultrasonic image is widely applied in clinical diagnosis due to the characteristics of low cost, no radiation and real-time property, and the handheld ultrasonic has the advantages of being free in scanning mode, capable of providing a larger imaging visual angle, higher image resolution and the like, is relatively in line with the habit of doctors and the environment of an operating room, is the main research direction of ultrasonic image guided interventional operation, and is greatly focused in recent years. The principle is that the two-dimensional images acquired by the ultrasonic instrument are reconstructed into three-dimensional information by utilizing the spatial conversion relation between the images, so that the risk of misdiagnosis can be reduced, and the method has wide application in the aspects of auxiliary diagnosis and treatment.
Three-dimensional ultrasound reconstruction algorithms can be divided into three categories depending on the implementation: voxel-based methods, pixel-based methods, and function-based methods. Voxel-based methods include voxel nearest neighbor and distance weighted interpolation. The pixel-based method includes pixel nearest neighbor and kernel regression interpolation. The function-based method comprises interpolation calculation functions such as radial basis functions and Bessel functions, wherein the radial basis functions are smoothly connected through global interpolation after block calculation, the efficiency is low, and the Bessel functions improve the reconstruction efficiency, but all acquired images are required to be arranged according to the position sequence, namely single unidirectional scanning, so that the application scene is limited.
The following situations exist in practical application: the image is unevenly distributed in space, so that global parameters are difficult to select, the calculation efficiency of a local self-adaptive method is low, additional filling operation is needed for possible holes, and the function-based interpolation method is limited by a function model. Recently, due to the development of deep learning technology, raphael et al estimate motion information from an ultrasonic sequence by adopting CNN, a reconstruction task under a complex condition is realized by combining an inertial measurement unit, guo et al adopts 3D convolution to extract features in the ultrasonic sequence, and a focus module focuses on a speckle region which is easy to extract the motion information, so that 3D ultrasonic reconstruction is performed under the condition of no tracking device. However, compared with the error of the conventional tracking device, the spatial position of the pixels is not recovered accurately enough by adopting the deep learning technology at the present stage, and the clinical use is difficult. A new reconstruction method is needed to meet the practical application requirements.
Disclosure of Invention
In order to overcome the defects of the prior art, the technical problem to be solved by the invention is to provide the three-dimensional reconstruction method of the ultrasonic image, which can reduce the influence of parameter imbalance caused by uneven acquisition density, has lower reconstruction error, increases the application range, has independence in the interpolation process of different voxels, can perform parallel calculation, and greatly improves the reconstruction efficiency.
The technical scheme of the invention is as follows: the three-dimensional reconstruction method of the ultrasonic image comprises the following steps:
(1) Spatial mapping: taking pixels in a two-dimensional image obtained from an ultrasonic acquisition system as plane point clouds with only central values, and obtaining point clouds in a corresponding space coordinate system through space mapping;
(2) Tetrahedral subdivision: dividing a space under a new coordinate system into a plurality of tetrahedrons consisting of point clouds as vertexes;
(3) Tetrahedral interpolation: interpolation is carried out on lattice points in tetrahedrons, tetrahedron vertexes are obtained through pixel mapping, interpolation calculation is carried out on the lattice points at positions corresponding to the voxels, the interpolation method adopts fusion of volume weight interpolation and space second-order polynomial interpolation, parallel calculation is not affected by interpolation of each tetrahedron, and three-dimensional volume data after reconstruction is obtained.
According to the method, the purpose of self-adaptive neighborhood segmentation is achieved by carrying out tetrahedral subdivision on the space, a proper neighborhood is provided for interpolation points under different acquisition densities, and the influence of parameter imbalance caused by uneven acquisition densities is reduced; the volume weight interpolation and the spatial second-order polynomial interpolation are fused to the voxels to be interpolated in the space, so that the reconstruction error is lower, and the application range of the method is increased; the interpolation process of different voxels has independence, can be calculated in parallel, and greatly improves the reconstruction efficiency.
There is also provided a three-dimensional reconstruction apparatus of an ultrasound image, comprising:
A space mapping module configured to treat pixels in a two-dimensional image obtained from an ultrasonic acquisition system as a plane point cloud with only a central value, and obtain a point cloud in a corresponding space coordinate system through space mapping;
the tetrahedron subdivision module is configured to divide the space under the new coordinate system into a plurality of tetrahedrons consisting of point clouds as vertexes;
The tetrahedron interpolation module is configured to interpolate the lattice points in the tetrahedron, obtain tetrahedron vertexes by using pixel mapping, interpolate and calculate the lattice points at the corresponding positions of the voxels, the interpolation method adopts the fusion of volume weight interpolation and space second order polynomial interpolation, and the interpolation of each tetrahedron does not affect parallel calculation, so as to obtain three-dimensional data after reconstruction.
Drawings
Fig. 1 is a flow chart of a method of three-dimensional reconstruction of an ultrasound image according to the present invention.
Fig. 2 is a block diagram of a three-dimensional reconstruction method of an ultrasound image according to the present invention.
Fig. 3 shows a schematic view of subdivision interpolation of the three-dimensional reconstruction method of an ultrasound image according to the present invention, (a) a spatial point cloud distribution map, (b) a tetrahedral subdivision result, and (c) a schematic view of tetrahedral interpolation.
Detailed Description
As shown in fig. 1, 2 and 3, the three-dimensional reconstruction method of the ultrasonic image comprises the following steps:
(1) Spatial mapping: taking pixels in a two-dimensional image obtained from an ultrasonic acquisition system as plane point clouds with only central values, and obtaining point clouds in a corresponding space coordinate system through space mapping;
(2) Tetrahedral subdivision: dividing a space under a new coordinate system into a plurality of tetrahedrons consisting of point clouds as vertexes; fig. 3 (a) shows: the grid points represent the point cloud positions obtained after the pixels are mapped to the spatial coordinate system, and fig. 3 (b) shows: vertices are tetrahedrons of lattice points.
(3) Tetrahedral interpolation: interpolation of lattice points within tetrahedral volume is performed as shown in fig. 3 (c): and obtaining tetrahedron vertexes by using pixel mapping, carrying out interpolation calculation on lattice points (open points pointed by arrows) at corresponding positions of the voxels, wherein the interpolation method adopts fusion of volume weight interpolation and space second-order polynomial interpolation, and the interpolation of each tetrahedron does not affect parallel calculation, so as to obtain three-dimensional volume data after reconstruction.
According to the method, the purpose of self-adaptive neighborhood segmentation is achieved by carrying out tetrahedral subdivision on the space, a proper neighborhood is provided for interpolation points under different acquisition densities, and the influence of parameter imbalance caused by uneven acquisition densities is reduced; the volume weight interpolation and the spatial second-order polynomial interpolation are fused to the voxels to be interpolated in the space, so that the reconstruction error is lower, and the application range of the method is increased; the interpolation process of different voxels has independence, can be calculated in parallel, and greatly improves the reconstruction efficiency.
Preferably, in the step (1), the planar point cloud is mapped to the same spatial coordinate system by using a spatial position transformation matrix t= { T 1,…,TM } obtained by the tracking device to obtain a spatial point cloud.
Preferably, in the step (2), the split tetrahedron has uniqueness and is most regular, i.e. the smallest interior angle is largest.
Preferably, in the step (3), interpolation is performed on lattice points in the tetrahedron by adopting a method of integrating volume weight interpolation and spatial second-order polynomial interpolation, and continuous volume weight interpolation is performed when the lattice points are close to the surface of the tetrahedron; and when the lattice point is close to the center of the tetrahedron, the spatial second-order polynomial interpolation is performed, so that the fitting capability is stronger.
Preferably, in the step (3), interpolation operations of each tetrahedron are not affected, and have independence, and GPU (graphics processor, graphics Processing Unit) may be used to accelerate the operations in parallel, so as to obtain three-dimensional volume data after reconstruction.
It will be understood by those skilled in the art that all or part of the steps in implementing the above embodiment method may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, where the program when executed includes the steps of the above embodiment method, and the storage medium may be: ROM/RAM, magnetic disks, optical disks, memory cards, etc. Accordingly, the invention also includes a three-dimensional reconstruction device of ultrasound images corresponding to the method of the invention. As shown in fig. 1, the apparatus includes:
A space mapping module configured to treat pixels in a two-dimensional image obtained from an ultrasonic acquisition system as a plane point cloud with only a central value, and obtain a point cloud in a corresponding space coordinate system through space mapping;
the tetrahedron subdivision module is configured to divide the space under the new coordinate system into a plurality of tetrahedrons consisting of point clouds as vertexes;
The tetrahedron interpolation module is configured to interpolate the lattice points in the tetrahedron, obtain tetrahedron vertexes by using pixel mapping, interpolate and calculate the lattice points at the corresponding positions of the voxels, the interpolation method adopts the fusion of volume weight interpolation and space second order polynomial interpolation, and the interpolation of each tetrahedron does not affect parallel calculation, so as to obtain three-dimensional data after reconstruction.
Preferably, in the spatial mapping module, the planar point cloud is mapped to the same spatial coordinate system by using a spatial position transformation matrix t= { T 1,…,TM }, which is obtained by the tracking device, to obtain a spatial point cloud.
Preferably, in the tetrahedral subdivision module, a smallest internal angle of the subdivision tetrahedron is largest.
Preferably, in the tetrahedron interpolation module, a method of integrating volume weight interpolation and spatial second order polynomial interpolation is adopted to interpolate lattice points in the tetrahedron, continuous volume weight interpolation is performed when the lattice points are close to the surface of the tetrahedron, and spatial second order polynomial interpolation is performed when the lattice points are close to the center of the tetrahedron.
Preferably, in the tetrahedral interpolation module, the GPU is utilized to accelerate the operation in parallel, so as to obtain the three-dimensional volume data after reconstruction.
The present invention is not limited to the preferred embodiments, but can be modified in any way according to the technical principles of the present invention, and all such modifications, equivalent variations and modifications are included in the scope of the present invention.
Claims (4)
1. The three-dimensional reconstruction method of the ultrasonic image is characterized by comprising the following steps of: which comprises the following steps:
(1) Spatial mapping: taking pixels in a two-dimensional image obtained from an ultrasonic acquisition system as plane point clouds with only central values, and obtaining point clouds in a corresponding space coordinate system through space mapping;
(2) Tetrahedral subdivision: dividing a space under a new coordinate system into a plurality of tetrahedrons consisting of point clouds as vertexes;
(3) Tetrahedral interpolation: interpolation is carried out on lattice points in tetrahedrons, tetrahedron vertexes are obtained through pixel mapping, interpolation calculation is carried out on the lattice points at positions corresponding to the voxels, the interpolation method adopts fusion of volume weight interpolation and space second-order polynomial interpolation, parallel calculation is not affected by interpolation of each tetrahedron, and three-dimensional volume data after reconstruction is obtained;
in the step (1), the spatial position transformation matrix obtained by the tracking device is utilized Mapping the planar point cloud to the same space coordinate system to obtain a space point cloud;
in the step (2), the minimum internal angle of the split tetrahedron is the largest;
In the step (3), interpolation is carried out on lattice points in the tetrahedron by adopting a method of integrating volume weight interpolation and space second-order polynomial interpolation, continuous volume weight interpolation is carried out when the lattice points are close to the surface of the tetrahedron, and space second-order polynomial interpolation is carried out when the lattice points are close to the center of the tetrahedron.
2. The method for three-dimensional reconstruction of an ultrasound image according to claim 1, wherein: in the step (3), the GPU is utilized to accelerate operation in parallel, and three-dimensional volume data after reconstruction is obtained.
3. The three-dimensional reconstruction device of the ultrasonic image is characterized in that: it comprises the following steps:
A space mapping module configured to treat pixels in a two-dimensional image obtained from an ultrasonic acquisition system as a plane point cloud with only a central value, and obtain a point cloud in a corresponding space coordinate system through space mapping;
the tetrahedron subdivision module is configured to divide the space under the new coordinate system into a plurality of tetrahedrons consisting of point clouds as vertexes;
The tetrahedron interpolation module is configured to interpolate the lattice points in the tetrahedrons, the tetrahedron vertexes are obtained by utilizing pixel mapping, interpolation calculation is carried out on the lattice points at the positions corresponding to the voxels, the interpolation method adopts fusion of volume weight interpolation and space second-order polynomial interpolation, the interpolation of each tetrahedron does not affect parallel calculation, and three-dimensional data after reconstruction is obtained;
In the space mapping module, a space position transformation matrix obtained by a tracking device is utilized Mapping the planar point cloud to the same space coordinate system to obtain a space point cloud;
In the tetrahedral subdivision module, the minimum internal angle of the subdivision tetrahedron is the largest;
In the tetrahedron interpolation module, a method of integrating volume weight interpolation and space second order polynomial interpolation is adopted to interpolate lattice points in the tetrahedron, continuous volume weight interpolation is executed when the lattice points are close to the surface of the tetrahedron, and space second order polynomial interpolation is executed when the lattice points are close to the center of the tetrahedron.
4. A three-dimensional reconstruction device of ultrasound images according to claim 3, wherein: in the tetrahedron interpolation module, GPU parallel is utilized to accelerate operation, and three-dimensional volume data after reconstruction is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210681854.5A CN114998150B (en) | 2022-06-15 | 2022-06-15 | Three-dimensional reconstruction method and device for ultrasonic image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210681854.5A CN114998150B (en) | 2022-06-15 | 2022-06-15 | Three-dimensional reconstruction method and device for ultrasonic image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114998150A CN114998150A (en) | 2022-09-02 |
| CN114998150B true CN114998150B (en) | 2024-04-30 |
Family
ID=83036052
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210681854.5A Active CN114998150B (en) | 2022-06-15 | 2022-06-15 | Three-dimensional reconstruction method and device for ultrasonic image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114998150B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117414154B (en) * | 2023-09-05 | 2024-09-20 | 骨圣元化机器人(深圳)有限公司 | Three-dimensional ultrasonic reconstruction method, device and ultrasonic system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799657A (en) * | 2012-07-05 | 2012-11-28 | 上海富瀚微电子有限公司 | System and method for realizing real-time data point mapping processing based on three-dimensional checking |
| CN109636912A (en) * | 2018-11-27 | 2019-04-16 | 中国地质大学（武汉） | Tetrahedron subdivision finite element interpolation method applied to three-dimensional sonar image reconstruction |
| EP3961551A1 (en) * | 2019-04-25 | 2022-03-02 | Spreadtrum Communications (Shanghai) Co., Ltd. | Tetrahedral interpolation calculation method and apparatus, gamut conversion method and apparatus, and medium |
| CN114463480A (en) * | 2020-11-09 | 2022-05-10 | 北京理工大学 | Ultrasonic volume reconstruction method and device based on pose parameter regularization |
-
2022
- 2022-06-15 CN CN202210681854.5A patent/CN114998150B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799657A (en) * | 2012-07-05 | 2012-11-28 | 上海富瀚微电子有限公司 | System and method for realizing real-time data point mapping processing based on three-dimensional checking |
| CN109636912A (en) * | 2018-11-27 | 2019-04-16 | 中国地质大学（武汉） | Tetrahedron subdivision finite element interpolation method applied to three-dimensional sonar image reconstruction |
| EP3961551A1 (en) * | 2019-04-25 | 2022-03-02 | Spreadtrum Communications (Shanghai) Co., Ltd. | Tetrahedral interpolation calculation method and apparatus, gamut conversion method and apparatus, and medium |
| CN114463480A (en) * | 2020-11-09 | 2022-05-10 | 北京理工大学 | Ultrasonic volume reconstruction method and device based on pose parameter regularization |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114998150A (en) | 2022-09-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Beers et al. | High-resolution medical image synthesis using progressively grown generative adversarial networks | |
| US11653900B2 (en) | Data augmentation for training deep learning models with ultrasound images | |
| KR101805619B1 (en) | Apparatus and method for creating optimal 2-dimensional medical image automatically from 3-dimensional medical image | |
| CN110675489B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN109754394B (en) | Three-dimensional medical image processing device and method | |
| Huang et al. | Bezier interpolation for 3-D freehand ultrasound | |
| US10499879B2 (en) | Systems and methods for displaying intersections on ultrasound images | |
| CN102436672B (en) | Ultrasonic image processing apparatus | |
| US20210077060A1 (en) | System and methods for interventional ultrasound imaging | |
| US11854158B2 (en) | Medical image enhancement | |
| CN114037714A (en) | 3D MR and TRUS image segmentation method for prostate system puncture | |
| CN117413300A (en) | Method and system for training quantized nerve radiation field | |
| CN114998150B (en) | Three-dimensional reconstruction method and device for ultrasonic image | |
| CN103761767A (en) | Quick three-dimensional ultrasound image reconstruction method based on sparse data | |
| CN113888698A (en) | Dynamic three-dimensional reconstruction method and system for hepatectomy | |
| CN103860197A (en) | Free type three-dimensional ultrasonic imaging method, device and system | |
| CN113706684A (en) | Three-dimensional blood vessel image reconstruction method, system, medical device and storage medium | |
| US20240005498A1 (en) | Method of generating trained model, machine learning system, program, and medical image processing apparatus | |
| Gaits et al. | Ultrasound volume reconstruction from 2D Freehand acquisitions using neural implicit representations | |
| Frisken et al. | Efficient estimation of 3D euclidean distance fields from 2D range images | |
| CN115619941A (en) | Ultrasonic imaging method and ultrasonic equipment | |
| Wen et al. | GPU-based volume reconstruction for freehand 3D ultrasound imaging | |
| AU2019429940B2 (en) | AI identification method of embolism based on VRDS 4D medical image, and product | |
| CN114419173A (en) | Truncation artifact correction method and device for three-dimensional X-ray imaging system | |
| JP6443574B1 (en) | Ray casting program, search control data, search control data generation method, and ray casting apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |