CN113436109B - Ultrafast high-quality plane wave ultrasonic imaging method based on deep learning - Google Patents
Ultrafast high-quality plane wave ultrasonic imaging method based on deep learning Download PDFInfo
- Publication number
- CN113436109B CN113436109B CN202110774364.5A CN202110774364A CN113436109B CN 113436109 B CN113436109 B CN 113436109B CN 202110774364 A CN202110774364 A CN 202110774364A CN 113436109 B CN113436109 B CN 113436109B
- Authority
- CN
- China
- Prior art keywords
- data
- plane wave
- synthetic aperture
- ultrasound
- deep
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention discloses an ultrafast high-quality plane wave ultrasonic imaging method based on deep learning, belonging to the technical field of medical ultrasonic imaging. Firstly, three-dimensional channel data of synthetic aperture ultrasound at different positions of a plurality of persons are collected, and two-dimensional channel data of plane waves are generated by utilizing the three-dimensional channel data; processing the two channel data by adopting a corresponding beam forming technology to obtain ultrasonic RF data of a synthetic aperture and a plane wave in pair, and establishing a data set for deep network training; then, using plane wave RF data in the pair of data sets established in the front as the input of the network, using synthetic aperture RF data as the output label of the network, and obtaining a deep network model after training; and finally, inputting the actually acquired plane wave RF data into the trained deep network model, and outputting the estimation of the synthetic aperture RF data corresponding to the depth network model. The invention not only keeps the ultra-fast imaging speed advantage of plane wave ultrasound, but also improves the imaging quality.
Description
Technical Field
The invention relates to the technical field of medical ultrasonic imaging, in particular to an ultrafast high-quality plane wave ultrasonic imaging method based on deep learning.
Background
The different acquisition modes of the ultrasonic imaging device have respective advantages and disadvantages in the aspects of imaging speed (frame rate), imaging quality (resolution and signal-to-noise ratio) and system complexity (cost), and need to be selected and compromised according to practical application scenarios. Plane wave ultrasound transmitted by all array elements and received by all array elements is a collection mode capable of realizing ultra-fast imaging, but imaging quality is sacrificed.
The ultrasonic equipment on the market at present adopts a line-by-line scanning mode, so that the imaging quality is improved, and the imaging speed is reduced. The document [1] Z.Zhou, Y.Wang, Y.Guo, X.Jiang and Y.Qi, "ultrasonic Plane Wave Imaging With Line-Scan-Quality Using an ultrasonic-Transfer general adaptive Network," in IEEE Journal of biological and Health information, vol.24, no.4, pp.943-956, april 2020, doi. In the data set established in document [1], a plane wave ultrasound image and a line-by-line scanning ultrasound image are respectively acquired by two sets of equipment, the two images are not strictly paired, and generally, compared with a model trained by using paired samples, the performance of the method for training a depth network by using unpaired samples to perform image mapping is reduced.
In documents [2] jensen, j.a., nikolov, s.i., gammelmark, k.l., & Pedersen, m.h. (2006). Synthetic aperture ultrasound ultrasounding.ultrasonics, 44, e5-e15., synthetic aperture ultrasound transmits signals by one array element, all the array elements simultaneously receive signals, all the array elements sequentially transmit and receive all the signals, and finally, all the channel data are added by using a beam forming technology to obtain an imaging result, so that dynamic focusing of the transmitting unit and the receiving unit can be realized. Synthetic aperture ultrasound has the disadvantages of large amount of data to be transmitted and large amount of imaging calculation, resulting in slow imaging speed and low frame rate.
In documents [3] r.ali, c.d. herockhoff, d.hyun, j.j.dahl and n.bottenus, "extended retroactive Encoding for Robust Recovery of the Multistatic Data Set," in IEEE Transactions on Ultrasonics, ferroelectronics, and Frequency Control, vol.67, no.5, pp.943-956, may 2020, doi.
The method provided by the invention utilizes the synthetic aperture ultrasonic data to generate plane wave data, constructs a paired data set, trains out a depth network model, and realizes mapping of the RF data of plane wave imaging to the RF data of synthetic aperture ultrasonic imaging. The method provided by the invention improves the imaging quality by utilizing deep learning while maintaining the advantage of ultra-fast imaging speed of the plane wave ultrasound.
Disclosure of Invention
The invention aims to provide an ultrafast high-quality plane wave ultrasonic imaging method based on deep learning, which is characterized by comprising the following steps of:
step 1: constructing a pair of RF data sets; acquiring three-dimensional channel data of synthetic aperture ultrasound by using an ultrasound platform, and establishing a pair RF (radio frequency) data set of synthetic aperture-plane wave ultrasound;
and 2, step: training a deep network model; constructing a deep network and a Loss function, and training the deep network by using the paired RF data sets obtained in the step (1) to obtain a deep network model;
and 3, step 3: deploying a deep network; and (3) inputting the RF data of the plane wave ultrasound acquired in real time into the deep network model trained in the step (2), and acquiring the output of the network as the enhanced ultrasonic RF data.
The step 1 comprises the following substeps:
step 11: for each sample, the synthetic aperture ultrasonic channel data d obtained by all the transmitting array elements j (x t ,x r T) are added together to obtain two-dimensional plane wave channel data
Wherein j =1 … N, N is the number of samples, x t As coordinates of the transmitting array element, x r In order to receive the coordinates of the array elements, t is the two-way propagation time of the sound waves;
step 12: using a synthetic aperture ultrasound beamformer B 1 Treatment d j (x t ,x r T) to obtain synthetic aperture RF data o j (x,t)＝B 1 {d j (x t ,x r T), (x, t) is the spatial coordinate of the imaging point;
step 13: using plane wave ultrasonic beam former B 2 Treatment of p j (x r T) obtaining plane wave RF data i j (x,t)＝B 2 {p j (x r ,t)}；
Step 14: after all samples are processed according to the steps 11 to 13, a pair RF data set D = { i } is obtained j (x,t),o j (x, t), j =1 … N }, N being the number of samples.
The Loss function in the step 2 is as follows:
wherein, f (i) j (x, t), W) represents a deep network computation process, and W represents a network parameter.
The invention has the beneficial effects that:
the method provided by the invention not only keeps the ultra-fast imaging speed advantage of plane wave ultrasound, but also improves the imaging quality by utilizing deep learning.
Drawings
FIG. 1 is a general flow chart of the present invention;
FIG. 2 (a) is a flow chart of a paired data set portion; FIG. 2 (b) is a flow chart of the deep network training portion; FIG. 2 (c) is a flow chart of the deep network deployment section;
FIG. 3 is a deep network employed;
wherein: "k3n16s1" indicates that the convolution kernel size is 3 × 3, the number of channels is 16, and the step size is 1; "× 2" indicates that this operation was repeated twice, with the other notations being the same;
FIG. 4 is a pair of RF data in a training set; wherein (a) is 16 plane wave RF data samples; (b) corresponding synthetic aperture RF data samples;
FIG. 5 is an example of a test set experiment of the present invention; wherein, (a) is B-mode ultrasound of plane wave RF data input by the depth network; (B) B-mode ultrasound outputting RF data for the deep network; and (c) B ultrasonic of synthetic aperture ultrasound.
Detailed Description
The invention provides an ultrafast high-quality plane wave ultrasonic imaging method based on deep learning, and the invention is further explained by combining the attached drawings and specific embodiments.
FIG. 1 is a general flow chart of the present invention, and FIG. 2 (a) is a flow chart of a paired data set section; FIG. 2 (b) is a flow chart of the deep network training portion; FIG. 2 (c) is a flow diagram of a deep network deployment section; specifically, it can be expressed as follows:
1) Establishing a paired data set:
a) Collecting three-dimensional channel data of synthetic aperture ultrasound of multiple persons and multiple positions by using an ultrasound platform, and recording the three-dimensional channel data as d j (x t ,x r T), where j =1 … N, N is the number of samples, x t Being coordinates of transmitting array elements, x r To receive the coordinates of the array elements, t is the two-way travel time of the acoustic wave.
b) For each sample, the synthetic aperture ultrasonic channel data d obtained by all the transmitting array elements j (x t ,x r T) are added together to obtain two-dimensional plane wave channel data
c) Using Liu Wenkai; efficient synthetic aperture ultrasonic imaging method 2021.5.18 Chinese CN 202110539280.3' synthetic aperture ultrasonic beam former B 1 Treatment d j (x t ,x r T) obtaining RF data o j (x,t)＝B 1 {d j (x t ,x r ,t)}。
d) A Plane Wave ultrasonic beam former B in M.Albulayli and D.Rakhmatov, "Fourier Domain Depth analysis for Plane-Wave ultrasonic Imaging," in IEEE Transactions on Ultrasonics, ferroelectrics, and Frequency Control, vol.65, no.8, pp.1321-1333, aug.2018, doi 10.1109/TUFFC.2018.2837000 2 Treatment of p j (x r T) obtaining RF data i j (x,t)＝B 2 {p j (x r ,t)}。
e) After all samples have been processed, a paired data set D = { i) } is obtained j (x,t),o j (x, t), j =1 … N {, N is likeThis number.
2) Training a deep network model:
a) RF data o relative to synthetic aperture ultrasound j (x, t), RF data i of plane wave ultrasound j (x, t) is degraded by the presence of relatively severe crosstalk noise. Aiming at the image enhancement problem, a depth network and a loss function are constructed, and the adopted depth network structure of the test is shown in figure 3. As shown in FIG. 3, we use a two-dimensional U-Net model. The U-Net model comprises an encoding process and a decoding process, and jumper wires exist between the encoding process and the decoding process and are used for extracting features of different scales. In addition, other reasonable end-to-end network architectures may be substituted. The loss function of the training process is:
wherein f (-) represents a deep network computing process, W represents a network parameter, and Adam or other reasonable optimizers are adopted in a training process to optimize the network parameter.
b) And (3) training the deep network by using the paired data set D established in the step 1 to obtain a deep network model f (i (x, t), W). Fig. 4 shows sample pairs in the training set.
3) Deploying a deep network:
a) Inputting the RF data i (x, t) of the plane wave ultrasonic waves obtained in real time into the deep network model f (i (x, t), W) trained in the second step to obtain the output of the network
As enhanced ultrasound RF data.
b) By using
And obtaining the B ultrasonic for display.
Acquiring three-channel synthetic aperture ultrasonic channel data of a plurality of 12 human parts, generating channel data corresponding to transmitted plane waves by using the synthetic aperture channel data, and then respectively carrying out synthetic aperture imaging and plane wave imaging to obtain 605 matched pair samples. Wherein, 424 groups are used as training set, 60 groups are used as verification set, and 121 groups are used as test set. The size of each sample image is 3100 × 256. During training, 10000 training samples with the size of 64 multiplied by 64 are obtained by randomly cutting blocks from a training set, the batch size is set to 64, the training iteration times are 100 times, the initial learning rate is 0.0001, and optimization is carried out by adopting an Adam optimizer. The experimental platform comprises Intel (R) Core (TM) i9-9820X CPU@3.30GHz, 64GB RAM, geForce RTX 2080Ti and GeForce RTX 3090. Figure 5 shows the results of the test of the present invention on the test set. 5 (a) B ultrasonic of plane wave RF data input by a depth network; 5 (B) B ultrasonic of the RF data output by the deep network; and 5 (c) is B-mode ultrasound of synthetic aperture ultrasound (standard answer to fig. 5 (B)). As can be seen from FIG. 5, the quality of the B-mode ultrasonic image obtained by the invention is obviously better than that of a plane wave B-mode ultrasonic image. Table 1 gives the performance index for the test set.
Table 1 test set performance index comparison table
It can be seen that after the enhancement by the method, the SSIM, PSNR and SNR of the RF image and the B ultrasonic image are all improved. Table 2 shows the time for processing a frame by the deep network on different GPU cards.
TABLE 2 comparison table of deep network processing time consumption (unit: frame/second) of different GPU cards
As can be seen from Table 2, the processing speed of the raw RF data on Geforce RTX 3090 is up to 130 frames per second, and if the raw RF data is properly down-sampled, the processing speed can be further increased.
In summary, the technology of the present invention greatly improves the imaging quality by deep learning while maintaining the advantage of ultra-fast imaging speed of the plane wave ultrasound.
The present invention is not limited to the above embodiments, and any changes or substitutions that can be easily made by those skilled in the art within the technical scope of the present invention are also within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (2)
1. A plane wave ultrasonic imaging method based on deep learning is characterized by comprising the following steps:
step 1: constructing a pair of RF data sets; acquiring three-dimensional channel data of synthetic aperture ultrasound by using an ultrasound platform, and establishing a pair RF (radio frequency) data set of synthetic aperture-plane wave ultrasound;
the step 1 comprises the following substeps:
step 11: for each sample, the synthetic aperture ultrasonic channel data d obtained by all the transmitting array elements j (x t ，x r T) are added together to obtain two-dimensional plane wave channel data
Wherein j =1 … N, N is the number of samples, x t Being coordinates of transmitting array elements, x r In order to receive the coordinates of the array elements, t is the two-way propagation time of the sound waves;
step 12: using a synthetic aperture ultrasound beamformer B 1 Treatment d j (x t ，x r T) to obtain synthetic aperture RF data o j (x，t)＝B 1 {d j (x t ，x r T), (x, t) is the spatial coordinate of the imaging point;
step 13: using plane wave ultrasonic beamformer B 2 Treatment of p i (x r T) obtaining plane wave RF data i j (x，t)＝B 2 {p j (x r ，t)}；
Step 14: after all samples are processed according to the steps 11 to 13, a pair RF data set D = { i } is obtained j (x，t)，o j (x，t)J =1 … N }, N being the number of samples;
step 2: training a deep network model; constructing a deep network and a Loss function, and training the deep network by using the paired RF data sets obtained in the step (1) to obtain a deep network model;
and step 3: deploying a deep network; and (3) inputting the RF data of the plane wave ultrasound acquired in real time into the deep network model trained in the step (2), and acquiring the output of the network as the enhanced ultrasonic RF data.
2. The deep learning-based plane wave ultrasonic imaging method according to claim 1, wherein the Loss function in the step 2 is:
wherein, f (i) j (x, t), W) represents a deep network computation process, and W represents a network parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110774364.5A CN113436109B (en) | 2021-07-08 | 2021-07-08 | Ultrafast high-quality plane wave ultrasonic imaging method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110774364.5A CN113436109B (en) | 2021-07-08 | 2021-07-08 | Ultrafast high-quality plane wave ultrasonic imaging method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113436109A CN113436109A (en) | 2021-09-24 |
| CN113436109B true CN113436109B (en) | 2022-10-14 |
Family
ID=77759700
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110774364.5A Active CN113436109B (en) | 2021-07-08 | 2021-07-08 | Ultrafast high-quality plane wave ultrasonic imaging method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113436109B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780329A (en) * | 2016-12-07 | 2017-05-31 | 华中科技大学 | A kind of plane of ultrasound wave imaging method based on the conversion of anti-perspective plane |
| CN109965905A (en) * | 2019-04-11 | 2019-07-05 | 复旦大学 | A kind of radiography region detection imaging method based on deep learning |
| CN110074813A (en) * | 2019-04-26 | 2019-08-02 | 深圳大学 | A kind of ultrasonic image reconstruction method and system |
| WO2020252463A1 (en) * | 2019-06-14 | 2020-12-17 | Mayo Foundation For Medical Education And Research | Super-resolution microvessel imaging using separated subsets of ultrasound data |
| CN112528731A (en) * | 2020-10-27 | 2021-03-19 | 西安交通大学 | Plane wave beam synthesis method and system based on double-regression convolutional neural network |
| CN112771374A (en) * | 2018-10-08 | 2021-05-07 | 洛桑联邦理工学院 | Image reconstruction method based on training nonlinear mapping |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3569154A1 (en) * | 2018-05-15 | 2019-11-20 | Koninklijke Philips N.V. | Ultrasound processing unit and method, and imaging system |
-
2021
- 2021-07-08 CN CN202110774364.5A patent/CN113436109B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780329A (en) * | 2016-12-07 | 2017-05-31 | 华中科技大学 | A kind of plane of ultrasound wave imaging method based on the conversion of anti-perspective plane |
| CN112771374A (en) * | 2018-10-08 | 2021-05-07 | 洛桑联邦理工学院 | Image reconstruction method based on training nonlinear mapping |
| CN109965905A (en) * | 2019-04-11 | 2019-07-05 | 复旦大学 | A kind of radiography region detection imaging method based on deep learning |
| CN110074813A (en) * | 2019-04-26 | 2019-08-02 | 深圳大学 | A kind of ultrasonic image reconstruction method and system |
| WO2020252463A1 (en) * | 2019-06-14 | 2020-12-17 | Mayo Foundation For Medical Education And Research | Super-resolution microvessel imaging using separated subsets of ultrasound data |
| CN112528731A (en) * | 2020-10-27 | 2021-03-19 | 西安交通大学 | Plane wave beam synthesis method and system based on double-regression convolutional neural network |
Non-Patent Citations (4)
| Title |
|---|
| Accelerated plane-wave destruction;Zhonghuan Chen等;《 Geophysics》;20131231;第1-16页 * |
| Image Quality Enhancement Using a Deep Neural Network for Plane Wave Medical Ultrasound Imaging;Yanxing Qi等;《IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL》;20210430;第926-934页 * |
| Ultrasound image reconstruction from plane wave radio-frequency data by self-supervised deep neural network;Jingke Zhang等;《Medical Image Analysis》;20210225;第1-18页 * |
| 超声平面波经颅成像相位校正方法;宋亚龙等;《应用声学》;20210131;第1-10页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113436109A (en) | 2021-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3187113B1 (en) | Method and device for ultrasonic imaging by synthetic focusing | |
| Zhou et al. | High spatial–temporal resolution reconstruction of plane-wave ultrasound images with a multichannel multiscale convolutional neural network | |
| US11846608B2 (en) | Image reconstruction method based on a trained non-linear mapping | |
| US10928514B2 (en) | Ultrasound method and system for extracting signal components relating to spatial locations in a target region in the spatial and temporal frequency domain | |
| CN104777484B (en) | The plane wave ultrasonic imaging of compression adaptive beam synthesis and the method and system of microvesicle imaging | |
| Perdios et al. | CNN-based image reconstruction method for ultrafast ultrasound imaging | |
| CN102727256B (en) | Dual focusing beam forming method and device based on virtual array elements | |
| Besson et al. | Ultrafast ultrasound imaging as an inverse problem: Matrix-free sparse image reconstruction | |
| Burshtein et al. | Sub-Nyquist sampling and Fourier domain beamforming in volumetric ultrasound imaging | |
| EP3232937A1 (en) | Ultrasound system for high-speed and high resolution imaging applications | |
| Goudarzi et al. | Fast multi-focus ultrasound image recovery using generative adversarial networks | |
| CN104546003B (en) | VFR (variable frame rate) color ultra-high speed ultrasonic imaging method based on plane wave transmission | |
| Schretter et al. | Ultrasound imaging from sparse RF samples using system point spread functions | |
| CN113994367A (en) | Method and system for generating synthetic elastographic images | |
| Chen et al. | ApodNet: Learning for high frame rate synthetic transmit aperture ultrasound imaging | |
| Goudarzi et al. | Ultrasound beamforming using mobilenetv2 | |
| CN112041699A (en) | Reconstruction system and method | |
| CN111248858A (en) | Photoacoustic tomography reconstruction method based on frequency domain wavenumber domain | |
| CN107137111A (en) | A kind of Ultrasound beamforming method | |
| CN113436109B (en) | Ultrafast high-quality plane wave ultrasonic imaging method based on deep learning | |
| US20180284249A1 (en) | Ultrasound imaging system and method for representing rf signals therein | |
| Liu et al. | A multiscale residual U-net architecture for super-resolution ultrasonic phased array imaging from full matrix capture data | |
| CN108784736A (en) | A kind of ultrasonic elastograph imaging strain method of estimation of two-dimensional iteration | |
| Toffali et al. | Improving the quality of monostatic synthetic-aperture ultrasound imaging through deep-learning-based beamforming | |
| Xiao et al. | Channel count reduction for plane wave ultrasound through convolutional neural network interpolation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |