CN116269496A - Heart three-dimensional ultrasonic imaging and heart function evaluation system based on implicit neural representation - Google Patents

Abstract

The invention proposes a heart three-dimensional ultrasound imaging and heart function evaluation system based on implicit neural representation. The system includes a two-dimensional ultrasound image acquisition module, a three-dimensional ultrasound imaging module based on implicit neural representation, and a cardiac function assessment module; among them, the two-dimensional ultrasound image acquisition module is used to acquire two-dimensional ultrasound images of the heart and estimate the position of the ultrasound images Parameters; the 3D ultrasound imaging module based on implicit neural representation is used for 3D reconstruction of the heart, which represents the whole heart as an implicit function whose input is 3D coordinates and output is the voxel value of the corresponding position; the cardiac function evaluation module is used for Automatically calculate the volume of the left and right ventricle of the heart and the ejection fraction. The three-dimensional heart reconstructed by the method of the present invention contains more detail information inside the cavity than the three-dimensional heart collected by the traditional three-dimensional probe.

Description

Cardiac 3D Ultrasound Imaging and Cardiac Function Assessment System Based on Implicit Neural Representation

technical field

The invention relates to the technical field of three-dimensional ultrasound imaging, in particular to a three-dimensional ultrasound imaging of the heart based on implicit neural representation and a system for evaluating cardiac function.

Background technique

In the past three decades, cardiovascular disease has affected more than 250 million people worldwide, and more than 6.5 million of them have lost their lives due to cardiovascular disease. Cardiac imaging is an important tool in the screening and diagnosis of cardiovascular disease, including echocardiography, chest x-rays, and magnetic resonance imaging. Cardiac ultrasound imaging has become the most convenient imaging method for clinical examination due to its safety, real-time, non-invasive, low cost, and easy operation. However, traditional two-dimensional ultrasound images can only provide a two-dimensional cross-sectional view of the heart, but cannot provide a clear three-dimensional heart structure. Doctors need to constantly adjust the detection angle of the ultrasound probe based on their years of clinical experience, and conduct multiple two-dimensional ultrasound images. Understand and restore its true three-dimensional structure in the brain. This process requires high clinical experience of doctors, which limits the accuracy and speed of diagnostic results and the promotion of ultrasound imaging technology. Therefore, performing three-dimensional ultrasound imaging of the heart and visualizing the complex anatomical structure and shape of the heart can overcome the limitations of two-dimensional ultrasound imaging technology and obtain better tissue contrast. With the semantic segmentation algorithm of the left and right ventricle of the heart, it can accurately calculate The ejection fraction of the left and right ventricles of the heart can be used to accurately evaluate the heart function.

In order to achieve 3D ultrasound imaging, existing imaging schemes can be divided into two categories: imaging methods based on 3D ultrasound probes, and imaging methods based on positioning sensors and 2D ultrasound probes. The spatial resolution of the three-dimensional heart collected by the former is poor, and manual calibration by experienced sonographers is required to evaluate cardiac function. The latter needs to obtain the position parameters of two-dimensional ultrasound images by calibrating complex and expensive positioning sensors (such as acoustic localizers, optical localizers, articulated arm localizers, and electromagnetic localizers, etc.), and use the method of splicing and superposition to further obtain three-dimensional structures. . This method of three-dimensional imaging based on positioning sensors and two-dimensional ultrasound probes has not been used for three-dimensional imaging of the heart and assessment of cardiac function.

Implicit neural representations can learn a continuous mapping from coordinates to signal values, and have been widely used in many fields, including image representation, multi-view synthesis, and linear inverse problem solving.

Contents of the invention

In view of the above prior art, the purpose of the present invention is to achieve high-quality three-dimensional ultrasound imaging of the heart. By introducing implicit neural representation into the problem of three-dimensional ultrasound imaging, there is no need for a large number of data sets, and only a set of two-dimensional ultrasound images can be used. Reconstruct a high-quality three-dimensional heart, and the reconstructed heart can be used for the evaluation of cardiac function, assisting doctors in better clinical diagnosis.

In order to achieve the above object, the technical scheme adopted in the present invention is:

A system for three-dimensional ultrasound imaging of the heart and cardiac function assessment based on implicit neural representation, including a two-dimensional ultrasound image acquisition module, a three-dimensional ultrasound imaging module and a cardiac function assessment module based on implicit neural representation, the two-dimensional ultrasound image acquisition module is used for Obtain a two-dimensional ultrasound image of the heart, and estimate the position parameters of the ultrasound image; the three-dimensional ultrasound imaging module based on implicit neural representation is used for three-dimensional reconstruction of the heart, and the module represents the whole heart as a three-dimensional coordinate input and output as An implicit function corresponding to the voxel value of the position; the heart function evaluation module is used to automatically calculate the volume and ejection fraction of the left and right ventricles of the heart.

Further, the two-dimensional ultrasound image acquisition module uses a two-dimensional ultrasound probe.

Further, the method includes the following steps:

Step 1, using the two-dimensional ultrasound probe to scan the heart region for several cycles, collecting two-dimensional ultrasound images of the heart from different angles of view, and estimating the position parameters of each two-dimensional ultrasound image;

Step 2, using the neural network to construct an implicit neural representation of the heart, which is used to represent the three-dimensional structure of the heart, the input is the three-dimensional coordinates of the heart space, and the output is an implicit function of the voxel value of the three-dimensional heart at the corresponding position;

Step 3, construct a loss function inspired by the physical process of two-dimensional ultrasound imaging, and use the loss function to supervise the training of neural network model parameters and ultrasound image position parameters, so as to perform three-dimensional ultrasound imaging of the heart;

Step 4: Use the implicit neural representation of the heart constructed in step 2 to generate a set of two-dimensional slices of the heart, and use the semantic segmentation algorithm to segment the left and right ventricular regions in the slices, and calculate the area of the cavity in each slice ; Then by adding up the area of the left and right ventricle in all slices, the volume of the left and right ventricle is obtained, and then the ejection fraction of the left and right ventricle is calculated.

Furthermore, in the second step, the implicit neural representation of the heart constructs a continuous mapping from 3D coordinates to 3D heart voxel values, and does not require a large data set for training.

The innovations and advantages of the present invention are:

(1) The present invention utilizes a neural network to realize three-dimensional ultrasonic imaging of the heart, which solves the physical limitations of low resolution and many artifacts of traditional three-dimensional ultrasonic probes. The present invention reconstructs a high-quality three-dimensional heart using only two-dimensional ultrasonic images collected by the most common clinical two-dimensional ultrasonic probe. Since the resolution of the two-dimensional ultrasound probe is higher than that of the traditional three-dimensional probe, the three-dimensional heart reconstructed by the present invention contains more detailed information inside the cavity than the three-dimensional heart acquired by the traditional three-dimensional probe.

(2) The present invention only uses a group of two-dimensional ultrasound images scanned around the apex of the heart to perform three-dimensional reconstruction of the heart, and does not rely on large-scale data sets for network model training. Therefore, the problem of data bias in traditional data-driven artificial intelligence algorithms is avoided. The present invention can be used as a non-data-driven artificial intelligence tool embedded in two-dimensional ultrasound machines widely used in hospitals to perform high-quality three-dimensional cardiac reconstruction for accurate, reliable and automatic cardiac function assessment in clinical diagnosis.

(3) The cardiac function evaluation module of the present invention can automatically segment the left and right ventricle in the three-dimensional heart without calibration by a doctor, and then can accurately obtain the volume and ejection fraction of the left and right ventricle of the heart, assisting the doctor to better Make a clinical diagnosis.

Description of drawings

Fig. 1 is the structural block diagram of the system of the present invention;

Fig. 2 is the flowchart of the two-dimensional ultrasonic image acquisition module in the embodiment of the present invention;

3 is an algorithm flow chart of the three-dimensional ultrasound imaging module based on implicit neural representation in an embodiment of the present invention;

Fig. 4 is the algorithm flowchart of the central function evaluation module of the embodiment of the present invention;

Fig. 5 is a comparison diagram of the three-dimensional heart reconstructed by the embodiment of the present invention and the three-dimensional heart acquired by the traditional three-dimensional probe.

Detailed ways

Embodiments of the present invention are described in detail below with reference to the accompanying drawings, examples of which are shown in the drawings, in which the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

Referring to Figures 1 and 2, the two-dimensional ultrasound image acquisition module is used to acquire two-dimensional ultrasound images of the heart and estimate the position parameters of the ultrasound images, wherein the acquisition equipment of two-dimensional ultrasound images is the most commonly used two-dimensional ultrasound probe in clinical practice , no need to use additional positioning sensors. The acquisition method is to use a two-dimensional ultrasound probe to conduct a circular scan at the apex of the heart.

Referring to Figure 3, the 3D ultrasound imaging module based on implicit neural representation is used for high-quality 3D reconstruction of the heart. The specific implementation method is as follows:

输出为对应位置体素值/>

的隐式函数F_θ，其中θ为隐式神经表示中的参数。心脏的隐式神经表示构建了一个从三维坐标到三维心脏体素值的连续映射，并且不需要大量的数据集进行训练：Firstly, an implicit neural representation of the three-dimensional heart is constructed. Unlike the traditional input of pictures, the present invention represents the entire heart as an input as three-dimensional coordinates

The output is the voxel value of the corresponding position />

The implicit function F _θ of , where θ is a parameter in the implicit neural representation. The implicit neural representation of the heart builds a continuous map from 3D coordinates to 3D cardiac voxel values and does not require large datasets for training:

然后利用所构建的隐式神经表示根据超声图像的位置参数生成相应的二维超声图像：Furthermore, the present invention models the physical process of two-dimensional ultrasound acquisition as a process of physically slicing the heart and obtaining a cross-section, and builds a loss function based on the physical process of acquisition to supervise the training of implicit neural representations. Specifically, firstly, the two-dimensional ultrasound image acquisition module is used to acquire images and estimate the position parameters of ultrasound images

The constructed implicit neural representation is then used to generate the corresponding 2D ultrasound image according to the location parameters of the ultrasound image:

Finally, the two-dimensional image I _i generated by the implicit neural representation and the real acquired two-dimensional image P _i are used to construct an error loss function to jointly optimize the network model parameters of the implicit neural representation and the position parameters of the ultrasound image:

Wherein, N is the number of two-dimensional images collected.

Referring to Figure 4, after reconstructing a high-precision three-dimensional heart, in order to obtain accurate end-diastolic and end-systolic volumes of the left and right ventricles, and further accurately calculate the ejection fraction, the present invention first utilizes the constructed implicit neural representation of the heart to generate A set of two-dimensional ultrasound slices of the heart, and then use the semantic segmentation algorithm to automatically segment the left and right ventricular regions separately, and calculate the area of the cavity in each slice. After the segmentation, the areas of the left and right ventricle in all slices were added up to obtain the volume of the left and right ventricle. Finally, the ejection fractions of the left and right ventricles were obtained according to the volume change rate of the left and right ventricles at the end of diastole and end of systole.

Referring to FIG. 5 , the three-dimensional heart reconstructed by the present invention contains more detail information inside the cavity than the three-dimensional heart acquired by the traditional three-dimensional probe. Among them, ① is the heart at the end of diastole, and ② is the heart at the end of systole.

As shown in Table 1, the accuracy rate of left ventricular ejection fraction calculation of the present invention is higher than that of Nature regular publication EchoNet method (Ouyang, D., He, B., Ghorbani, A., Yuan, N., Ebinger, J., Langlotz,C.P.,...&Zou,J.Y.(2020).Video-based AI for beat-to-beat assessment of cardiac function.Nature,580(7802),252-256.), and can also calculate the EchoNet method cannot calculate The ejection fraction of the right ventricle, here, the mean absolute error (MAE) and root mean square error (RMSE) were used to evaluate the accuracy of ejection fraction estimation.

Table 1 Quantitative comparison of the present invention and the cardiac function evaluation accuracy rate of EchoNet method

The invention introduces the implicit neural representation into the three-dimensional ultrasound imaging scheme of the heart, and solves the physical limitations of low resolution and many artifacts of the traditional three-dimensional ultrasound probe. At the same time, the present invention does not rely on large-scale data sets for network model training, and only uses a set of two-dimensional ultrasound images scanned around the apex of the heart to perform three-dimensional reconstruction of the heart. The problem of deviation. After the reconstruction is completed, the present invention can automatically segment the left and right ventricle in the three-dimensional heart without calibration by the doctor, and then can accurately obtain the volume and ejection fraction of the left and right ventricle of the heart, assisting the doctor to make a better clinical diagnosis.

Claims (4)

1. A heart three-dimensional ultrasound imaging and cardiac function assessment system based on implicit neural representation, comprising a two-dimensional ultrasound image acquisition module, a three-dimensional ultrasound imaging module and a cardiac function assessment module based on implicit neural representation, characterized in that the two-dimensional The ultrasonic image acquisition module is used to obtain the two-dimensional ultrasonic image of the heart and estimate the position parameters of the ultrasonic image; the three-dimensional ultrasonic imaging module based on implicit neural representation is used for three-dimensional reconstruction of the heart, which represents the whole heart as an input is a three-dimensional coordinate, and the output is an implicit function of the corresponding position voxel value; the heart function evaluation module is used to automatically calculate the volume and ejection fraction of the left and right ventricles of the heart. 2. The three-dimensional cardiac ultrasound imaging and cardiac function assessment system based on implicit neural representation according to claim 1, wherein the two-dimensional ultrasound image acquisition module adopts a two-dimensional ultrasound probe. 3. utilize the method for cardiac three-dimensional ultrasound imaging and cardiac function assessment system based on implicit neural representation as claimed in claim 1, it is characterized in that, the method comprises the following steps: Step 1, using the two-dimensional ultrasound probe to scan the heart region for several cycles, collecting two-dimensional ultrasound images of the heart from different angles of view, and estimating the position parameters of each two-dimensional ultrasound image; Step 2, using the neural network to construct an implicit neural representation of the heart, which is used to represent the three-dimensional structure of the heart, the input is the three-dimensional coordinates of the heart space, and the output is an implicit function of the voxel value of the three-dimensional heart at the corresponding position; Step 3. Construct a loss function inspired by the physical process of two-dimensional ultrasound imaging, and use the loss function to supervise the training of neural network model parameters and ultrasound image position parameters, so as to perform three-dimensional ultrasound imaging of the heart; Step 4: Use the implicit neural representation of the heart constructed in step 2 to generate a set of two-dimensional slices of the heart, and use the semantic segmentation algorithm to segment the left and right ventricular regions in the slices, and calculate the area of the cavity in each slice ; Then by adding up the area of the left and right ventricle in all slices, the volume of the left and right ventricle is obtained, and then the ejection fraction of the left and right ventricle is calculated. 4. The method according to claim 3, characterized in that, in the second step, the implicit neural representation of the heart constructs a continuous mapping from three-dimensional coordinates to three-dimensional heart voxel values, and does not require a large amount of data sets to perform train.

Images