CN109528196B - Hepatic vein pressure gradient non-invasive evaluation method - Google Patents
Hepatic vein pressure gradient non-invasive evaluation method Download PDFInfo
- Publication number
- CN109528196B CN109528196B CN201811353945.6A CN201811353945A CN109528196B CN 109528196 B CN109528196 B CN 109528196B CN 201811353945 A CN201811353945 A CN 201811353945A CN 109528196 B CN109528196 B CN 109528196B
- Authority
- CN
- China
- Prior art keywords
- hvpg
- modal
- neural network
- parameters
- estimated value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 11
- 210000002989 hepatic vein Anatomy 0.000 title claims description 12
- 238000012549 training Methods 0.000 claims abstract description 29
- 238000013528 artificial neural network Methods 0.000 claims abstract description 17
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 15
- 238000005457 optimization Methods 0.000 claims abstract description 10
- 210000004185 liver Anatomy 0.000 claims abstract description 9
- 238000000611 regression analysis Methods 0.000 claims abstract description 7
- 238000000034 method Methods 0.000 claims description 47
- 230000006870 function Effects 0.000 claims description 17
- 238000003384 imaging method Methods 0.000 claims description 15
- 230000017531 blood circulation Effects 0.000 claims description 13
- 230000000004 hemodynamic effect Effects 0.000 claims description 10
- 230000002440 hepatic effect Effects 0.000 claims description 9
- 210000000952 spleen Anatomy 0.000 claims description 9
- 210000003240 portal vein Anatomy 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 230000001575 pathological effect Effects 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 210000002569 neuron Anatomy 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 210000001367 artery Anatomy 0.000 claims description 2
- 238000002091 elastography Methods 0.000 claims description 2
- 210000002767 hepatic artery Anatomy 0.000 claims description 2
- 238000003062 neural network model Methods 0.000 claims description 2
- 238000011176 pooling Methods 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 claims description 2
- 230000000295 complement effect Effects 0.000 abstract 1
- 239000013589 supplement Substances 0.000 abstract 1
- 230000004927 fusion Effects 0.000 description 7
- 238000003745 diagnosis Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 208000007232 portal hypertension Diseases 0.000 description 4
- 230000000740 bleeding effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 206010073071 hepatocellular carcinoma Diseases 0.000 description 3
- 231100000844 hepatocellular carcinoma Toxicity 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002595 magnetic resonance imaging Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 206010003445 Ascites Diseases 0.000 description 1
- 208000000624 Esophageal and Gastric Varices Diseases 0.000 description 1
- 208000032843 Hemorrhage Diseases 0.000 description 1
- 206010019663 Hepatic failure Diseases 0.000 description 1
- 206010041660 Splenomegaly Diseases 0.000 description 1
- 238000003556 assay Methods 0.000 description 1
- 208000034158 bleeding Diseases 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 208000019425 cirrhosis of liver Diseases 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 208000019423 liver disease Diseases 0.000 description 1
- 208000007903 liver failure Diseases 0.000 description 1
- 231100000835 liver failure Toxicity 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008506 pathogenesis Effects 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000011158 quantitative evaluation Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000002271 resection Methods 0.000 description 1
- 210000002563 splenic artery Anatomy 0.000 description 1
- 230000003393 splenic effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000008320 venous blood flow Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
- A61B6/03—Computed tomography [CT]
- A61B6/032—Transmission computed tomography [CT]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
- A61B6/504—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of blood vessels, e.g. by angiography
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
- A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
- A61B6/5229—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image
- A61B6/5247—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
- A61B8/5238—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image
- A61B8/5261—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image combining images from different diagnostic modalities, e.g. ultrasound and X-ray
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Surgery (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Radiology & Medical Imaging (AREA)
- High Energy & Nuclear Physics (AREA)
- Optics & Photonics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Fuzzy Systems (AREA)
- Physiology (AREA)
- Mathematical Physics (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Pulmonology (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Vascular Medicine (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention discloses a liver venous pressure gradient non-invasive evaluation method based on multi-modal images and experience knowledge, which comprises the steps of utilizing a convolutional neural network to extract the characteristics of multi-modal medical images and obtain an HVPG estimated value based on the multi-modal images, utilizing a deep neural network to carry out regression analysis on HVPG related experience knowledge parameters and obtain the HVPG estimated value based on the experience knowledge, fusing the HVPG estimated values based on the multi-modal images and the experience knowledge obtained by the steps to obtain a fused HVPG estimated value, carrying out combined training on the convolutional neural network and the deep neural network by adopting an optimization algorithm, and carrying out accurate prediction on HVPG after the training is finished so as to obtain the HVPG quantitative estimated value based on the multi-modal images and the experience knowledge. The invention gives consideration to the complementary information of the medical images in various modes, and utilizes corresponding experience knowledge to supplement the characteristics, thereby being more in line with the medical pertinence.
Description
Technical Field
The invention relates to the technical field of medical images, in particular to a hepatic vein pressure gradient non-invasive assessment method based on multi-modal images and empirical knowledge.
Background
Portal hypertension is one of the most common serious complications of liver cirrhosis, and is clinically manifested by esophageal and gastric varices, rupture bleeding, ascites, splenomegaly and splenic hyperfunction and the like. With increasing portal vein pressure, the risk of esophageal fundal variceal bleeding increases, and the incidence of Hepatocellular Carcinoma (HCC), the incidence of liver failure after HCC resection, and the associated risk of death also increases. Therefore, accurate quantification and dynamic monitoring of portal hypertension level are of great importance for the research of the pathogenesis, diagnosis and treatment of portal hypertension.
The currently accepted "gold standard" for assessing portal pressure is the Hepatic Venous Pressure Gradient (HVPG), which is a technique that involves inserting a wedge catheter or balloon into the hepatic vein by a puncture method and measuring the hepatic venous wedging pressure and the hepatic venous free pressure, the difference between the two being the HVPG. However, HVPG assay is an invasive test with high technical requirements, certain technical difficulties and bleeding risks, and high cost, so there is a great need for a non-invasive and accurate portal pressure assessment method in clinical practice.
In the conventional imaging method, various imaging techniques including ultrasound, CT, magnetic resonance imaging, etc. have been used in the study of portal pressure gradient assessment, but the conventional imaging techniques have the following problems:
1) because the secondary changes of the liver and spleen shape, hardness, hemodynamics and the like during portal hypertension are very complex, the traditional imaging method is mostly qualitative diagnosis and is greatly influenced by subjective experience, and more accurate quantitative evaluation is lacked;
2) images of different modalities usually reflect different pathological features, and a traditional imaging method is mostly researched from the perspective of a single-modality image technology, so that multi-dimensional comprehensive evaluation cannot be achieved.
Deep learning (Deep learning) can fit complex data and mine features with extremely high generalization ability in the complex data through a data-driven method due to a special Deep neural network structure. On one hand, an effective data set is established by clinical acquisition and precise standard calibration of a case, and a depth model is used for learning and modeling a quantitative estimation relation between multi-modal image data and HVPG; on the other hand, clinical experience is used as experience knowledge, a neural network calculation model for estimating HVPG (high pressure gradient prediction) by significant experience parameters in multi-modal image data is established, and the HVPG quantitative estimation integrating data learning and experience knowledge is realized, so that the problem of portal vein pressure evaluation is solved.
Disclosure of Invention
The invention aims to provide a hepatic vein pressure gradient assessment method based on multi-modal images and empirical knowledge, which utilizes deep learning to model multi-modal image data and HVPG, and is assisted by empirical parameters such as morphology, functions, hemodynamics and the like in the images to realize quantitative estimation of HVPG value.
In order to achieve the purpose, the invention adopts the following technical scheme: firstly, establishing an associated representation of effective high-dimensional multi-modal image data; then, modeling a data-driven HVPG quantitative estimation regression model by utilizing multi-modal association expression to obtain a data-driven HVPG quantitative estimation value; meanwhile, by fully utilizing qualitative estimation experience of HVPG in clinic such as form, function, hemodynamics and the like as experience knowledge, an HVPG quantitative estimation relation model based on the experience knowledge is established; and finally, establishing an HVPG quantitative estimation fusion calculation model based on data drive and experience knowledge by adopting a fusion layer decision strategy, so as to realize more accurate HVPG estimation.
A hepatic vein pressure gradient non-invasive assessment method based on multi-modal images and empirical knowledge comprises the following steps:
step 1.1, acquiring medical image sequences of three Modes of Resonance Elastography (MRE), multi-phase dynamic enhanced magnetic resonance portal imaging (DCE-MRPV) and multi-flip-angle flat scanning (T1 mapping), and splicing the three medical image sequences after processing to obtain a multi-mode medical image;
step 1.2, performing feature extraction on the multi-modal medical image by using a convolutional neural network, and obtaining an HVPG estimated value H based on the multi-modal image1；
Step 2, carrying out regression analysis on the HVPG related experience knowledge parameters by utilizing the deep neural network to obtain an HVPG estimated value H based on experience knowledge2The method specifically comprises the following steps:
step 2.1, collecting magnetic resonance form, function and hemodynamic parameters related to HVPG and splicing the parameters into HVPG experience knowledge parameters;
step 2.2, carrying out regression analysis on the HVPG experience knowledge parameters by utilizing the deep neural network to obtain an HVPG estimated value H based on experience knowledge2；
Step 3, aiming at the HVPG estimated value H based on the multi-modal image and the experience knowledge obtained in the step1And H2And fusing to obtain a fused HVPG estimated value H', wherein a specific calculation formula is as follows: h ═ aH1+(1-α)H2Wherein alpha is used as a fused weight for balancing the importance of the multi-modal image and the experience knowledge to the estimation of the HVPG, and the value range is more than 0 and less than 1;
step 4, performing combined training on the convolutional neural network and the deep neural network related to the steps 1 and 2 by adopting an optimization algorithm;
and 5, after the training is finished, accurately predicting the HVPG to obtain the HVPG quantitative estimation value based on the multi-modal image and experience knowledge.
The method for preprocessing and splicing the three medical image sequences in the step 1.1 comprises the following steps: normalizing the sizes of the image sequences of the three modes by an interpolation method, wherein the sizes of the image sequences of the three modes after the normalization processing are M multiplied by N multiplied by K, and M, N, K respectively refer to the length, the width and the height of an image, then respectively calculating the average values of the image sequences of the three modes on a channel dimension, and the sizes of the image sequences are changed into M multiplied by N multiplied by 1, and then splicing the image sequences on the channel dimension, so that the multi-mode medical image with the size of M multiplied by N multiplied by 3 is obtained.
The convolutional neural network adopted in step 1.2 is a 6-layer neural network model, and comprises two convolutional layers, two pooling layers and two full-connected layers, and the HVPG estimated value based on the multi-modal image is obtained by calculating the average value of the output of the last full-connected layer, or AlexNet and ResNet are adopted.
The empirical knowledge parameters related to the HVPG and the splicing method selected in the step 2.1 are as follows: the magnetic resonance morphological parameters include: the upper and lower maximum diameters of the liver S1, the three-dimensional maximum diameter of the spleen S2, the diameter of the portal vein vessel S3 and the diameter of the hepatic vein vessel S4; acquiring magnetic resonance functional parameters includes: liver elasticity value F1, spleen elasticity value F2, liver parenchyma T1 value F3, spleen parenchyma T1 value F4; collecting hemodynamic parameters includes: portal vein blood flow velocity V1, hepatic vein blood flow velocity V2, hepatic artery blood flow velocity V3, and splenic artery blood flow velocity V4. Then the above-mentioned form, function and blood flow dynamic parameter are constructed into matrix
As an HVPG empirical knowledge parameter.
The deep neural network used in step 2.2 is a 5-layer network, and includes an input layer, an output layer, and three hidden layers, each hidden layer including five neurons.
The optimization algorithm in step 4 adopts the exponential decay rate alpha of the first moment estimation10.9, exponential decay Rate α of second moment estimation20.999 and initial learning rate e 0.09, the optimization objective is to adjust the trainable parameters in the network to minimize the loss function.
The training method adopted in the step 4 is network joint training: specifically, an alternate training method is adopted, namely, the parameters of one network are kept unchanged, only the other network is trained, and vice versa; the training data are magnetic resonance images and pathological data, wherein 100 samples are trained, 50 samples are verified, and the training label is an HVPG true value H obtained by a puncture method; the Loss function is the least square method based Loss of square (H-H')2。
Drawings
FIG. 1 is a logic block diagram of the present invention;
FIG. 2 is a flow chart of a method of the present invention;
FIG. 3 is a schematic diagram of an HVPG estimation model based on multi-modal imagery;
FIG. 4 is a schematic diagram of an HVPG estimation model based on empirical knowledge;
advantageous effects
The invention can obtain the accurately quantized noninvasive HVPG estimation value by adopting a deep learning method for medical images of different modes to carry out multidimensional learning and assisting the empirical knowledge diagnosis information of doctors. Compared with the traditional clinical puncture diagnosis method, the method can be completely noninvasive, and compared with the traditional imaging diagnosis method, the method eliminates the influence of subjective factors and considers the high latitude relation among the multi-modal medical images, thereby ensuring that the obtained result is more accurate.
Detailed Description
The present invention will be described in further detail below with reference to specific embodiments and with reference to the attached drawings.
The flow chart of the method of the invention is shown in fig. 2, and specifically comprises the following steps:
step 1.1, acquiring medical image sequences of three modes, namely resonance elastic imaging (MRE), multi-phase dynamic enhanced magnetic resonance portal imaging (DCE-MRPV) and multi-flip angle flat scanning (T1 mapping), processing the three medical image sequences and splicing to obtain a multi-mode medical image;
and acquiring medical image sequences of three modes of MRE, DCE-MRPV and T1 mapping through magnetic resonance imaging examination, wherein the three image sequences are three-dimensional stereo data. Since the different examination apparatuses may be different, the sizes of the three-mode image sequences need to be normalized, and the sizes of the three-mode image sequences are M × N × K (M, N, K is the length, width and height of the image respectively) after the normalization process. Then, the image sequences of the three modes are respectively averaged in the channel dimension (the size is changed into M multiplied by N multiplied by 1), and then the multi-mode medical images with the size of M multiplied by N multiplied by 3 are obtained by splicing in the channel dimension.
Step 1.2, performing feature extraction on the multi-modal medical image by using a convolutional neural network, and obtaining an HVPG estimated value H based on the multi-modal image1；
The invention adopts the convolutional neural network shown in figure 3 to extract the characteristics of the multi-modal medical image, and calculates the average value of the output of the last full-connected layer to obtain the HVPG estimated value based on the multi-modal image. In the network implementation, all convolution kernels are initialized by random values output from normal distribution with a truncated mean value of 0 and a standard deviation of 0.01; using a rectifier linear unit (ReLU) as a nonlinear activation function of the convolutional layer and the fully-connected layer except the last layer; using a Dropout method with a ratio of 0.5 for the fully connected layers to prevent the network from overfitting;
step 2, carrying out regression analysis on the HVPG related experience knowledge parameters by utilizing a Deep Neural Network (Deep Neural Network) to obtain an HVPG estimated value H based on experience knowledge2The method specifically comprises the following steps:
step 2.1, collecting magnetic resonance form, function and hemodynamic parameters related to HVPG and splicing the parameters into HVPG experience knowledge parameters;
acquiring magnetic resonance morphological parameters includes: maximum diameter S of liver1Spleen three-dimensional direction maximum diameter S2Diameter S of portal vein vessel3Diameter of hepatic vein vessel S4(ii) a Acquiring magnetic resonance functional parameters includes: liver elasticity value F1Spleen elasticity F2Liver parenchyma T1 value F3Spleen parenchyma T1 value F4(ii) a Acquiring hemodynamic parameters includes: portal vein blood flow velocity V1Hepatic venous blood flow velocity V2Hepatic artery blood flow velocity V3Blood flow velocity V of the spleen artery4. Then the above-mentioned form, function and blood flow dynamic parameter are constructed into matrix
As an HVPG empirical knowledge parameter.
In the step 2.2, the step of the method,carrying out regression analysis on HVPG experience knowledge parameters by utilizing a deep neural network to obtain an HVPG estimated value H based on experience knowledge2；
The present invention employs a 5-layer deep neural network as shown in fig. 4, which includes an input layer, an output layer, and three hidden layers (each hidden layer includes five neurons), where the HVPG empirical knowledge parameters are used as the input of the network, and the output layer of the network outputs the HVPG estimated value based on the empirical knowledge. In the network implementation, the initialization of the network parameters adopts random initialization and uses the linear unit of the rectifier as the nonlinear activation function of the hidden layer.
Step 3, aiming at the HVPG estimated value H based on the multi-modal image and the experience knowledge obtained in the step1And H2And fusing to obtain a fused HVPG estimated value H', wherein a specific calculation formula is as follows: h ═ aH1+(1-α)H2Wherein alpha is taken as the weight of fusion, and the value range is (alpha is more than 0 and less than 1);
in order to fully utilize the complementarity of the multi-modal image data and the empirical knowledge data in the HVPG estimation, the invention adopts the HVPG estimation value H based on the multi-modal image1And the HVPG estimated value H based on the experience knowledge2Further fusion was performed to obtain a more accurate HVPG estimate H', as shown below:
H′＝αH1+(1-α)H2
where α (0 < α < 1) is used as a weight for fusion to weigh the importance of multimodal imagery and empirical knowledge to estimate HVPG.
Step 4, performing combined training on the convolutional neural network and the deep neural network related to the steps 1 and 2 by adopting a mainstream optimization algorithm;
the network joint training specifically adopts an alternate training method, namely, the parameters of one network are kept unchanged, only the other network is trained, and vice versa; the training data are magnetic resonance image and pathological data of 150 chronic liver disease patients, that is, each sample comprises the medical image sequence of three modalities described in step 1.1, and the magnetic resonance morphology, function and hemodynamic parameters related to HVPG described in step 2.1, and is spliced into HVPG empirical knowledgeParameters and HVPG real values H of the patient obtained by a puncture method as training labels, wherein 150 samples are divided into 100 training samples for training a network and 50 verification samples for verifying the network training effect; the Loss function adopts a least square method-based square Loss (H-H')2(ii) a Exponential decay rate alpha estimated by first moment is adopted in optimization algorithm10.9, exponential decay Rate α of second moment estimation2Adaptive motion estimation (Adam) with initial learning rate e of 0.09 at 0.999, the optimization goal is to adjust the trainable parameters in the network to minimize the loss function.
Step 5, after the training is finished, the HVPG can be accurately predicted to obtain the HVPG quantitative estimation value based on the multi-modal image and experience knowledge
The performance of the network was verified using 50 validation samples and 10 experiments were repeated, with the mean and standard deviation calculated for the 10 experimental results as shown in the table:
method1 shows the result of training and verifying the multi-modal image data of 150 patients by using the network of step 1; method2 is the result of training and verifying HPVG-related empirical knowledge parameter data of 150 patients using the network of step 2 only; method1+2 is the result of the combined training and verification of the fusion network finally proposed by the present invention on the multi-modal images and empirical knowledge parameter data of 150 patients, wherein the fusion weight is 0.8, i.e. more emphasizes the estimation degree of the HVPG value by the multi-modal images.
Thus, the present invention has been described.
Claims (7)
1. A hepatic vein pressure gradient non-invasive assessment method based on multi-modal images and empirical knowledge comprises the following steps:
step 1, performing feature extraction on a multi-modal medical image by using a convolutional neural network to obtain an HVPG estimated value H based on the multi-modal image1The method specifically comprises the following steps:
step 1.1, acquiring medical image sequences of three Modes of Resonance Elastography (MRE), multi-phase dynamic enhanced magnetic resonance portal imaging (DCE-MRPV) and multi-flip-angle flat scanning (T1 mapping), and splicing the three medical image sequences after processing to obtain a multi-mode medical image;
step 1.2, performing feature extraction on the multi-modal medical image by using a convolutional neural network, and obtaining an HVPG estimated value H based on the multi-modal image1；
Step 2, carrying out regression analysis on the HVPG related experience knowledge parameters by utilizing the deep neural network to obtain an HVPG estimated value H based on experience knowledge2The method specifically comprises the following steps:
step 2.1, collecting magnetic resonance form, function and hemodynamic parameters related to HVPG and splicing the parameters into HVPG experience knowledge parameters;
step 2.2, carrying out regression analysis on the HVPG experience knowledge parameters by utilizing the deep neural network to obtain an HVPG estimated value H based on experience knowledge2；
Step 3, aiming at the HVPG estimated value H based on the multi-modal image and the experience knowledge obtained in the step1And H2And fusing to obtain a fused HVPG estimated value H', wherein a specific calculation formula is as follows: h ═ aH1+(1-α)H2Wherein alpha is used as a fused weight for balancing the importance of the multi-modal image and the experience knowledge to the estimation of the HVPG, and the value range is more than 0 and less than 1;
performing combined training on the convolutional neural network and the deep neural network related to the steps 1 and 2 by adopting an optimization algorithm;
after the training is finished, the HVPG can be accurately predicted to obtain a HVPG quantitative estimation value based on multi-modal images and experience knowledge.
2. The method for noninvasive evaluation of hepatic venous pressure gradient based on multi-modal imaging and empirical knowledge according to claim 1, characterized in that:
the method for preprocessing and splicing the three medical image sequences in the step 1.1 comprises the following steps: normalizing the sizes of the image sequences of the three modes by an interpolation method, wherein the sizes of the image sequences of the three modes after the normalization processing are M multiplied by N multiplied by K, and M, N, K respectively refer to the length, the width and the height of an image, then respectively calculating the average values of the image sequences of the three modes on a channel dimension, and the sizes of the image sequences are changed into M multiplied by N multiplied by 1, and then splicing the image sequences on the channel dimension, so that the multi-mode medical image with the size of M multiplied by N multiplied by 3 is obtained.
3. The method for noninvasive evaluation of hepatic venous pressure gradient based on multi-modal imaging and empirical knowledge according to claim 1, characterized in that:
the convolutional neural network adopted in step 1.2 is a 6-layer neural network model, and comprises two convolutional layers, two pooling layers and two full-connected layers, and the HVPG estimated value based on the multi-modal image is obtained by calculating the average value of the output of the last full-connected layer, or AlexNet and ResNet are adopted.
4. The method for noninvasive evaluation of hepatic venous pressure gradient based on multi-modal imaging and empirical knowledge according to claim 1, characterized in that:
the empirical knowledge parameters related to the HVPG and the splicing method selected in the step 2.1 are as follows: the magnetic resonance morphological parameters include: the upper and lower maximum diameters of the liver S1, the three-dimensional maximum diameter of the spleen S2, the diameter of the portal vein vessel S3 and the diameter of the hepatic vein vessel S4; acquiring magnetic resonance functional parameters includes: liver elasticity value F1, spleen elasticity value F2, liver parenchyma T1 value F3, spleen parenchyma T1 value F4; collecting hemodynamic parameters includes: portal vein blood flow velocity V1, hepatic vein blood flow velocity V2, hepatic artery blood flow velocity V3, spleen artery blood flow velocity V4; then the above-mentioned form, function and blood flow dynamic parameter are constructed into matrix
As an HVPG empirical knowledge parameter.
5. The method for noninvasive evaluation of hepatic venous pressure gradient based on multi-modal imaging and empirical knowledge according to claim 1, characterized in that:
the deep neural network used in step 2.2 is a 5-layer network, and includes an input layer, an output layer, and three hidden layers, each hidden layer including five neurons.
6. The method for noninvasive evaluation of hepatic venous pressure gradient based on multi-modal images and empirical knowledge according to claim 1, characterized in that:
the optimization algorithm adopts the exponential decay rate alpha of first moment estimation10.9, exponential decay Rate α of second moment estimation20.999 and initial learning rate e 0.09, the optimization objective is to adjust the trainable parameters in the network to minimize the loss function.
7. The method for noninvasive evaluation of hepatic venous pressure gradient based on multi-modal imaging and empirical knowledge according to claim 1, characterized in that:
the training method is network joint training: specifically, an alternate training method is adopted, namely, the parameters of one network are kept unchanged, only the other network is trained, and vice versa; the training data are magnetic resonance images and pathological data and are divided into training samples and verification samples, and the training labels are HVPG true values H obtained by a puncture method; the Loss function is the least square method based Loss of square (H-H')2。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811353945.6A CN109528196B (en) | 2018-11-14 | 2018-11-14 | Hepatic vein pressure gradient non-invasive evaluation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811353945.6A CN109528196B (en) | 2018-11-14 | 2018-11-14 | Hepatic vein pressure gradient non-invasive evaluation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109528196A CN109528196A (en) | 2019-03-29 |
| CN109528196B true CN109528196B (en) | 2022-07-01 |
Family
ID=65847272
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811353945.6A Active CN109528196B (en) | 2018-11-14 | 2018-11-14 | Hepatic vein pressure gradient non-invasive evaluation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109528196B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110298383B (en) * | 2019-05-28 | 2021-07-13 | 中国科学院计算技术研究所 | Multi-mode deep learning-based pathology classification method and system |
| CN110074809B (en) * | 2019-06-17 | 2023-06-20 | 祁小龙 | Hepatic vein pressure gradient classification method of CT image and computer equipment |
| CN112216379A (en) * | 2019-07-12 | 2021-01-12 | 刘璐 | Disease diagnosis system based on intelligent joint learning |
| CN110895817B (en) * | 2019-11-01 | 2023-06-30 | 复旦大学 | MRI (magnetic resonance imaging) image liver fibrosis automatic grading method based on image histology analysis |
| CN111598864B (en) * | 2020-05-14 | 2023-07-25 | 北京工业大学 | Liver cell cancer differentiation evaluation method based on multi-modal image contribution fusion |
| CN114373095A (en) * | 2021-12-09 | 2022-04-19 | 山东师范大学 | Alzheimer disease classification system and method based on image information |
| CN114678105B (en) * | 2022-03-21 | 2023-10-17 | 南京圣德医疗科技有限公司 | Method for automatically calculating balloon parameters by combining artificial intelligence technology |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6135966A (en) * | 1998-05-01 | 2000-10-24 | Ko; Gary Kam-Yuen | Method and apparatus for non-invasive diagnosis of cardiovascular and related disorders |
| WO2012011872A1 (en) * | 2010-07-23 | 2012-01-26 | National Cancer Centre Singapore | A method and/or system for determining portal hemodynamics of a subject |
| CN106037710A (en) * | 2014-11-24 | 2016-10-26 | 西门子公司 | Synthetic data-driven hemodynamic determination in medical imaging |
| CN106456078A (en) * | 2013-10-17 | 2017-02-22 | 西门子保健有限责任公司 | Method and system for machine learning based assessment of fractional flow reserve |
| CN107480675A (en) * | 2017-07-28 | 2017-12-15 | 祁小龙 | A kind of construction method of the HVPG computation model based on radiation group |
| CN108109140A (en) * | 2017-12-18 | 2018-06-01 | 复旦大学 | Low Grade Gliomas citric dehydrogenase non-destructive prediction method and system based on deep learning |
| CN108095716A (en) * | 2017-11-21 | 2018-06-01 | 郑州鼎创智能科技有限公司 | A kind of electrocardiograph signal detection method based on confidence rule base and deep neural network |
| CN108376558A (en) * | 2018-01-24 | 2018-08-07 | 复旦大学 | A kind of multi-modal nuclear magnetic resonance image Case report no automatic generation method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140200824A1 (en) * | 2008-09-19 | 2014-07-17 | University Of Pittsburgh Of The Commonwealth System Of Higher Education | K-partite graph based formalism for characterization of complex phenotypes in clinical data analyses and disease outcome prognosis |
| WO2012106729A1 (en) * | 2011-02-04 | 2012-08-09 | Phase Space Systems Corporation | System and method for evaluating an electrophysiological signal |
| GB201406304D0 (en) * | 2014-04-08 | 2014-05-21 | Isis Innovation | Medical imaging |
| US10729337B2 (en) * | 2015-05-05 | 2020-08-04 | The Johns Hopkins University | Device and method for non-invasive left ventricular end diastolic pressure (LVEDP) measurement |
| US10610302B2 (en) * | 2016-09-20 | 2020-04-07 | Siemens Healthcare Gmbh | Liver disease assessment in medical imaging |
-
2018
- 2018-11-14 CN CN201811353945.6A patent/CN109528196B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6135966A (en) * | 1998-05-01 | 2000-10-24 | Ko; Gary Kam-Yuen | Method and apparatus for non-invasive diagnosis of cardiovascular and related disorders |
| WO2012011872A1 (en) * | 2010-07-23 | 2012-01-26 | National Cancer Centre Singapore | A method and/or system for determining portal hemodynamics of a subject |
| CN106456078A (en) * | 2013-10-17 | 2017-02-22 | 西门子保健有限责任公司 | Method and system for machine learning based assessment of fractional flow reserve |
| CN106037710A (en) * | 2014-11-24 | 2016-10-26 | 西门子公司 | Synthetic data-driven hemodynamic determination in medical imaging |
| CN107480675A (en) * | 2017-07-28 | 2017-12-15 | 祁小龙 | A kind of construction method of the HVPG computation model based on radiation group |
| CN108095716A (en) * | 2017-11-21 | 2018-06-01 | 郑州鼎创智能科技有限公司 | A kind of electrocardiograph signal detection method based on confidence rule base and deep neural network |
| CN108109140A (en) * | 2017-12-18 | 2018-06-01 | 复旦大学 | Low Grade Gliomas citric dehydrogenase non-destructive prediction method and system based on deep learning |
| CN108376558A (en) * | 2018-01-24 | 2018-08-07 | 复旦大学 | A kind of multi-modal nuclear magnetic resonance image Case report no automatic generation method |
Non-Patent Citations (2)
| Title |
|---|
| Serum tests, liver stiffness and artificial neural networks for diagnosing cirrhosis and portal hypertension;Procopet Bogdan et al;《Digestive and Liver Disease》;20150207;第47卷(第5期);全文 * |
| 计算机辅助软件及影像学在肝纤维化诊断中的应用价值;龙莉玲;《中华医学会第十八次全国放射学学术会议论文汇编》;20111231;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109528196A (en) | 2019-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109528196B (en) | Hepatic vein pressure gradient non-invasive evaluation method | |
| US11887305B1 (en) | Systems and methods for identifying personalized vascular implants from patient-specific anatomic data | |
| CN110197493B (en) | Fundus image blood vessel segmentation method | |
| US10249048B1 (en) | Method and system for predicting blood flow features based on medical images | |
| Li et al. | Vessels as 4-D curves: Global minimal 4-D paths to extract 3-D tubular surfaces and centerlines | |
| CN106456078B (en) | Method and system for the assessment based on machine learning to blood flow reserve score | |
| CN110866914B (en) | Evaluation method, system, equipment and medium for cerebral aneurysm hemodynamic index | |
| US10398386B2 (en) | Systems and methods for estimating blood flow characteristics from vessel geometry and physiology | |
| Jafari et al. | Automatic biplane left ventricular ejection fraction estimation with mobile point-of-care ultrasound using multi-task learning and adversarial training | |
| CN108492300B (en) | Lung blood vessel tree segmentation method combining tubular structure enhancement and energy function | |
| CN110461240A (en) | System, method and computer accessible for ultrasonic analysis | |
| CN108665449B (en) | Deep learning model and device for predicting blood flow characteristics on blood flow vector path | |
| Kurugol et al. | Automated quantitative 3D analysis of aorta size, morphology, and mural calcification distributions | |
| CN111415321B (en) | Aneurysm rupture risk detection device and equipment | |
| Maaliw et al. | An enhanced segmentation and deep learning architecture for early diabetic retinopathy detection | |
| JP7369437B2 (en) | Evaluation system, evaluation method, learning method, trained model, program | |
| WO2020102154A1 (en) | Noninvasive quantitative flow mapping using a virtual catheter volume | |
| Moses et al. | Automatic segmentation and analysis of the main pulmonary artery on standard post-contrast CT studies using iterative erosion and dilation | |
| Lermé et al. | A fully automatic method for segmenting retinal artery walls in adaptive optics images | |
| Li et al. | RPS‐Net: An effective retinal image projection segmentation network for retinal vessels and foveal avascular zone based on OCTA data | |
| Marin-Castrillon et al. | 4D segmentation of the thoracic aorta from 4D flow MRI using deep learning | |
| Aghilinejad et al. | Framework development for patient-specific compliant aortic dissection phantom model fabrication: magnetic resonance imaging validation and deep-learning segmentation | |
| CN117813056A (en) | Ultrasound imaging for visualization and quantification of mitral regurgitation | |
| Baracho et al. | A hybrid neural system for the automatic segmentation of the interventricular septum in echocardiographic images | |
| Maurya et al. | Parse challenge 2022: Pulmonary arteries segmentation using swin u-net transformer (swin unetr) and u-net |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |