CN117064446A - Intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system - Google Patents
Intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system Download PDFInfo
- Publication number
- CN117064446A CN117064446A CN202311322255.5A CN202311322255A CN117064446A CN 117064446 A CN117064446 A CN 117064446A CN 202311322255 A CN202311322255 A CN 202311322255A CN 117064446 A CN117064446 A CN 117064446A
- Authority
- CN
- China
- Prior art keywords
- image sequence
- frame
- intravascular ultrasound
- dimensional reconstruction
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000002608 intravascular ultrasound Methods 0.000 title claims abstract description 47
- 230000002792 vascular Effects 0.000 title claims abstract description 20
- 210000004204 blood vessel Anatomy 0.000 claims abstract description 26
- 238000000605 extraction Methods 0.000 claims abstract description 13
- 238000002604 ultrasonography Methods 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000006243 chemical reaction Methods 0.000 claims description 27
- 239000011159 matrix material Substances 0.000 claims description 22
- 238000000034 method Methods 0.000 claims description 15
- 239000013598 vector Substances 0.000 claims description 15
- 238000013507 mapping Methods 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 6
- 238000004088 simulation Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 230000000737 periodic effect Effects 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- 230000008602 contraction Effects 0.000 claims description 2
- 230000003902 lesion Effects 0.000 claims description 2
- 150000002632 lipids Chemical class 0.000 claims description 2
- 238000007620 mathematical function Methods 0.000 claims description 2
- 230000007246 mechanism Effects 0.000 claims description 2
- 238000007689 inspection Methods 0.000 claims 1
- 230000000747 cardiac effect Effects 0.000 abstract description 8
- 230000006870 function Effects 0.000 description 15
- 238000003384 imaging method Methods 0.000 description 6
- 238000003745 diagnosis Methods 0.000 description 4
- 238000012800 visualization Methods 0.000 description 4
- 208000024172 Cardiovascular disease Diseases 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000000523 sample Substances 0.000 description 3
- 208000007536 Thrombosis Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 208000029078 coronary artery disease Diseases 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 230000002526 effect on cardiovascular system Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000010247 heart contraction Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000013146 percutaneous coronary intervention Methods 0.000 description 2
- 230000001154 acute effect Effects 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000017531 blood circulation Effects 0.000 description 1
- 230000007211 cardiovascular event Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 210000002808 connective tissue Anatomy 0.000 description 1
- 210000004351 coronary vessel Anatomy 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012631 diagnostic technique Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000001990 intravenous administration Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000035790 physiological processes and functions Effects 0.000 description 1
- 230000000541 pulsatile effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000241 respiratory effect Effects 0.000 description 1
- 238000002560 therapeutic procedure Methods 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 231100000216 vascular lesion Toxicity 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000036642 wellbeing Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/08—Detecting organic movements or changes, e.g. tumours, cysts, swellings
- A61B8/0891—Detecting organic movements or changes, e.g. tumours, cysts, swellings for diagnosis of blood vessels
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/46—Ultrasonic, sonic or infrasonic diagnostic devices with special arrangements for interfacing with the operator or the patient
- A61B8/461—Displaying means of special interest
- A61B8/466—Displaying means of special interest adapted to display 3D data
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
- A61B8/5238—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/5269—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving detection or reduction of artifacts
- A61B8/5276—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving detection or reduction of artifacts due to motion
Abstract
The invention relates to the technical field of vascular reconstruction, and discloses a vascular dynamic three-dimensional reconstruction system based on intravascular ultrasound, which comprises the following components: the ultrasonic equipment is used for collecting ultrasonic information in the blood vessel to be detected; a processor coupled to the ultrasound device, the processor comprising: the data processing module is used for generating an analog image sequence from intravascular ultrasound information; the gating frame extraction module is used for extracting gating frame images from the analog image sequence to obtain a gating frame image sequence; the missing frame generation module is used for extracting time sequence characteristics of the gating frame image sequence to generate a missing frame image sequence; and the three-dimensional reconstruction module is used for obtaining a three-dimensional reconstruction image based on the gating frame image sequence and the missing frame image sequence. Not only can the motion artifact problem in intravascular ultrasound be effectively removed, but also the intravascular ultrasound sequence after the key frame is extracted can be subjected to missing frame compensation, and finally the dynamic three-dimensional reconstruction of the cardiac blood vessel is realized based on the cardiac motion period.
Description
Technical Field
The invention relates to the technical field of vascular reconstruction, in particular to a vascular dynamic three-dimensional reconstruction system based on intravascular ultrasound.
Background
The statements in this section merely relate to the background of the present disclosure and may not necessarily constitute prior art.
Cardiovascular disease has a significant impact on the health, well-being and life expectancy of the population. Currently, imaging diagnostic techniques have focused on the display and assessment of target organ reconstruction and function prior to the occurrence of an acute cardiovascular event. Intravascular ultrasound (IVUS) imaging has long been attracting interventional doctors for its utility in Percutaneous Coronary Intervention (PCI) procedures. Intravascular ultrasound is a new technique applied to clinical diagnosis of vascular lesions in recent years. It is a medical imaging technique combining non-invasive ultrasound technology and invasive catheter technology using a special catheter with an ultrasound probe attached at its end.
Intravascular ultrasound can not only react to changes in the lumen of the blood vessel, but also to plaque-containing lumen cross-sectional structures, vessel wall thickness, morphology, plaque components, and the like. According to the lumen cross section information provided by intravascular ultrasound, not only can the diameter and volume of the lumen and the size of the plaque be accurately measured, but also the tissue information of the plaque can be provided, thereby assisting the diagnosis of coronary heart disease and effective interventional therapy.
However, the two-dimensional image obtained by intravascular ultrasound imaging does not reflect the spatial structure information of the interior of the blood vessel well, and therefore, three-dimensional reconstruction of the two-dimensional image by image processing techniques is a necessary requirement for the development of modern medicine. Compared with the traditional two-dimensional visualization, the three-dimensional visualization can better show conditions such as intravascular stent adherence and thrombosis. This is a time-consuming and lengthy large project requiring long and intensive research, but it has great significance for diagnosis and treatment of cardiovascular diseases.
The main difficulties of reconstructing a three-dimensional model of a vessel segment of interest using intravascular ultrasound images are: motion artifacts exist in the acquired intravascular ultrasound image sequences, so that extraction of the vessel wall cannot be accurately performed. In addition, the optical fiber of the imaging system is longitudinally scanned and retracted along the blood vessel at the speed of 1-3 mm/s, so that the interval between the ultrasonic frame images in the adjacent blood vessels is larger; in addition, the intravascular ultrasound one-time imaging process comprises a plurality of cardiac cycles, and the area of the vascular lumen is changed by 10% on average when the blood vessel expands and contracts in the same cycle; meanwhile, when the blood vessel is rebuilt, the surface of the blood vessel with the branch and the bending degree is intersected automatically; these factors greatly affect the accuracy of the three-dimensional reconstruction.
Disclosure of Invention
In order to solve the defects in the prior art, the invention provides a vascular dynamic three-dimensional reconstruction system based on intravascular ultrasound; not only can the motion artifact problem in intravascular ultrasound be effectively removed, but also the intravascular ultrasound sequence after the key frame is extracted can be subjected to missing frame compensation, and finally the dynamic three-dimensional reconstruction of the cardiac blood vessel is realized based on the cardiac motion period.
An intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system comprising:
the ultrasonic equipment is used for collecting ultrasonic information in the blood vessel to be detected;
a processor coupled to the ultrasound device, the processor comprising:
the gating frame extraction module extracts gating frame images from the real image sequence to obtain a gating frame image sequence;
the data processing module is used for generating a simulation image sequence under a complete sequence based on the proposed gating frame image;
the missing frame generation module is used for carrying out time sequence feature extraction on the simulated image sequence to generate a missing frame image sequence;
and the three-dimensional reconstruction module is used for obtaining a three-dimensional reconstruction image based on the gating frame image sequence and the missing frame image sequence.
One of the above technical solutions has the following advantages or beneficial effects:
according to the invention, by collecting and processing the intravascular ultrasound images of the heart and carrying out dynamic three-dimensional reconstruction according to the law of heart beating by using a time sequence manifold method, not only can the motion artifact problem in intravascular ultrasound be effectively removed, but also the intravascular ultrasound sequence after key frames are extracted can be subjected to missing frame compensation, and finally, the dynamic three-dimensional reconstruction of the heart blood vessel is realized based on the heart motion period, and the accuracy of the three-dimensional reconstruction of the blood vessel is improved. Compared with the traditional two-dimensional visualization, the three-dimensional visualization can better show the conditions of intravascular stent adherence, thrombosis and the like, and has great significance for diagnosis and treatment of cardiovascular diseases.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
Fig. 1 is a functional block diagram of a system according to a first embodiment of the present invention.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
Example 1
The embodiment provides a vascular dynamic three-dimensional reconstruction system based on intravascular ultrasound;
as shown in fig. 1, an intravascular ultrasound-based intravascular dynamic three-dimensional reconstruction system includes:
the ultrasonic equipment is used for collecting ultrasonic information in the blood vessel to be detected;
a processor coupled to the ultrasound device, the processor comprising:
the gating frame extraction module extracts gating frame images from the real image sequence to obtain a gating frame image sequence;
the data processing module is used for generating a simulation image sequence under a complete sequence based on the proposed gating frame image;
the missing frame generation module is used for carrying out time sequence feature extraction on the simulated image sequence to generate a missing frame image sequence;
and the three-dimensional reconstruction module is used for obtaining a three-dimensional reconstruction image based on the gating frame image sequence and the missing frame image sequence.
Further, the ultrasonic device is used for acquiring intravascular ultrasonic image information, namely placing a guide catheter at a coronary artery opening, sending a guide wire to the far end of a target blood vessel, sending the intravascular ultrasonic catheter to the far end of a lesion position to be inspected along the guide wire, inspecting by adopting a mode of continuously retracting from the far end to the near end of the target blood vessel at a set speed, and displaying a blood vessel cross section image and blood vessel wall plaque distribution through sound wave scanning and reflection, thereby continuously acquiring intravascular images within a set time period.
Illustratively, intravascular ultrasound images are acquired with an interventional ultrasound device. Intravascular ultrasound (intravenous ultrasound, IVUS) refers to medical imaging techniques using special catheters with ultrasound probes attached at their ends, in combination with non-invasive ultrasound techniques and invasive catheter techniques. The miniaturized ultrasonic transducer is placed in the cardiovascular cavity by the cardiac catheter, and the annular array of the multi-chip transducer or the single-chip transducer is rotated at high speed (1800 rpm) to finishDynamic blood vessel section scanning, through the imaging processing system, echo signal intensity is displayed in a gray scale mode, a two-dimensional cross section is imaged, and cardiovascular section morphology is displayed.
Further, the gating frame extraction module includes:
the input unit is used for inputting a real acquisition image sequence;
the transformation unit is used for mapping the real image sequence from a high-dimensional space to a low-dimensional space by adopting Laplace feature mapping to obtain a low-dimensional feature vector;
and the clustering unit is used for clustering the low-dimensional feature vectors of different frames, selecting a clustering center and taking the clustering center as a gating frame image.
Gating frame refers to the frame at the end of diastole of the heart.
Further, the transforming unit is configured to map the simulated image sequence from a high-dimensional space to a low-dimensional space by using laplace feature mapping, so as to obtain a low-dimensional feature vector, and specifically includes:
assuming that the number of data instances is n, the dimension of the target subspace (i.e., the dimension of the final dimension-reduction target) is m; definition of the definitionA matrix Y of size, wherein each row vector +.>Is data instance->Vector representation in the target m-dimensional subspace (i.e. data instance after dimension reduction +.>) Then the objective function of the laplacian mapping optimization is:
wherein,，/>is a data sample->，/>Vector representation in m-dimensional subspace, < ->To construct the adjacency matrix, we specifically state:
wherein,is a specified constant.
If the two data samples i and j are similar, then i and j are as close as possible in the reduced dimension subspace as in the original space.
Gating is understood to mean either by triggering data acquisition when a physiological event is detected, or by continuously acquiring data and then correlating it with the cardiac or respiratory cycle; gating key frames refer to image frames acquired by the heart at end diastole.
It should be understood that, firstly, continuous intravascular ultrasound IVUS image frames are input, laplace feature mapping in a manifold learning method is applied, a high-dimensional intravascular ultrasound IVUS image sequence is reduced to a low-dimensional manifold, a distance function is constructed to reflect the heart motion law by using the low-dimensional feature vector, and a clustering center is finally selected by continuously transmitting information between different points by using a proximity propagation clustering algorithm AP (Affinity propagation) to complete clustering. And taking the end diastole position in the intravascular ultrasound IVUS image as a clustering center, and extracting key frames to form a gating sequence.
Further, the data processing module includes:
an area determining unit for determining an area of the lumen of the blood vessel;
a contour generation unit for generating a continuous vessel cross-sectional contour along with a long axis of the vessel according to the periodical change of the lumen area;
and the image sequence generating unit is used for generating a simulation image sequence according to the intravascular lumen area and the blood vessel cross-section outline.
Further, the area determining unit is configured to determine an intravascular lumen area, and specifically includes:
area of lumen of vesselThe expression of (2) is defined as:
wherein,expansion factor representing a determined maximum area, +.>Is a time parameter, +.>Is to->Heart rate in units of a number of heart rates,is a constant that determines the time at which the maximum area occurs, the lumen area reaches a minimum value of lumen area +.>In the followingReach maximum value of lumen area->。
Further, the profile generating unit is configured to generate a continuous vessel cross-section profile along with a long axis of a vessel according to a periodic variation of a lumen area, and specifically includes:
assuming that the first frame is acquired at end diastole, indicating that the lumen area reaches a minimum in the first frame (i.e., n=0), the lumen profile in the n+1th frame is obtained by expanding or contracting the profile in frame n as follows:
wherein,and->Representing +.sup.th on lumen contour in frame n and frame n+l, respectively>Individual pointsPolar coordinates of>And->Respectively indicates the polar angle and the polar radius, and +.>Is a scaling factor determining the extent of expansion or contraction of the lumen contour, +.>In the range of [0,1 ]]Within the interval of->Is a time parameter.
For the lumen profile of the tube,is set to 1.
In the case of calcified plaque,because of their poor elasticity compared to the lumen.
For fibrous lipid plaques, set upBecause they have better elasticity than the lumen.
In this way, the vessel wall contours in subsequent frames, including lumen contours, media/adventitia boundaries, and plaque contours, are obtained from the contours in previous frames. Wherein the first frameThe image is real data, i.e. a gating frame image.
It should be appreciated that the coronary lumen periodically expands or contracts with the heartbeat and pulsatile blood flow during the cardiac cycle. Due to the regular periodic structure of heart rate, lumen area is expressed as: time function，/>Reaching a minimum +.>Maximum +.>。
Further, the missing frame generating module includes:
and inputting the simulated image sequence into the trained time sequence data generation model to generate a missing frame image.
It should be appreciated that based on popular supervision and metric learning, in conjunction with generating the model, a missing frame is generated.
Further, the trained time sequence data generates a model, and the training stage comprises:
constructing a generator and a discriminator which are connected with each other;
the method comprises the steps of inputting random noise into a generator, completing conversion from Euclidean space to Riemann space through calculating a covariance matrix to obtain first conversion data, inputting the first conversion data into a first flow attention module, extracting features to obtain first features, and converting the first features from Riemann space to Euclidean space to obtain second conversion data;
the real image sequence is converted from Euclidean space to Riemann space through calculating a covariance matrix to obtain third conversion data, the third conversion data are input into a second flow attention module to perform feature extraction to obtain second features, and the second features are converted from Riemann space to Euclidean space to obtain fourth conversion data;
and (3) calculating the loss function of the discriminator as the difference degree of the second conversion data and the fourth conversion data, wherein the smaller the loss function value is, the smaller the difference degree is, and when the loss function value is not reduced any more, stopping training to obtain a trained time sequence data generation model.
Generator loss function:
wherein,is noise data +.>Refers to generating a function, +.>As a discriminant function->For the two classification cross entropy loss->The functional expression of (2) is:
where x is the input, target is the target (typically 0 or 1), and the ln function is a logarithmic function with the base of the constant e (euler number, approximately equal to 2.71828).
The discriminator compares the difference between the second conversion data and the fourth conversion data, the difference is obtained by a discriminator loss function,
r is real data and z is noise data. />Representing the difference between the judgment result of the discriminator and the real data, the smaller the loss value is, the more real the data generated by the generator is.
Representing the difference between the real data and 1, the smaller the value is, which indicates that the discrimination capability of the discriminator on the real data is better.
The purpose is to make the discriminator discriminate the data generated by the generator as false, and make the generator generate more real data.
For the arbiter, if a picture generated by the generator is obtained, the arbiter should output 0, and if a true picture should output 1.
The generator and the arbiter of the GAN continuously perform iterative optimization on the two models by means of gaming. The parameters of two models are initialized when training is started, samples are acquired from a real data set, noise is randomly generated, the initialized samples are used for training the discriminators, and the step is to acquire a discriminator capable of accurately discriminating the true image from the false image, so that the generator can be optimized according to feedback of the discriminators. After a better discriminator is obtained, the generated sample is discriminated by the discriminator, and the result is used for optimizing the generator. After completion of a training, the process will be repeated, except that the arbiter and generator at that time have been optimized through the previous stage.
The whole generating module comprises two parts, a generator and a discriminator. The generator inputs random noise, the conversion from Euclidean space to Riemann space is completed through calculating covariance matrix, then feature extraction is carried out through manifold attention module, and finally inverse transformation of Riemann space-Euclidean space is carried out. The discriminator is used for judging whether the real image sequence is generated by the generator or the actual data, firstly receives the input real image sequence, then realizes conversion from Euclidean space to Riemann space through calculating covariance matrix, inputs the converted data into the manifold attention module, and carries out Riemann space-Euclidean space conversion on the output characteristics, and the converted data and the data generated by the generator judge the authenticity.
Further, a first flow attention module and a first flow attention moduleThe working principle of the two-stream attention module is the same, and the first stream attention module is combined with the self-attention mechanism in the transducer to input sequence dataConversion matrix->:
Wherein,refers to a transformation matrix, ">Refers to the input data +.>The query matrix, the key matrix, and the value matrix after the matrix is transformed. Softmax is a mathematical function commonly used to convert a set of arbitrary real numbers intoReal numbers representing probability distributions. Exp refers to an exponential function underlying e, representing the power of e.
Further, the conversion from Euclidean space to Riemangio space, or from Riemangio space to Euclidean space, is achieved byThe operation is performed by using->The manifold is reduced to a flat space.
Wherein,refers to the expansion function, +.>Is a eigenvalue function, +.>Is->Is a matrix of feature vectors of (a),is->Is a characteristic value of (a).
Further, the three-dimensional reconstruction module is configured to obtain a three-dimensional reconstructed image based on the gating frame image sequence and the missing frame image sequence, and the specific reconstruction process includes:
and arranging the real first frame intravascular ultrasound image and the generated missing frame data according to the time law of heart beating, and finally realizing dynamic three-dimensional reconstruction of intravascular ultrasound.
Further, the system further comprises: and the display device is connected with the processor and used for displaying the dynamic three-dimensional reconstruction result in real time.
The invention utilizes the manifold generation mode to carry out dynamic three-dimensional reconstruction in the blood vessel, better displays the space structure information of the blood vessel on the premise of improving the three-dimensional reconstruction speed and the accuracy, is more beneficial to the visual judgment of doctors, improves the treatment efficiency of patients, reduces the workload of doctors and has great significance for the auxiliary diagnosis of coronary heart diseases.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system, comprising:
the ultrasonic equipment is used for collecting ultrasonic information in the blood vessel to be detected;
a processor coupled to the ultrasound device, the processor comprising:
the gating frame extraction module extracts gating frame images from the real image sequence to obtain a gating frame image sequence;
the data processing module is used for generating a simulation image sequence under a complete sequence based on the proposed gating frame image;
the missing frame generation module is used for carrying out time sequence feature extraction on the simulated image sequence to generate a missing frame image sequence;
and the three-dimensional reconstruction module is used for obtaining a three-dimensional reconstruction image based on the gating frame image sequence and the missing frame image sequence.
2. The intravascular ultrasound-based dynamic three-dimensional reconstruction system of a vessel according to claim 1, wherein the ultrasound device is configured to acquire intravascular ultrasound image information by placing a guide catheter at a coronary ostium, delivering a guide wire to a distal end of a target vessel, delivering the intravascular ultrasound catheter along the guide wire to a distal end of a lesion site to be inspected, and continuously retracting the catheter from the distal end to the proximal end of the target vessel at a set speed for inspection, and displaying a vessel cross-sectional image and a vessel wall plaque distribution by acoustic wave scanning and reflection, thereby continuously acquiring intravascular images within a set period of time.
3. The intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system of claim 1, wherein the gating frame extraction module comprises:
the input unit is used for inputting a real acquisition image sequence;
the transformation unit is used for mapping the real image sequence from a high-dimensional space to a low-dimensional space by adopting Laplace feature mapping to obtain a low-dimensional feature vector;
and the clustering unit is used for clustering the low-dimensional feature vectors of different frames, selecting a clustering center and taking the clustering center as a gating frame image.
4. The intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system according to claim 3, wherein the transformation unit is configured to map the simulated image sequence from a high-dimensional space to a low-dimensional space by using laplace feature mapping, so as to obtain a low-dimensional feature vector, and specifically comprises:
assuming that the number of data instances is n, the dimension of the target subspace is m; definition of the definitionA matrix Y of size, wherein each row vector +.>Is data instance->Vector representation in the target m-dimensional subspace, then the objective function of laplace mapping optimization is:
wherein,，/>is a data sample->，/>Vector representation in m-dimensional subspace, < ->To construct the adjacency matrix, we specifically state:
wherein,is a specified constant.
5. The intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system of claim 1, wherein the data processing module comprises:
an area determining unit for determining an area of the lumen of the blood vessel;
a contour generation unit for generating a continuous vessel cross-sectional contour along with a long axis of the vessel according to the periodical change of the lumen area;
and the image sequence generating unit is used for generating a simulation image sequence according to the intravascular lumen area and the blood vessel cross-section outline.
6. The intravascular ultrasound based vascular dynamic three-dimensional reconstruction system according to claim 5, wherein the area determination unit is configured to determine an intravascular lumen area, and specifically comprises:
area of lumen of vesselThe expression of (2) is defined as:
wherein,expansion factor representing a determined maximum area, +.>Is a time parameter, +.>Is to->Heart rate in units>Is a constant that determines the time at which the maximum area occurs, the lumen area reaches a minimum value of lumen area +.>In the followingReach maximum value of lumen area->。
7. The intravascular ultrasound based vascular dynamic three-dimensional reconstruction system according to claim 5, wherein the profile generating unit is configured to generate a continuous vascular cross-sectional profile along the long axis of the vessel according to a periodic variation of the lumen area, in particular comprising:
assuming that the first frame is acquired at end diastole, indicating that the lumen area reaches a minimum in the first frame, the lumen profile in the n+1th frame is obtained by expanding or contracting the profile in frame n as follows:
wherein,and->Representing +.sup.th on lumen contour in frame n and frame n+l, respectively>Polar coordinates of the individual points>And->Respectively indicates the polar angle and the polar radius, and +.>Is a scaling factor determining the extent of expansion or contraction of the lumen contour, +.>In the range of [0,1 ]]Within the interval of->Is a time parameter; for lumen profile->Is set to 1; for calcified plaques, the->The method comprises the steps of carrying out a first treatment on the surface of the For fibrous lipid plaques, set +.>。
8. The intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system of claim 1, wherein the missing frame generation module comprises:
inputting the simulated image sequence into a trained time sequence data generation model to generate a missing frame image;
the trained time sequence data generates a model, and the training stage comprises the following steps:
constructing a generator and a discriminator which are connected with each other;
the method comprises the steps of inputting random noise into a generator, completing conversion from Euclidean space to Riemann space through calculating a covariance matrix to obtain first conversion data, inputting the first conversion data into a first flow attention module, extracting features to obtain first features, and converting the first features from Riemann space to Euclidean space to obtain second conversion data;
the real image sequence is converted from Euclidean space to Riemann space through calculating a covariance matrix to obtain third conversion data, the third conversion data are input into a second flow attention module to perform feature extraction to obtain second features, and the second features are converted from Riemann space to Euclidean space to obtain fourth conversion data;
and (3) calculating the loss function of the discriminator as the difference degree of the second conversion data and the fourth conversion data, wherein the smaller the loss function value is, the smaller the difference degree is, and when the loss function value is not reduced any more, stopping training to obtain a trained time sequence data generation model.
9. The intravascular ultrasound based vascular dynamic three-dimensional reconstruction system of claim 8, wherein the first and second flow attention modules operate in the same manner, the first flow attention module being combined withTransformerIn (a) self-attention mechanism, inputting sequence dataConversion matrix-> ：
Wherein,refers to a transformation matrix, ">Refers to the input data +.>A query matrix, a key matrix and a value matrix after the matrix is transformed; softmax is a mathematical function for converting a set of arbitrary real numbers into real numbers representing a probability distribution; exp refers to an exponential function underlying e, representing the power of e.
10. The intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system of claim 8, wherein the conversion from euclidean space to euclidean space or vice versa is byThe operation is performed by using->Reducing manifold to flat space:
wherein,refers to the expansion function, +.>Is a eigenvalue function, +.>Is->Feature vector matrix, ">Is thatIs a characteristic value of (a).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311322255.5A CN117064446B (en) | 2023-10-13 | 2023-10-13 | Intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311322255.5A CN117064446B (en) | 2023-10-13 | 2023-10-13 | Intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117064446A true CN117064446A (en) | 2023-11-17 |
| CN117064446B CN117064446B (en) | 2024-01-12 |
Family
ID=88711979
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311322255.5A Active CN117064446B (en) | 2023-10-13 | 2023-10-13 | Intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117064446B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5476096A (en) * | 1994-02-09 | 1995-12-19 | Vingmed Sound A/S | Analysis and measurement of temporal tissue variations |
| CN101283929A (en) * | 2008-06-05 | 2008-10-15 | 华北电力大学 | Rebuilding method of blood vessel three-dimensional model |
| US20120283569A1 (en) * | 2011-05-04 | 2012-11-08 | Boston Scientific Scimed, Inc. | Systems and methods for navigating and visualizing intravascular ultrasound sequences |
| US10140733B1 (en) * | 2017-09-13 | 2018-11-27 | Siemens Healthcare Gmbh | 3-D vessel tree surface reconstruction |
| CN111771138A (en) * | 2018-02-27 | 2020-10-13 | 皇家飞利浦有限公司 | Ultrasound system with neural network for generating images from undersampled ultrasound data |
| CN112652052A (en) * | 2020-12-15 | 2021-04-13 | 山东大学 | Coronary artery three-dimensional reconstruction method and system based on blood vessel branch registration |
| CN114331877A (en) * | 2021-12-14 | 2022-04-12 | 山东大学 | Intravascular ultrasonic image restoration method and system |
| CN115393534A (en) * | 2022-10-31 | 2022-11-25 | 深圳市宝润科技有限公司 | Deep learning-based cone beam three-dimensional DR reconstruction method and system |
| CN115496672A (en) * | 2021-06-17 | 2022-12-20 | 通用电气精准医疗有限责任公司 | System and method for computed tomography image reconstruction |
| CN115661225A (en) * | 2022-12-26 | 2023-01-31 | 深圳微创踪影医疗装备有限公司 | Three-dimensional blood vessel image reconstruction method and device, computer equipment and storage medium |
| CN115965750A (en) * | 2023-03-16 | 2023-04-14 | 深圳微创踪影医疗装备有限公司 | Blood vessel reconstruction method, device, computer equipment and readable storage medium |
| CN116188677A (en) * | 2022-12-19 | 2023-05-30 | 同济大学 | Three-dimensional reconstruction method, system and device for vascular intervention operation area |
| CN116188452A (en) * | 2023-03-17 | 2023-05-30 | 昆明理工大学 | Medical image interlayer interpolation and three-dimensional reconstruction method |
-
2023
- 2023-10-13 CN CN202311322255.5A patent/CN117064446B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5476096A (en) * | 1994-02-09 | 1995-12-19 | Vingmed Sound A/S | Analysis and measurement of temporal tissue variations |
| CN101283929A (en) * | 2008-06-05 | 2008-10-15 | 华北电力大学 | Rebuilding method of blood vessel three-dimensional model |
| US20120283569A1 (en) * | 2011-05-04 | 2012-11-08 | Boston Scientific Scimed, Inc. | Systems and methods for navigating and visualizing intravascular ultrasound sequences |
| US10140733B1 (en) * | 2017-09-13 | 2018-11-27 | Siemens Healthcare Gmbh | 3-D vessel tree surface reconstruction |
| CN111771138A (en) * | 2018-02-27 | 2020-10-13 | 皇家飞利浦有限公司 | Ultrasound system with neural network for generating images from undersampled ultrasound data |
| CN112652052A (en) * | 2020-12-15 | 2021-04-13 | 山东大学 | Coronary artery three-dimensional reconstruction method and system based on blood vessel branch registration |
| CN115496672A (en) * | 2021-06-17 | 2022-12-20 | 通用电气精准医疗有限责任公司 | System and method for computed tomography image reconstruction |
| CN114331877A (en) * | 2021-12-14 | 2022-04-12 | 山东大学 | Intravascular ultrasonic image restoration method and system |
| CN115393534A (en) * | 2022-10-31 | 2022-11-25 | 深圳市宝润科技有限公司 | Deep learning-based cone beam three-dimensional DR reconstruction method and system |
| CN116188677A (en) * | 2022-12-19 | 2023-05-30 | 同济大学 | Three-dimensional reconstruction method, system and device for vascular intervention operation area |
| CN115661225A (en) * | 2022-12-26 | 2023-01-31 | 深圳微创踪影医疗装备有限公司 | Three-dimensional blood vessel image reconstruction method and device, computer equipment and storage medium |
| CN115965750A (en) * | 2023-03-16 | 2023-04-14 | 深圳微创踪影医疗装备有限公司 | Blood vessel reconstruction method, device, computer equipment and readable storage medium |
| CN116188452A (en) * | 2023-03-17 | 2023-05-30 | 昆明理工大学 | Medical image interlayer interpolation and three-dimensional reconstruction method |
Non-Patent Citations (3)
| Title |
|---|
| 吴清江: "基于生成对抗性网络的低剂量CT图像重建系统开发", 中国优秀硕士学位论文全文数据库（电子期刊）, pages 138 - 1851 * |
| 李锦欣: "基于生成对抗网络的医学图像超分辨率重建", 中国优秀硕士学位论文全文数据库（电子期刊）, pages 060 - 12 * |
| 毛海群: "基于流形学习的血管内超声图像序列关键帧的提取及应用", 南方医科大学学报, vol. 35, no. 4, pages 492 - 498 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117064446B (en) | 2024-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5944917B2 (en) | Computer readable medium for detecting and displaying body lumen bifurcation and system including the same | |
| US7215802B2 (en) | System and method for vascular border detection | |
| PANDIAN et al. | Three‐dimensional and four‐dimensional transesophageal echocardiographic imaging of the heart and aorta in humans using a computed tomographic imaging probe | |
| Nelson et al. | Three-dimensional ultrasound imaging | |
| CN108038848A (en) | Quick calculation method and system based on medical image sequence plaque stability index | |
| CN114145719B (en) | Method and system for three-dimensional fusion of dual-mode coronary vessel images | |
| WO2003077765A1 (en) | Ultrasonographic system and ultrasonography | |
| CN110415248A (en) | A kind of blood vessel monitoring method, device, equipment and storage medium based on ultrasound | |
| US20230008714A1 (en) | Intraluminal image-based vessel diameter determination and associated devices, systems, and methods | |
| CN116705330B (en) | Method, computing device and medium for determining elastic characteristics of a vessel wall | |
| CN116030968A (en) | Blood flow reserve score prediction method and device based on intravascular ultrasound image | |
| Zheng et al. | A deep learning method for motion artifact correction in intravascular photoacoustic image sequence | |
| TW201936120A (en) | Ultrasonic wave image uniformity detection method and system | |
| JP7371105B2 (en) | Methods and systems for investigating vascular properties | |
| CN117064446B (en) | Intravascular ultrasound-based vascular dynamic three-dimensional reconstruction system | |
| Zheng et al. | A computer-based simulator for intravascular photoacoustic images | |
| Ciompi et al. | Reconstruction and analysis of intravascular ultrasound sequences | |
| Xiao | Key Technologies of New Type of Intravascular Ultrasound Image Processing | |
| CN111180075B (en) | Dynamics model established based on heart murmur and computer simulation method | |
| Jansen et al. | Local and global distensibility assessment of abdominal aortic aneurysms in vivo from probe tracked 2D ultrasound images | |
| Yang et al. | A review on artery wall segmentation techniques and intima-media thickness measurement for carotid ultrasound images | |
| Rahimi | Investigation of Trimodality Image Registration for Guiding Cardiovascular Diseases Treatment Procedure | |
| Li | Ultrasonic pulse wave imaging for in vivo assessment of vascular wall dynamics and characterization of arterial pathologies | |
| Ding et al. | Ultrasound computed tomography imaging technique advances in HUST | |
| CN114463267A (en) | Fractional flow reserve prediction method and device based on optical coherence tomography and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |