CN114758020A - DSA and OCT fusion developing-based method and device and electronic equipment - Google Patents

Abstract

The invention provides a DSA and OCT fusion developing-based method, a DSA and OCT fusion developing-based device and electronic equipment, wherein the method comprises the following steps: DSA sequence images and OCT sequence images are synchronously and continuously acquired; time marking the acquired DSA sequence image and the OCT sequence image; evenly distributing the obtained DSA sequence images to different GPU units according to the time sequence of the marks for processing, and calculating and marking the positions of the developing rings in the DSA sequence images; and synchronously displaying the OCT sequence image and the DSA sequence image after marking the position of the developing ring on the same display according to the marked time. According to the invention, DSA and OCT fusion images can be rapidly and accurately obtained by matching the position of the developing ring on the DSA sequence image and the OCT sequence image obtained by the synchronous lens, so that a doctor can more clearly and intuitively see the withdrawal condition of the OCT lens in the DSA image in a blood vessel, and image reference and technical support are provided for the subsequent positioning image to position the lesion condition of the blood vessel.

Description

DSA and OCT fusion developing-based method and device and electronic equipment

Technical Field

The invention belongs to the technical field of medical equipment, and particularly relates to a DSA and OCT fusion developing-based method, a DSA and OCT fusion developing-based device and electronic equipment.

Background

The Optical Coherence Tomography (OCT) integrates a semiconductor laser technology, an Optical technology, a computer image processing technology and the like, realizes non-contact and non-invasive in vivo morphological detection on a human body, and obtains a cross-sectional image of an internal microstructure of a biological tissue for guiding interventional therapy of a percutaneous coronary artery, including immediate observation after stent implantation and evaluation of long-term intravascular repair after stent implantation.

The OCT technology is used for imaging coronary vessels after the stent is implanted, an observer can clearly observe each atherosclerotic plaque characteristic, whether the stent is well attached to a vessel wall or not is evaluated, whether stent position covering, tissue prolapse, tissue tearing, in-stent restenosis, plaque or thrombus exists or not and the like are determined, and the OCT technology is of great significance to clinical diagnosis and treatment.

The imaging catheter is a catheter used with an optical interference tomography system for coronary artery imaging, and is used with an optical interference tomography system for coronary artery imaging of a patient in need of intraluminal interventional treatment in a medical facility.

The basic principle of Digital Subtraction Angiography (DSA) is to digitally input two frames of X-ray images taken before and after the injection of a contrast agent into an image computer, and remove bone and soft tissue images on the angiographic image through subtraction, enhancement and re-imaging processes to obtain a clear angiographic image. DSA has high contrast resolution, clear image and less contrast agent consumption, and has very important significance in vascular intervention treatment and diagnosis.

The DSA blood vessel image fusion technology is characterized in that an OCT imaging catheter is inserted into a blood vessel to reach a coronary artery lesion or a stent far end by utilizing cardiac imaging catheterization in coronary angiography, and then contrast agent is injected into the coronary artery to enable the coronary artery to be visible on an X-ray photograph, so that a doctor can master the blockage condition in the artery by observing DSA images and the morphological structure of the cross section of the OCT blood vessel. However, because the physical distance between the DSA and OCT devices is limited, it is time-consuming and labor-consuming to manually match DSA and OCT images, and a contrast result cannot be given on site, so the DSA image fusion technique is adopted to quickly match DSA and OCT images and accurately track the position of the developing ring, which is convenient for a doctor to judge the position of the imaging catheter and the vascular lesion, and further determine a diagnosis result and a treatment scheme.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a DSA and OCT fusion developing-based method and a DSA and OCT fusion developing-based device.

Specifically, the present invention relates to the following aspects:

1. a method based on DSA and OCT fusion visualization, the method comprising:

synchronously and continuously acquiring DSA sequence images and OCT sequence images;

time marking the acquired DSA sequence image and the OCT sequence image;

Evenly distributing the obtained DSA sequence images to different GPU units according to the time sequence of marking for processing, and calculating and marking the positions of developing rings in the DSA sequence images;

and synchronously displaying the OCT sequence image and the DSA sequence image after marking the position of the developing ring on the same display according to the marked time.

2. The method of item 1, wherein the processing is selected from one or more of Hessian enhanced multi-scale filtering processing, Gaussian Laplace point processing, and local feature point processing.

3. The method of item 1, wherein the processing comprises Hessian enhanced multi-scale filtering, laplacian of gaussian processing, and local feature point processing.

4. The method of item 1, wherein the processing yields vessel paths, local feature points, and laplacian of gaussian points for the DSA sequence images for calculating and labeling the location of the visualization ring in the DSA sequence images.

5. An apparatus based on DSA and OCT fusion visualization, the apparatus comprising:

an image acquisition unit for continuously acquiring DSA sequence images and OCT sequence images synchronously;

a time-stamping unit for time-stamping the acquired DSA sequence images and OCT sequence images;

The image processing unit is used for evenly distributing the obtained DSA sequence images to different GPU units according to the marked time sequence for processing, and calculating and marking the positions of the developing rings in the DSA sequence images;

and the image fusion unit is used for synchronously presenting the OCT sequence images and the DSA sequence images marked with the positions of the developing rings on the same display according to the marked time respectively.

6. The apparatus of item 5, wherein the GPU unit comprises one or more of a Hessian enhanced multiscale filter processing sub-unit, a Gaussian Laplace processing sub-unit, and a local feature point processing sub-unit.

7. The apparatus of item 5, wherein the GPU unit comprises a plurality of Hessian enhanced multi-scale filtering processing sub-units, a plurality of Gaussian Laplace point processing sub-units, and a plurality of local feature point processing sub-units.

8. The apparatus of item 6, wherein the Hessian enhanced multi-scale filtering processing sub-unit is configured to compute a vessel path that results in DSA sequence images.

9. The apparatus according to item 6, wherein the laplacian of gaussian processing subunit is configured to calculate the laplacian of gaussian for obtaining the DSA sequence images.

10. The apparatus according to item 6, wherein the local feature point processing subunit is configured to calculate local feature points of the DSA sequence images.

11. An electronic device, characterized in that the electronic device comprises:

a processor; and

a memory having stored therein computer program instructions that, when executed by the processor, cause the processor to perform the method of any of items 1-4.

According to the invention, DSA and OCT fusion images can be rapidly and accurately obtained by matching the position of the developing ring on the DSA sequence image and the OCT sequence image obtained by the synchronous lens, so that a doctor can more clearly and intuitively see the withdrawal condition of the OCT lens in the DSA image in a blood vessel, and image reference and technical support are provided for the subsequent positioning image to position the lesion condition of the blood vessel.

Drawings

Fig. 1 is a flowchart of a DSA and OCT fusion visualization based method according to an embodiment of the present invention.

Fig. 2 is a block diagram of a DSA and OCT fusion visualization based apparatus according to an embodiment of the present invention.

The method comprises the steps of 1, an image acquisition unit, 2, a time marking unit, 3, an image processing unit and 4, an image fusion unit.

Detailed Description

The present invention is further illustrated by the following examples, which are intended to be purely exemplary and are not intended to limit the scope of the invention.

Unless defined otherwise, technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although methods and materials similar or equivalent to those described herein can be used in the practice or experimental applications, the materials and methods are described below. In case of conflict, the present specification, including definitions, will control, and the materials, methods, and examples are illustrative only and not intended to be limiting. The present invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.

As described above, in order to solve the problems existing in the prior art of the DSA image and OCT image fusion, the present invention provides a method based on DSA and OCT fusion visualization, as shown in fig. 1, the method includes:

s1, DSA sequence images and OCT sequence images are synchronously and continuously acquired;

s2, the acquired DSA sequence images and OCT sequence images are time-stamped;

s3, evenly distributing the obtained DSA sequence images to different GPU units according to the time sequence of marking for processing, calculating and marking the position of the developing ring in the DSA sequence images;

S4 displays the OCT sequence images and the DSA sequence images after marking the position of the developing ring on the same display in time synchronization with each other.

In step S1, the DSA sequence images and the OCT sequence images are acquired simultaneously, and the DSA sequence images and the OCT sequence images are actually a collection of a plurality of DSA sequence images and OCT sequence images that are continuous in time.

In step S2, the acquired DSA sequence images and OCT sequence images are time-stamped, on the one hand, for the subsequent DSA sequence images to be calculated in time series, and on the other hand, for the matching of the final OCT sequence images and DSA sequence images.

In step S3, the DSA sequence images obtained are evenly distributed to different GPU units according to the time order of the markers for processing, which may specifically adopt GPU-CPU cooperative computing.

The GPU-CPU cooperative computing is used as a special parallel processing mode, the capability of different computing resources can be exerted according to the characteristics of related tasks, and great advantages are achieved in the aspects of improving the computing performance, the energy efficiency ratio and the real-time performance of equipment. After DSA sequence image signals are obtained, due to the fact that the data size is large and the time cost is large when the images are processed in a single-flow water line mode, a large amount of waiting time in the operation process can be reduced through GPU-CPU cooperative computing.

Specifically, the processing of the DSA sequence images in the GPU unit may be selected from one or more of Hessian enhanced multi-scale filtering processing, laplacian of gaussian processing, and local feature point processing. The Hessian enhanced multi-scale filtering processing can calculate the vascular path of the DSA sequence image, the Gaussian point processing can calculate the Gaussian point of the DSA sequence image, and the local feature point processing can calculate the local feature point of the DSA sequence image.

In a specific embodiment, the processing includes Hessian enhanced multi-scale filtering processing, gaussian laplacian point processing, and local feature point processing.

In a specific embodiment, the processing results in a vessel path, local feature points, and laplacian of gaussian points for the DSA sequence images for calculating and labeling the location of the visualization ring in the DSA sequence images.

Specifically, after a blood vessel path, a local characteristic point and a Gaussian point of a DSA sequence image are obtained, a DSA image signal with clear contrast is selected, the position of an OCT lens (development ring) on the DSA image blood vessel, the starting point and the stopping point of an observation channel are marked, and the path of the DSA image observation channel is solved;

And solving the observation channel path of the DSA sequence image in batch by using a motion equation, obtaining the estimated position of the development ring of the DSA sequence image, performing self-learning template matching on the estimated position of the development ring and the position of the Gaussian Laplace and a unit bitmap of the development ring position of the DSA image signal with clear contrast, and obtaining the position of the development ring of the DSA sequence image.

When the number of DSA sequence images is 45, the time spent in DSA processing is shortened to 1/(45 x 3) calculated by the sequence before the method is adopted, so the processing efficiency is greatly improved.

The invention also provides a DSA and OCT fusion visualization based device, as shown in FIG. 2, the device includes:

an image acquisition unit 1 for continuously acquiring DSA sequence images and OCT sequence images in synchronization;

a time-stamping unit 2 for time-stamping the acquired DSA sequence images and OCT sequence images;

the image processing unit 3 is used for evenly distributing the acquired DSA sequence images to different GPU units according to the marked time sequence for processing, and calculating and marking the positions of the developing rings in the DSA sequence images;

and the image fusion unit 4 is used for synchronously presenting the OCT sequence images and the DSA sequence images marked with the development ring positions on the same display according to the marked time respectively.

In a specific embodiment, the image acquisition unit 1 is an image acquisition card of an OCT apparatus.

In a specific embodiment, the GPU unit includes one or more of a Hessian enhanced multi-scale filtering processing sub-unit, a laplacian of gaussian processing sub-unit, and a local feature point processing sub-unit. The Hessian enhanced multi-scale filtering processing subunit is used for calculating a blood vessel path of a DSA sequence image. The Gaussian Laplace point processing subunit is used for calculating the Gaussian Laplace points of the DSA sequence images. The local characteristic point processing subunit is used for calculating and obtaining local characteristic points of the DSA sequence images.

In a specific embodiment, the GPU unit includes a plurality of multi-scale filtering processing sub-units, a plurality of laplacian of gaussian processing sub-units, and a plurality of local feature point processing sub-units.

In addition to the above method and apparatus, an embodiment of the present invention further provides an electronic apparatus, including: a processor; and

a memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform the method as described above.

The processor may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

The memory may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium and executed by a processor to implement the DSA and OCT fusion visualization-based methods described above and/or other desired functions. Various contents such as a prediction result may also be stored in the computer-readable storage medium.

The computer program may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages, to carry out operations according to embodiments of the present application. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Further, the memory may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Examples

Step 1: and (4) signal acquisition. After relevant preparation operations such as vessel intervention, contrast agent injection, PCU motion control unit connection and the like are carried out through an HDMI cable, the PCU motion control unit is driven to carry out vessel retraction, meanwhile, the acquisition card is driven to acquire sequence image signals of the DSA device, the frequency of the DSA sequence image signals is assumed to be 15fps (30 fps, 45fps and the like), the frequency of the OCT sequence image signals is assumed to be 90fps (60 fps, 120fps and the like), and the retraction time is assumed to be 3 s. 45 DSA sequence image signals and 270 OCT sequence image signals are synchronously acquired. Thus, a correspondence of 1: 6, one DSA sequence image signal corresponds to 6 OCT sequence image signals.

And 2, step: feature extraction and GPU parallel acceleration. Due to the independence of the Hessian enhanced multi-scale filtering module, the Gaussian Laplace point cloud module and the local feature point cloud module, a plurality of working units, namely threads, can be distributed in each flow structure of the GPU, and the threads can synchronously perform sub-calculation on 3 threads without interference. And simultaneously, carrying out data blocking on the DSA sequence image signals, wherein a stream structure exists in the GPU, the calculation in different stream structures can be carried out simultaneously, and the calculation in the same stream is carried out in sequence. And the 45 image signals of the DSA sequence are mutually independent, the task is calculated to be highly repetitive, and the DSA sequence is distributed into grids and blocks corresponding to different stream structures of the GPU, wherein each Grid comprises NB blocks, and each Block comprises NT threads. The data are divided and distributed to each thread and the calculation required to be executed by the thread, so that the multi-thread parallel calculation of the N image signals is realized.

Therefore, assuming that the single subtask time is Tw, the conventional CPU needs 3 × 45 × Tw to complete the computation of 3 subtasks of 45 image signals in the current DSA sequence, and the computation time needs only Tw through the GPU in parallel acceleration.

And 3, step 3: the observation channel path P and the development ring position R of the single image signal are drawn. Selecting a DSA image signal with clear contrast from the DSA sequence image signals on the integrated device, and selecting the position R of an OCT lens (development ring) and the positions of a starting point S and a stopping point E of an observation channel on the DSA image blood vessel for marking. And taking a Hessian enhanced multi-scale filtering vessel mask as a boundary, taking local feature point clouds (points with the maximum gray value in a block matrix) as a map of an observation channel, and obtaining an observation channel path of a single image through Dijkstra shortest path planning.

Compared with a topological refined central line method, the method considers the bending characteristic of the catheter and the characteristic that the catheter is not in the central position of the blood vessel, increases the probability that the developing ring appears in the catheter, provides reference for subsequent developing ring identification, and increases the identification accuracy.

And 4, step 4: and drawing an observation channel path PN of the DSA sequence image signal. Because the blood vessel is translated, stretched and contracted due to the pacing of the heart, the change of the DSA sequence image in the previous and subsequent frames is large, and therefore, the motion equation of each image needs to be solved to obtain the observation channel path of the DSA sequence image signal. Firstly, according to the characteristics that the position of a stopping point E is the position of a proximal end point of a main blood vessel, the proximal end point is close to the heart, and main branches are more and well recognized, all stopping point positions EN of DSA sequence image signals are obtained through template matching, and offset XN and YN of all stopping points EN relative to the stopping point E of a marked image are obtained. And then carrying out affine matching on the marked observation channel path to obtain an angle offset A and a scale C of the DSA sequence image, and combining the offset X and Y of the current image signal relative to the marked signal to obtain an affine matrix D, wherein D is a motion equation of a single image. And secondly, solving the motion equation in a parallelization manner to obtain all motion equations DN of the DSA sequence image signals, and solving the initial point SN by substituting the coordinates of the initial point S into the motion equations DN. And finally, obtaining all observation channel paths PN of the DSA sequence image signal according to the step 3 through the starting point SN and the stopping point EN.

And 5: and solving the position RN of the development ring. At the moment, RN is a point on an observation channel path PN, a plurality of reference points are obtained by combining an estimation point on the PN obtained by calculation and a Gaussian Laplace point with a Hessian enhanced multi-scale filtering blood vessel mask as a boundary, a template of a developing ring is obtained by iterative learning, the similarity between the reference points of DSA sequence image signals and the learning template of the developing ring is matched by adopting multi-scale template matching, the position RN of the developing ring is obtained, and the verification is carried out by the algorithms of similarity between a front frame and a back frame and a marked template, the anti-backward of the developing ring and the like.

Step 6: interpolation of DSA sequence image signals. Since one DSA sequence image signal corresponds to 6 OCT sequence image signals and the position of the development ring on the DSA sequence image signal needs to be filled up, the position of the development ring of the remaining 5 images needs to be inserted into the current DSA sequence image signal. The positions of the developing rings of the remaining 5 images are obtained by averaging the moving distances according to the position information of the previous and next frames. And when the vessel occlusion is met, an estimation point of the development ring needs to be given, and the similarity of the local characteristic point taking the Hessian enhanced multi-scale filtering vessel mask as a boundary and the learning template of the development ring is obtained. And finally, marking the obtained position of the development ring 6 × RN on the DSA sequence image, and synchronously playing OCT corresponding to the DSA sequence image.

The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is provided for purposes of illustration and understanding only, and is not intended to limit the application to the details which are set forth in order to provide a thorough understanding of the present application.

The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations should be considered as equivalents of the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims (11)

1. A method for DSA and OCT based fusion visualization, the method comprising:

DSA sequence images and OCT sequence images are synchronously and continuously acquired;

Time marking the acquired DSA sequence image and the OCT sequence image;

evenly distributing the obtained DSA sequence images to different GPU units according to the time sequence of the marks for processing, and calculating and marking the positions of the developing rings in the DSA sequence images;

and synchronously displaying the OCT sequence image and the DSA sequence image after marking the position of the developing ring on the same display according to the marked time.

2. The method of claim 1, wherein the processing is selected from one or more of Hessian enhanced multi-scale filtering, Gaussian Laplace point processing, and local feature point processing.

3. The method of claim 1, wherein the processing comprises Hessian enhanced multi-scale filtering, laplacian of gaussian processing, and local feature point processing.

4. The method of claim 1 wherein the processing results in a vessel path, local feature points, and laplacian of gaussian points for the DSA sequence images for calculating and labeling the location of the visualization ring in the DSA sequence images.

5. An apparatus based on DSA and OCT fusion visualization, the apparatus comprising:

An image acquisition unit for continuously acquiring DSA sequence images and OCT sequence images synchronously;

a time-stamping unit for time-stamping the acquired DSA sequence images and OCT sequence images;

the image processing unit is used for evenly distributing the obtained DSA sequence images to different GPU units according to the marked time sequence for processing, and calculating and marking the positions of the developing rings in the DSA sequence images;

and the image fusion unit is used for synchronously presenting the OCT sequence images and the DSA sequence images marked with the positions of the developing rings on the same display according to the marked time respectively.

6. The apparatus of claim 5, wherein the GPU unit comprises one or more of a Hessian enhanced multiscale filter processing sub-unit, a Gaussian Laplace processing sub-unit, and a local feature point processing sub-unit.

7. The apparatus of claim 5, wherein the GPU unit comprises a plurality of multi-scale filtering processing sub-units, a plurality of Gaussian Laplace point processing sub-units, and a plurality of local feature point processing sub-units.

8. The apparatus of claim 6, wherein the Hessian enhanced multi-scale filtering processing sub-unit is configured to compute a vessel path for obtaining DSA sequence images.

9. The apparatus of claim 6, wherein the Gaussian Laplace processing subunit is configured to calculate the Gaussian Laplace points for obtaining DSA sequence images.

10. The apparatus of claim 6, wherein the local feature point processing subunit is configured to compute local feature points for obtaining DSA sequence images.

11. An electronic device, characterized in that the electronic device comprises:

a processor; and

memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform the method of any of claims 1-4.

Images