JP2020163216A - Method of operating processor device - Google Patents

Abstract

To provide a method of operating a processor device.SOLUTION: A method of operating a processor device for displaying angiographic and intravascular representations of a blood vessel includes generating a set of optical coherence tomography image data in response to distance measurement of the blood vessel obtained during a pullback of a probe through the blood vessel using an optical coherence tomography system, where the set of optical coherence tomography image data comprising a plurality of cross-sectional images at a plurality of positions along the blood vessel.SELECTED DRAWING: Figure 1

Description

In part, the present invention relates generally to the field of imaging and data collection of vasculature and peripheral vasculature.

The intervening cardiologist employs a variety of diagnostic tools during the catheterization procedure to plan, guide, and evaluate therapy. Fluorescence fluoroscopy is generally used to perform angiographic imaging of blood vessels. Such vascular imaging is then used by physicians to diagnose, locate and treat vascular disease during interventions such as bypass surgery or stent placement. Intravascular imaging techniques such as optical coherence tomography (OCT), acoustic techniques such as intravascular ultrasound (IVUS) and other techniques also fluoresce to obtain high resolution data on the condition of the blood vessels of a given subject. It is a valuable tool that can be used in place of fluoroscopy or in combination with fluorescence fluoroscopy.

The coronary flow reserve ratio (FFR) can also be used to evaluate vessels during imaging and angiography. Intravascular OCT, IVUS and FFR are invasive catheter-based systems that collect optical, ultrasound and pressure data, respectively, from internal vessels or for specimens of interest. Angiography is a non-invasive radiography method that collects data from outside the body during injection of an X-ray opaque contrast medium.

Intravascular optical coherence tomography is a catheter-based imaging modality that uses light to look into the walls of coronary arteries and generate images for research. OCT can leverage coherence tomography, interference spectroscopy and microoptics to provide in vivo tomography of video rates in diseased blood vessels at micrometer-level resolution. By viewing subsurface structures at high resolution using fiber optic probes, OCT becomes particularly useful for minimally invasive imaging of internal tissues and organs. These levels of detail made possible with OCT allow clinicians to diagnose as well as monitor the progression of coronary artery disease.

Considering the complexity of the various techniques described above and the complexity of the datasets generated by each of the related techniques, co-registration (co-) between two image-based techniques such as OCT and angiography. It takes a lot of time to execute (registration). Therefore, there remains a problem regarding real-time registration of intravascular data and angiographic image data. Some registration techniques rely heavily on user interaction. Unfortunately, important users during the registration, such as the need to manually match the corresponding points in the image, the long wait time for the algorithm to return the registration, and the validation of the final result. This approach is difficult to implement in many clinical scenarios due to the heavy burden on the operator due to the dialogue with. In addition, other techniques use data from asynchronous or third-party controlled sources, which results in irregular time adjustments. In addition, contrast media such as dyes are used with some intravascular imaging modality that interferes with other non-invasive imaging modality, resulting in imaging artifacts and errors that interfere with registration between such modality. There is.

Therefore, there is a need to address one or more of the issues identified above related to intravascular imaging and angiographic imaging. Embodiments of the present invention address these and other issues.

One embodiment of the present invention relates to angiography and methods for alignment between two imaging modality such as OTC. One embodiment of the present invention relates to one or more methods for performing registration between angiographic and OCT images.

One embodiment of the present invention relates to a method for performing detection of a marker band stationary on a frame without using a contrast agent such as a dye and using a contrast agent. Further, one embodiment of the present invention provides tracking of the marker band as it moves through the lumen of a blood vessel, such that the marker band is tracked in a subsequent pullback frame. Includes tracking from a frame without contrast agent to a frame with contrast agent.

In one embodiment, the period for alignment between about 20 to 100 frames of the angiographic image frame and about 100 to about 1500 frames of the OCT image frame ranges from about 2 seconds to about 30 seconds. .. In one embodiment, the alignment of the angiographic image data with the OCT image data obtained during OCT pullback is co-registered in less than about 10 seconds. In one embodiment, the pullback of the data acquisition probe ranges from about 4 seconds to about 10 seconds. In one embodiment, the angiography frame is obtained in real time using a frame grabber. Frames of angiographic data are captured in synchronization with the OCT image data frame obtained as a result of pullback.

In one embodiment, the registration method aligns the OCT frame of the image data obtained during the pullback imaging with the angiographic data frame obtained during the pullback within an alignment period of about 3 seconds to about 5 seconds. Superimpose and register in.

In one embodiment, the present invention is configured to perform imaging during pullback of a frame grabber, a data acquisition probe having markers through a blood vessel, to generate time-stamped OCT image data for the blood vessel, an OCT system. For an image data processing system that includes one or more computing devices and a user interface, the frame grabber is configured to obtain a time stamped frame of angiography image data with respect to the blood vessel.

In one embodiment, video capture of angiographic image data occurs in the OCT system. In one embodiment, the user manually specifies a marker band on the angiographic image. In one embodiment, the designated marker band appears on the angiographic image without the use of contrast media. In one embodiment, the user interface includes a longitudinal OCT image panel, a cross-section OCT image panel, one or more controls, and an angiographic image panel. In one embodiment, the user interface comprises a register control or button that causes the computing device to execute one or more software modules configured to superimpose and register OCT and angiographic image data. In one embodiment, the time stamp is used to provide a primary match between the angiography frame and its corresponding OCT frame, which allows the position of the closest angiography frame to be located for each OCT frame. , And vice versa. In addition, time-stamped events such as pullback start and stop are also recorded to assist the registration process.

In one embodiment, a cursor or other identifier on the angiographic image indicates the location of an OCT catheter reference marker that coincides with the selected OCT pullback frame. In one embodiment, the cursor or other identifier can indicate the proximal and distal reference frames selected by the user, with which the MLA has been calculated, and can also indicate the average diameter of the blood vessel. .. The scrolling of the superimposed OCT and angiographic images can be controlled via the OCTL mode, i.e. the cursor on the angiographic frame as a remote controller or as part of the user interface.

In one embodiment, a filter kernel, such as a convolution matrix, is configured to perform row and column, as well as image processing to perform sensitization, sharpening, pattern identification, detection, tracking, and other image processing tasks. It is implemented as a matrix containing the elements. The filter kernel can be used in various pre-processing and other processing steps to perform image processing on angiographic image data or other image data.

In one embodiment, the invention relates to a processor-based method of displaying angiographic and intravascular representations of blood vessels. The method is: to generate a set of OCT image data containing multiple cross-sectional images at multiple locations along the blood vessel in response to vascular distance measurements using an optical coherence stromography system; multiple along the blood vessel. Generating a set of angiographic image data, including multiple two-dimensional images at the location of; as well as time stamps, relationships between time stamps, matching of features in OCT images with features in angiographic images, It also includes determining the centerline of the blood vessel and superimposing and registering the angiographic image and the OCT image based on one or more of the superposition registration of the OCT image and the angiographic image using the centerline.

In one aspect, the invention relates to a processor-based method of displaying angiographic and intravascular representations of blood vessels. The method is: to generate a set of optical coherent smography image data in response to a distance measurement of the vessel obtained during pulling back of the probe through the vessel using an optical coherence stromography system. The set of optical coherence stromography image data comprises generating a plurality of cross-sectional images at multiple positions along the vessel; during the pullback of the probe through the vessel using the optical coherence stromography system. The angiography system is used to generate a set of angiographic image data, wherein the set of angiographic image data includes a plurality of two-dimensional images obtained at different time points during the pullback. To display a first panel containing a first longitudinal view of the blood vessel generated using the OCT image data; and to include a frame of the angiographic image data identifying the blood vessel. The second panel comprises displaying with one or more points in the frame and a blood vessel centerline passing through the one or more points.

In one embodiment, the present invention further superimposes and registers the OCT image data and the angiographic data using the blood vessel centerline in order to generate continuous alignment of the markers to be tracked. The included and tracked markers are placed on an OCT data acquisition probe. In one embodiment, the invention presents the OCT image data and the blood vessel so that the frame identifier in the first longitudinal view is changed by selecting a point along the blood vessel centerline through the user interface. It further includes superimposing and registering the contrast data. In one embodiment, the present invention performs an iterative search using a pullback rate or pullback length to select candidates for the tracked marker based on the pullback length and / or the pullback rate. Further includes exclusion based on possible positions.

In one embodiment, the present invention creates said vessel centerline using multiple processing steps from the shortest path technique and Dijkstra algorithm. In one embodiment, the invention further comprises the step of removing a guide catheter image from one or more frames of the angiographic data using the superposition of luminance profiles. In one embodiment, the invention creates the blood vessel centerline using pathway information generated from one or more angiographic frames in the substantial absence of contrast medium. In one embodiment, the present invention 1 generates a confidence score for each of the detection of the angiography data and the optical coherence stormography data, and the superposition registration between the angiography data and the optical coherence stormography data. Including that further.

In one aspect, the invention is a method of detecting an intravascular probe marker: obtaining a first frame of angiographic image data that is substantially free of contrast agent and contains said intravascular probe marker; Obtaining a second frame of the angiographic image data, including contrast agent image data in the vicinity of the intravascular probe marker; and detecting the intravascular probe marker in the first frame and the second frame. including.

In one embodiment, the present invention applies an image processing transformation to the second frame to remove or modify features within the second frame; and increases the brightness of the plurality of pixels. The plurality of pixels further include an ascending step that includes a guidewire image within the second frame. In one embodiment, the invention further comprises the step of generating an average luminance value for a plurality of images and subtracting the average luminance from the first or second frame. In one embodiment, the present invention further comprises applying the bottom hat operator to the second frame and applying the morphology close operation.

In one embodiment, the present invention detects the intravascular probe marker: the first frame by applying the multiscale LoG (Laplatian of Gaussian) operator to the first frame and the second frame. Filtering candidate markers that include pixels in one frame and the second frame; and performing a non-maximum suppression process to identify small chunks with maximum values in the vicinity of the pixels.

In one embodiment, the invention further comprises applying Euclidean distance transformation to a binary image to generate a guidewire-based potential. This method involves applying a power exponent to the negative fractional power of the distance transformation to calculate the potential function. In one embodiment, the method further comprises determining a plurality of geodesic distances based on the guidewire-based potential using a high speed marching method.

In one embodiment, the present invention removes shading from the first frame and the second frame; increasing the contrast level of one of the first frame or the second frame. It further includes performing morphological image reconstruction for each marker candidate. In one embodiment, the present invention uses a hesian-based vascular filter to process multiple pullback frames; and template matching is used from either the first frame or the second frame. Further includes tracking the intravascular probe marker through the plurality of pullback frames to all the pullback frames. In one embodiment, the invention further comprises tracking said intravascular probe markers through multiple frames obtained during pullback using Viterbi dynamic programming.

In one aspect, the present invention relates to a processor-based method of superimposing and registering angiographic image data and intravascular image data obtained during pullback through a blood vessel. The method is: storing multiple frames of optical coherence stromography data in memory; storing multiple frames of the angiographic image data in memory; so that one or more shadows are substantially reduced. To process the plurality of frames of the angiographic image data; to detect the catheter in the plurality of frames of the angiographic image data; to detect the detected angiographic image data in the plurality of frames. Removing the catheter; generating a vascular centerline for the multiple frames of the angiographic image data; detecting probe markers in the multiple frames of the angiographic image data; one or more of the blood vessels Tracking the position of the probe marker along the centerline; and using the traced positions of the plurality of frames of the angiographic image data and the plurality of frames of the optical coherent stromography data. Including superimposing registration.

In one embodiment, the invention further comprises generating a score indicating the level of confidence in overlay registration between the frame with the angiographic image data and the frame with the optical coherence tomography data. In one embodiment, the invention performs the step of removing the detected catheter using a superposition of brightness profiles generated based on sampling of the area of the detected catheter.

In one embodiment, in the present invention, the step of superimposing and registering the plurality of frames of the angiographic image data and the plurality of frames of the optical coherence tomography data is performed using a computing device. The registration table includes angiographic image frames, a plurality of OCT time stamps per frame, a plurality of angiographic time stamps per frame, and an optical coherence tomography image frame, which comprises generating a table. In one embodiment, the invention further comprises displaying a stent representation on OCT and angiographic images within the user interface using the registration table and computing device.

In one embodiment, the present invention uses the registration table and a user interface configured to display the side branches to identify the side branches in one or more of the OCT images and the angiographic image. Including that further. In one embodiment, the present invention adjusts the pullback speed change by setting frame spacing of OCT data based on the registration table and displays a longitudinal view on the user interface based on the spacing. Including that further.

The figure is not necessarily proportional to the actual size, and overall, it emphasizes the policy for illustration. The figures should be considered exemplary in all respects and are not intended to limit the invention and the scope of the invention is defined solely by the claims.

It is a schematic diagram of the angiography method and the intravascular imaging and data collection system according to the exemplary embodiment of the present invention. FIG. 5 is a schematic representation of the site of interest for a subject and the features of a catheter-based data acquisition probe according to an exemplary embodiment of the invention. FIG. 6 is a schematic representation of a catheter-based data acquisition probe comprising a marker according to an exemplary embodiment of the invention. Images of a graphical user interface suitable for controlling or studying the data and images generated by the methods and software modules described herein according to the system of FIG. 1 and / or exemplary embodiments of the invention. It is a figure which shows. Another graphical user interface suitable for controlling or examining the data and images generated by the methods and software modules described herein according to the system of FIG. 1 and / or exemplary embodiments of the invention. It is a figure which shows the image. To FIG. 6 is a schematic diagram showing a processing step or processing step suitable for processing and using image data according to an exemplary embodiment of the present invention. FIG. 6 is a flow chart relating to some typical pretreatment steps or pretreatment steps according to embodiments of the present invention. It is a flowchart related to some typical blood vessel center line generation steps or blood vessel center line generation steps according to an exemplary embodiment of the present invention. FIG. 6 is a flow chart relating to some typical marker detection and registration steps or steps according to an exemplary embodiment of the invention. FIG. 5 is a diagram of a typical bottom hat filter configured to improve the image quality of a small mass or other pixel region in an angiographic image that may be a marker from a probe according to an exemplary embodiment of the invention. FIG. 6 is a diagram of a typical nodule corresponding to a subset of pixels from the angiographic region where image quality has been improved by applying the filter from FIG. 6A according to an exemplary embodiment of the invention. FIG. 5 is a diagram of the original angiographic image without contrast medium prior to wire detection according to an exemplary embodiment of the invention. It is a figure of the typical angiography image which shows the result of the guide wire detection on the frame which does not use a contrast medium by the exemplary embodiment of this invention. It is a figure of the typical angiography image which shows the result of the improvement of the image quality of a guide wire after application of a bottom hat operator by an exemplary embodiment of the present invention. It is a figure of a typical angiographic image which shows the result of the improvement of the guide wire image quality after application of the Hesian operator having a scale value 1 according to the exemplary embodiment of the present invention. FIG. 5 is a diagram of typical potential generated based on a guide wire suitable for use with a fast marching process (FMM) process according to an exemplary embodiment of the invention. FIG. 6 is a diagram of a distance map generated using the FMM process according to an exemplary embodiment of the present invention. FIG. 5 is a diagram of an original angiographic image using a contrast agent prior to catheter removal and shadow removal according to an exemplary embodiment of the invention. FIG. 5 is a diagram of a typical angiographic image showing the results of catheter removal and shadow removal according to an exemplary embodiment of the present invention. FIG. 5 is a diagram of an original angiographic image using a contrast agent prior to shading removal according to an exemplary embodiment of the present invention. It is a figure of the typical angiography image which shows the result of catheter removal and shadow removal by an exemplary embodiment of this invention. It is a figure of the original angiography image using a contrast agent before the guide wire detection by the exemplary embodiment of the present invention. It is a figure of a typical angiographic image which shows the result of the guide wire detection with respect to the original image of FIG. To FIG. 5 illustrates the application of different software-based image processing steps to generate graphs based on frames of angiographic image data according to an exemplary embodiment of the invention. To FIG. 6 shows a variety of best paths detected through the graph generated in FIG. 7F, based on a graph search algorithm according to an exemplary embodiment of the invention. To It is a figure which shows various image processing steps related to catheter detection and removal by an exemplary embodiment of this invention. To It is a figure which shows the influence of the contrast medium on the typical model of a catheter, and its brightness profile by an exemplary embodiment of the present invention. To FIG. 5 illustrates the characteristics of the use of a superposition-based catheter removal method according to an exemplary embodiment of the present invention. It is a schematic diagram of various software and hardware components suitable for processing intravascular image data and angiographic image data according to an exemplary embodiment of the present invention. To It is a schematic diagram of a typical OCT data table and angiography data table for pullback according to an exemplary embodiment of the present invention. FIG. 6 is a schematic representation of typical angiographic data for pullback according to an exemplary embodiment of the present invention. It is a schematic diagram of a typical registration table according to an exemplary embodiment of the present invention.

The following description refers to the accompanying drawings showing specific embodiments of the present invention. Other embodiments may be considered and modifications may be made to the above embodiments without departing from the spirit and scope of the invention. Therefore, the following detailed description is not intended to limit the invention and the scope of the invention is defined by the claims.

As described above, there are issues related to vascular diagnostic systems and peripheral vascular diagnostic systems, such as issues related to implementing registrations for multiple imaging techniques such as angiography, OCT and IVUS. In part, the invention is a diverse system, diverse, for collecting data from subjects and using it in catheter laboratories or other facilities to help improve one or more of these limitations. System components and methods. The data collected typically relates to the patient's cardiovascular or peripheral vascular system and may include imaging data, pressure data, heart rate and other types of data as described herein. ..

Further, in one embodiment, the image data is collected using an optical coherence tomography probe and other related OCT components. In one embodiment, image data is collected using an IVUS probe and other related IVUS components. Further, in one embodiment, pressure data is collected using an FFR probe and other related FFR components. In a further embodiment, EKG, heart rate and other subject data are collected using electrodes and other related components.

In addition, some embodiments of the present invention are suitable for dealing with multiple imaging modality. Accordingly, in part, the present invention relates to multimodal diagnostic systems and components of multimodal diagnostic systems configured to superimpose and register one or more of OCT, IVUS, FFR and angiography. The OCT data and image processing results can be used to improve the processing of frames of angiographic images by providing inputs to a software module dedicated to angiography.

The IVUS imaging feature can also be incorporated into a data acquisition probe used in conjunction with collecting angiographic data in one embodiment. In addition, FFR pressure measurements can also be performed using suitable pressure transducers and probes. In one embodiment, the FFR data acquisition probe or transducer can include a wireless transmitter, which can utilize the wireless receiver to receive FFR data and communicate the FFR data to a server. Comparison and registration of OCT and / or IVUS images with angiographic images is accomplished by interoperating the system with an angiographic device or hospital data network in which the angiographic data is stored.

In one embodiment, a user, such as a clinician, provides a relevant user interface for displaying an image of a longitudinal section of a subject's blood vessel from top to bottom, or a section substantially parallel to the longitudinal axis of the vessel. Interact with your workstation or server. The registration process may include various steps as well as image processing and feature detection software modules. In one embodiment, the user or system activates intravascular imaging during acquisition of angiographic images. Blood vessels and imaging catheters that are imaged within the vessel can be displayed as part of a graphical user interface. The boundaries of the vascular lumen can be identified and associated with each intravascular and angiographic image, which allows the same vascular segment to be maintained in different views.

Since the imaging catheter is introduced by a guidewire, the guidewire can be used as an anchor pathway and also provides directional information such as which endpoints are distal and which endpoints are proximal within the relevant imaging segment. Can be used to do. In one embodiment, the guide catheter slides along the guide wire to position the probe tip with one or more imaging devices within the blood vessel. In one embodiment, the angiographic image data is processed to remove the guide catheter from the image after identifying the guide catheter.

In one embodiment, one or more software modules are used to generate and track a blood vessel centerline for a given frame of angiographic data. In one embodiment, the blood vessel centerline, also referred to herein as the centerline, is angiography for an optical sensor or acoustic sensor introduced during angiography data acquisition or a marker band associated with another imaging or data acquisition sensor. A model or simulation generated based on an iterative evaluation of each candidate subset of a frame of data. In one embodiment, a dynamic program software module, such as a software module that implements one or more steps of the Viterbi algorithm, can be used to track the marker band. In one embodiment, the Viterbi algorithm is used for X-ray opaque marker tracking. Centerline creation and tracking is usually processed by other algorithms or combinations of algorithms. In one embodiment, the vessel centerline is generated by a combination of algorithms or processes for finding the shortest path between two distant points, such as a fast marching algorithm on a Hessian image and a modified Dijkstra algorithm.

FIG. 1 is a system 5 containing various data collection subsystems suitable for collecting data, detecting the characteristics of subject 10, detecting the condition of subject 10, or making other diagnoses of subject 10. Is shown. In one embodiment, the subject is placed on a suitable support 12, such as a table bed to a chair or other suitable support. Subject 10 is typically a human or another animal having a particular site of interest 25.

In part, embodiments of the present invention relate to the registration of an intravascular image or data obtained by an imaging catheter across a blood vessel with an external angiographic image of that blood vessel taken upon crossing the catheter. .. FIG. 2A shows a general schematic view in which the target portion is enlarged.

In a typical OCT data acquisition procedure, the catheter is inserted through a guide wire to advance the probe to the distal end of the target vessel. The probe 30 can include one or more markers. In one embodiment, the marker placed on the probe 30 is an X-ray opaque marker band. The torque wire 110 that partially surrounds the optical fiber 33 is also shown in FIG. 2A. The probe 30 is placed in the lumen 50 of the blood vessel. The guide wire 115 is also shown in the lumen 50. The guide wire 115 is used to position the probe tip and torque wire arranged in the catheter with respect to the lumen 50. The light λ from the tip of the probe is shown directed towards the wall of the blood vessel having the lumen 50.

Additional details relating to a typical intravascular data acquisition probe are shown in FIG. 2B. As shown in FIG. 2B, the intravascular data acquisition probe 120, such as an OCT, IVUS, FFR or other data acquisition probe, is configured to direct light as indicated by a dotted line as part of the probe tip. The optical fiber 33 is included. A sheath, such as the polymer sheath 125, surrounds the probe tip containing a beam-directing element such as a lens or reflector. The light λ that exits the beam-directed device along the dotted line is shown. The optical fiber 33 is arranged on the torque wire 110, which is also arranged inside the sheath 120. The optical fiber 33 is coupled to the PIU 35 as shown.

As shown in FIG. 2B, a marker such as an X-ray opaque marker or a marker band 130 is a part of the data acquisition probe 120. The markers are detectable by the angiography system and can be tracked according to the movement of the markers over the frame of the angiography data. As shown, the distance from the right end of the torque wire 127 to the beam directional element such as a lens or reflector is L1.

Further, the distance from the right end of the torque wire 127 to the right end of the marker 130 is L2. The thickness of the marker 130 is L3. The distance from the distal edge of the marker 130 (shown as the left side of the marker) to the torque wire 127 is L3 + L2. In one embodiment, L1 ranges from about 0.3 mm to about 0.9 mm. In one embodiment, L2 is about 0.6 mm to about 1.4 mm. In one embodiment, L3 is about. It ranges from 5 mm to about 1.5 mm.

In one embodiment, a data acquisition probe such as an OCT probe can include three X-ray opaque marker bands. The distal marker, located at the distal end of the probe, remains stationary throughout acquisition. The central marker is located in the imaging core resident 27 mm from the distal marker before pullback. The proximal marker is located 50 mm from the imaging core and this distance remains fixed during pullback.

During pullback, a processor-based system, such as system 22 of FIG. 1, records a live algorithm and displays the blood vessel with a contrast agent and a marker or probe. Typically, the markers are visible at most times. Optionally, some frames are recorded without contrast medium, as shown in FIG. 6C, which makes the guide wires and markers clearly visible. This provides good indication of the pullback path through the blood vessel.

FIG. 3A shows a typical graphical user interface configured to display multiple panels. The graphical user interface can be implemented using a computing device such as server 50 or workstation 87 or another suitable computing device. The upper right panel shows frame angiographic image data. As shown in the image, the upper point, i.e. the section of the blood vessel located between the cursor 3 and the lower point, i.e. the cursor 4, was imaged using intravascular imaging techniques as part of the pullback. Specifically, angiographic data was obtained during OCT pullback.

A typical cross section of an artery is shown in the upper left panel. In the upper left OCT image, the side branch is shown on the right side of the cross section of the data acquisition probe. The lower panel, which substantially spans the user interface, includes a point, a longitudinal image of the blood vessel placed between the distal and proximal endpoints shown in the angiographic image by cursors 3, 4. The magnifying glass icon can be used to zoom in or out on either the OCT image or the angiographic image. The pencil icon can be used to make measurements on either an OCT image or an angiographic image. Angiographic frames of data can be reproduced as video in the upper right panel by using replay, review or forward video user interface controls.

In the upper left OCT image, the diagonal axis indicates the cut plane used to display the longitudinal mode in the lower panel. The longitudinal mode is generated by combining multiple cross-sections as shown in the upper left quadrant interface. In L mode, the triangle 4'is configured to indicate the bookmarked location of the desired frame. The longitudinal view, or L mode, can be reviewed, played or forwarded using the L mode user interface to advance or review or display as a moving image, while the vertical lines shown in the L mode are the cross-section OCT images. Corresponds to the cross-sectional slice of the blood vessel shown in. By selecting the play and review buttons in L mode, the corresponding vertical lines move forward or backward so that the vertical lines move in L mode on the lower panel as different cross sections are shown in the upper OCT image. ..

In one embodiment, the computing device used to display and execute the user interface of FIGS. 3A and 3B includes a memory area containing image data such as a cross-sectional view of a blood vessel. The computing device may include machine-readable media or other memory, including one or more software modules for displaying a graphical user interface such as interface 142. The interface may include multiple panels, menus or other viewable areas. These panels or areas can be displayed on one or more monitors, such as the display 82. The computing device can exchange data, such as image data, with the monitor 23 using a network that may include one or more wired, optical, wireless or other data exchange connections.

The controller or input device 127 can perform wired communication, optical communication or other communication with other devices or systems shown on the network 120. The controller can be used to send command signals to the computing system 100 running interface 142. The interface 142 can display data from the system 5 of FIG. 1, the system 300 of FIG. 14, or the data sources, systems or software modules described herein. Interface 142 may include one or more menus and other sections that change in response to control signals from controller 127. Controller 127 may include a processor or a suitable programmable ASIC. The control signal can be transmitted over network 120 or over another connection.

The computing device 100 is a server computer, client user computer, personal computer (PC), laptop computer, tablet PC, desktop computer, control system, microprocessor, or (specifying the action that the computing device will take). It may include any computing device capable of executing a set of instructions (sequentially or otherwise). Further, although a single computing device is shown, the term "computing device" is one of instructions to execute any one or more of the features or methods of software, such as interface 142. It shall also be construed as including any collection of computing devices that perform one set (or multiple sets) individually or jointly.

FIG. 3B shows a representation of the graphical user interface 142. Interface 142 includes a plurality of panels. As illustrated, in one embodiment there are four main panels 150, 155, 160 and 165. These include an auxiliary display panel 150 showing angiographic data in this embodiment, a cross-sectional view or B-mode display panel 155, a lumen profile panel 160, and an L-mode display panel 165. In one embodiment, the interface also includes a plurality of toolbars B1, B2 and B3. On the panel 150, three markers are shown as a cross overlaid on the angiographic image. The top marker corresponds to the proximal reference frame shown on panel 160. The central marker corresponds to the smallest lumen region frame shown on panel 160 or the active OCT frame shown on panel 155. The lower marker corresponds to the distal reference frame shown on panel 160. The angiographic and OCT frames that can be displayed using the interface in FIGS. 3A and 3B can be processed and superposed as outlined herein. In one embodiment, the computing device accesses the registration table to display the frames that are superimposed and registered.

FIG. 3B shows a minimum lumen region plot as part of the lumen profile for blood vessels imaged during pullback of the OCT probe on panel 160. The D and P arrows indicate the proximal and distal directions along the imaged blood vessel. The cut plane, shown as a line with sections L1 and L2, is shown in the cross section of panel 155 and is also shown by sections L1 and L2 of the L-mode panel 165. The information bar B1, the measurement bar B2, and the menu bar B3 are shown.

As shown, the distance between blood vessels such as arteries can be measured with respect to two endpoints, as shown by the typical measurement distance of 119.88 mm. Further, the average diameter can be shown at the end of each selected reference frame for measuring blood vessels, eg, average diameter values of 39.2 mm and 44.2 mm in the distal reference frame and the proximal reference frame, respectively. Is. As shown, the MLA is about 22 mm ² . In the MLA frame, the average vessel diameter is about 2.11 mm and the percent diameter stenosis is 25.4% relative to the average diameter of the proximal and distal reference frames.

All three images shown in the user interface of FIGS. 3A and 3B can indicate movement along the line between the ends of the blood vessels in the angiographic image by moving points in the OCT image that are synchronized with the frame. It is registered in a superposed manner. Thus, as one image moves along the vascular segment, the movement along the centerline shown in the angiographic image is also indicated by the cross-section OCT image, the L-mode OCT image, or both movement frame identifiers.

Initially, the proximal marker band may reside near the coronary ostium of the coronary branch, which is blocked by contrast turbidity during pullback. The catheter is pulled back at a constant rate through the blood vessel. The markers do not move at a constant rate in the angiographic image plane (2D) because the multiple vessel segments exhibit different shrinkage along the pullback. In addition, due to the movement of the heart, the markers exhibit a unique "sawing" movement with respect to the anatomy of the blood vessels. At some of the positions of the angiographic frame, the marker band appears blurry / blurred due to the fast pullback movement combined with the fast heart movement. The contrast of markers in the local neighborhood may be low. Other features such as reduced bifurcations, background structure, etc. can be mistaken for any of the marker bands.

The data acquisition system 5 includes a minimally invasive imaging system such as nuclear magnetic resonance, X-rays, computed tomography, or other suitable minimally invasive imaging techniques. As a non-limiting example of such a minimally invasive imaging system, an angiography system 20 such as one suitable for producing cine is shown, as illustrated. The angiography system 20 may include a fluorescence perspective system. The angiography system 20 is configured to image the subject 10 in a minimally invasive manner, whereby a frame of angiography data, typically in the form of a frame of image data, is pulled back using the probe 30. The blood vessels at site 25 of subject 10 are imaged using angiography in one or more imaging techniques, such as OCT or IVUS, while being generated.

The angiography system 20 communicates with the angiography data storage and image management system 22, which in one embodiment can be implemented as a workstation or server. In one embodiment, data processing related to the angiographic signal collected is performed directly on the detector of the angiography system 20. The image from the system 20 is stored and managed by the angiography data storage and the image management 22. In one embodiment, system server 50 or workstation 87 handles the functionality of system 22. In one embodiment, the entire system 20 produces electromagnetic radiation such as X-rays. The system 20 also receives such radiation after passing through subject 10. The data processing system 22 then uses the signal from the angiography system 20 to image one or more sites of the subject 10 including the site 25.

As shown in this particular example, the site of interest 25 is a subset of the vascular system, such as a particular blood vessel, or a peripheral vascular system. This can be imaged using OCT. The catheter-based data acquisition probe 30 is introduced into the subject 10 and placed in the lumen of a particular blood vessel, such as a coronary artery. The probe 30 collects various types of data, such as OCT probes, FFR probes, IVUS probes, probes that combine two or more of the above features, and other probes suitable for imaging inside blood vessels. It may be a probe. The probe 30 typically includes a probe tip, one or more X-ray opaque markers, an optical fiber and a torque wire. Further, the probe tip includes one or more data acquisition subsystems such as an optical beam directional device, an acoustic beam directional device, a pressure detector sensor, another transducer or detector, and a combination of the above.

In the case of a probe including an optical beam directional device, the optical fiber 33 is in optical communication with a probe with the beam directional device. The torque wire defines a perforation in which the optical fiber is located. In FIG. 1, the optical fiber 33 is shown without the torque wire surrounding it. Further, the probe 30 also includes a sheath such as a polymer sheath (not shown) that forms a portion of the catheter. In the case of the OCT system, the optical fiber 33, which is part of the sample arm of the interferometer, is optically coupled to the patient interface unit (PIU) 35 as shown.

The patient interface unit 35 includes a probe connector suitable for receiving the end of the probe 30 and being optically coupled to the end of the probe 30. Typically, the data acquisition probe 30 is disposable. The PIU35 includes suitable junctions and elements based on the type of data acquisition probe used. For example, a combination of OCT and IVUS data acquisition probes requires OCT and IVUS PIU. The PIU 35 also typically includes a motor suitable for pulling back the torque wire, sheath and fiber optic 33 disposed therein as part of the pulling procedure. In addition to being pulled back, the probe tip is typically rotated by the PIU35. In this way, the blood vessels of subject 10 can be imaged longitudinally or through a cross section. The probe 30 can also be used to measure certain parameters such as FFR or other pressure measurements.

The PIU 35 is then connected to one or more intravascular data acquisition systems 40. The intravascular data acquisition system 40 may be an OCT system, an IVUS system, another imaging system and a combination of the above. For example, the system 40 when the probe 30 is an OCT probe may include an interferometer sample arm, an interferometer reference arm, a photodiode, a control system and a patient interface unit. Similarly, as another example, in the case of an IVUS system, the intravascular data acquisition system 40 may include an ultrasonic signal generation and processing circuit, a noise filter, a rotatable junction, a motor, and an interface unit. In one embodiment, the data acquisition system 40 and the angiography system 20 have a shared clock signal or other timing signal configured to synchronize the angiography video frame time stamp with the OCT image frame time stamp.

In addition to the invasive and non-invasive image data acquisition systems and devices of FIG. 1, various other types of data can be collected with respect to the subject's site 25 and other objective parameters of the subject. For example, the data acquisition probe 30 may include one or more pressure sensors, such as pressure wires. Pressure wires can be used without OCT or ultrasonic components. Pressure measurements are obtained along the segment of blood vessels at site 25 of subject 10.

Such measurements can be relayed via either a wired or wireless connection. As shown in the blood flow reserve ratio data acquisition system 45, the wireless transceiver 47 receives pressure measurements from the probe 30 and sends them to the system to generate FFR measurements, or to the vessel being measured. It is configured to send to more locations along. One or more displays 82 can also be used to show angiographic frames of data, OCT frames, user interfaces for OCT, and angiographic data, as well as other controls and features of interest.

Intravascular image data such as frames of intravascular data generated using the data collection probe 30 can be sent to the data collection processing system 40 coupled to the probe via the PIU 35. Minimally invasive image data generated using the angiography system 22 can be transmitted, stored, and stored, and one or more to one or more servers or workstations such as the registration server 50, workstation 87, etc. Can be processed by the server or workstation of. A video frame grabber device 55, such as a computer board, configured to capture angiographic image data from the system 22 can be used in a variety of embodiments.

In one embodiment, the server 50 includes one or more registration software modules 60 stored in memory 70 and executed by processor 80. The server 50 may include other typical components for a processor-based computing server. One or more databases, such as database 90, are image data generated by one or more of the systems, devices or components shown in FIG. 1, one of the systems, devices or components shown in FIG. It can be configured to receive subject parameters, and other information generated, received, or transferred to database 90 by one or more. The database 90 is shown as being stored in memory on workstation 87 while being connected to server 50, which is just one typical configuration. For example, the software module 60 may run on the processor of workstation 87 and the database 90 may be located in the memory of the server 50. Devices or systems used to run various software modules are provided as examples. In various embodiments, the hardware and software described herein can be used to obtain frames of image data, process the image data, and align the image data.

Alternatively, as referred to herein, the software module 60 is intended to facilitate the registration of various types of image data by other software-based components 60, or in other ways. Software such as preprocessing software, variants, matrices and other software-based components used to process image data or respond to patient triggers to perform the ration may be included.

The database 90 can be configured to receive and store angiographic image data 92, such as image data generated by the angiography system 20 and obtained by the frame grabber 55 server 50. The database 90 can be configured to receive and store OCT image data 95, such as image data generated by the OCT system 40 and obtained by the frame grabber 55 server 50. The database 90 can be configured to receive and store an angiography table such as the table shown in FIG. 14 and a registration table such as the table shown in FIG.

Further, the subject 10 can be electrically coupled to one or more monitors, such as the monitor 49, via one or more electrodes. The monitor 49 may include, but is not limited to, an electrocardiogram monitor configured to generate data showing various states of the subject, such as, but not limited to, cardiac contraction and diastole. Knowing the stage of the heart can be used to assist in tracking the vascular centerline, as the geometry of the heart, including the coronary arteries, is approximately identical at a particular stage of the heart, even in different cardiac cycles.

Thus, if the angiographic data spans several cardiac cycles, primary matching of the vascular centerline at the same cardiac stage may assist in tracking the centerline throughout the pullback. In addition, since most of the cardiac movement occurs during systole, vascular movement is expected to be higher near systole and decay towards diastole. This provides data as an indication of the amount of expected movement between consecutive angiographic frames in one or more software modules. Knowledge of expected movements can be used by one or more software modules to improve the quality of tracking and the quality of vascular centerlines by enabling adaptive constraints based on expected movements.

The use of arrowheads to indicate directionality or lack of directionality in a given figure is not intended to limit or require the direction in which information may flow. For example, in the case of a given connector such as an arrow and a line shown to connect the elements shown in FIG. 1, the information is in one or more directions, or one, as appropriate for a given embodiment. It may flow only in the direction. The connection may include a variety of suitable data transmission connections such as optical connections, wired connections, power connections, wireless connections or electrical connections.

Further, although the FFR data acquisition system 45 is shown to have an appropriate wireless system 47 for transmitting and receiving information wirelessly, the other systems and components shown in FIG. 1 also include wireless systems such as the system 47. It may include, and in one embodiment information can be sent and received wirelessly.

One or more software modules can be used to process frames of angiography data received from angiography systems such as system 22 shown in FIG. A variety of software modules that may include, without limitation, software, components of software, or one or more steps of software-based methods or methods performed by a processor can be used in certain embodiments of the present invention.

Examples of such software modules are video processing software module, preprocessing software module, image file size reduction software module, catheter removal software module, shading removal software module, vascular image quality improvement software module, small mass image quality improvement software module, LoG filter or Deformation software module, guide wire detection software module, anatomical structure feature detection software module, static marker detection software module, background subtraction module, Frangi vascular software module, image brightness sampling module, moving marker software detection module, Iterative centerline test software module, background subtraction software module, morphological close operation software module, feature tracking software module, catheter detection software module, bottom hat filter software module, path detection software module, dixtra software module, Vitabi software Modules, software modules based on high-speed marching methods, vascular centerline generation software modules, vascular centerline tracking module software modules, Hessian software modules, brightness sampling software modules, image brightness superimposition software modules, and others described herein. Appropriate software modules may be included without limitation. The software module 60 shown in FIG. 1 may include one or more of the software modules described above and other software modules described herein.

Image Data Processing Features and Typical Embodiments As shown in FIGS. 4A and 4B, various processing steps, processing steps or processing software modules are generalized to use intravascular imaging techniques such as OCT, IVUS, or others. Provides a high-level summary of the angiographic image data and the process of superimposing and registering the resulting angiographic image data. In one embodiment, frames of angiographic data are captured on an OCT or IVUS server or workstation using a frame grabber or other data capture device. Capturing images from both imaging modalities in real time ensures accurate time stamping of the two sources against each other. It is essentially not possible to calibrate the DICOM angiography data acquisition time to match the timing of the OCT data. For example, a video software module can be controlled via a user interface to obtain individual frames of similarly time-stamped angiographic data and present the angiographic video to a memorable frame grabber. In one embodiment, the OCT and angiographic data are run in parallel on the same computer and are therefore date stamped by two respective processes that share the same time reference.

If the angiographic data frames are cached or otherwise stored, each stored frame can be modified during the pretreatment phase. Various matrices such as convolution matrix, hesian, etc. can be applied pixel by pixel to change the brightness, remove a given angiographic image frame, or make other modifications. As described herein, the pretreatment step effectively enhances, modifies or removes the features of the angiographic image to enhance the accuracy, speed, success rate and other properties of the subsequent processing steps. ..

As shown in FIG. 4A, various software-based processing steps 140 are shown. Initially, one or more frames of the angiographic image are processed during the pretreatment step 140a prior to the various detection and tracking steps, and another imaging technique such as OCT, IVUS, etc. and combinations thereof. Supports superimposition registration of other image data obtained by using and the related frame. The next step is step 140b of determining or calculating the blood vessel centerline. As shown in the user interface of FIG. 3, the vessel centerline is generated by one or more software modules and superimposed on the angiographic image or displayed in other fashion.

In one embodiment, the centerline represents the locus of a probe such as the data acquisition probe 30 of FIG. 1 passing through a blood vessel imaged during pullback. In one embodiment, the centerline is also referred to as a trace. Another step is the detection of marker bands on the angiographic frame 140c. In one embodiment, the final step is the registration step. These steps and the other steps and methods described herein can be performed in different orders, interactively, in parallel, in succession, or in combination. Additional steps and steps can also be added before, after or between predetermined steps or steps. Additional examples of typical steps and steps enumerating additional details are shown in FIGS. 4B and 5A-5C.

As shown in FIG. 4B, various software-based processing steps or processing steps 145 are shown, including additional details regarding the processing steps or processing steps shown in FIG. 4A. First, 150a, where pretreatment of the angiographic frame is performed. Detection of the guide wire on the frame without contrast medium is performed as shown in FIG. 6D 150c. FIG. 6N is a typical angiographic image showing the result of guide wire detection. As shown in FIG. 6N, the distal portion of the guide wire is detected.

Next, in one embodiment, the generation of the blood vessel centerline in one frame is performed 150e. In one embodiment, user input, such as the selection of a guidewire endpoint in the lumen that is imaged through the user interface, is stored as an endpoint selected by the user, which is also referred to as a hint point. Such hint points can be used to generate a vessel centerline on one frame such that a trace between the hint point and the peripheral site is generated for the relevant frame of the angiographic data. In one embodiment, the relevant frame is obtained without placing a contrast agent in the blood vessel.

Again, referring to FIG. 4B, 150f, where tracking of the vessel centerline along the angiographic frame is performed. In one embodiment, such tracking of the vessel centerline is performed on all or substantially all of the angiographic frames obtained during pullback. Tracking and / or marker detection of X-ray opaque markers in the angiographic frame is performed 150 h. In one embodiment, a Viterbi algorithm is used to perform marker tracking. 150j where superposition registration of the OCT image and the angiographic image is executed. 150 liters for which confidence score / figure of merit generation is performed.

Confidence score / figure of merit (FOM) generation is performed using one or more software modules 150 liters. In one embodiment, the confidence score or (FOM) provides, for example, a color code on an X-ray image or OCT image indicating a region of OCT pullback with high or low confidence that overlay registration has been performed. This is provided to the user by a graphic display on the computer monitor. The unreliable region may be indicated by red strips or bars, for example, on radiographs near the vascular segment where low FOM was obtained. The FOM / score reflects the confidence criteria of the returned results. The score is in the range [0,1], where 0 reflects the lowest confidence and 1 reflects the highest confidence. The FOM threshold can be selected to demarcate the boundary between high-reliability and low-reliability registration results. This threshold can be chosen to give the desired sensitivity and specificity for identifying high error locations by generating a receiver operating characteristic (ROC). If a given pullback yields a low FOM value for most of the frames and therefore the overall quality of the registration is questionable, the registration result may not be visible to the user.

The FOM determination is based on the quality of the detected lumps (contrast or brightness of the detected lumps compared to the contrast or brightness in the immediate vicinity, shape, size, etc.), (based on pullback rate, frame rate calculation). Distance from the nominally expected position of the detected lumps, the number of lump candidates detected in the same neighborhood (the more candidates, the lower the FOM), and the brightness-based z-score, of the Viterbi algorithm. A scoring process based on the overall score (how well the overall set of detected nodules represents pullback), and one or more factors such as other factors and criteria. In one embodiment, a weighted average containing one or more of the parameters listed herein can be used to generate the FOM or score.

The various steps and steps as shown in FIGS. 4A and 4B and as described herein can be performed automatically, in whole or in part, in various embodiments. Additional details relating to some particular examples, such as those relating to FIGS. 5A-5C, among some of the steps and methods of FIGS. 4A and 4B, are described herein. For example, FIG. 5A shows a flowchart relating to some typical pretreatment steps or pretreatment steps.

Typical Angiographic Image Data Preprocessing Embodiments Partly as shown in FIGS. 4A and 4B, and as described herein, the present invention is partly a collection of angiographic data. Includes one or more pre-processing steps, pre-processing software modules, and related methods for the frame. In one embodiment, image preprocessing is performed frame by frame with respect to frames of angiographic data, such as the data generated by system 20 of FIG. In the pretreatment stage, for example, multiscale LoG (Laplacian of Gaussian), detection of anatomical structural features, skeleton generation, angiographic image size reduction, background subtraction, bottom hat filter, etc. are applied to improve vascular image quality, catheter. It may include, without limitation, methods, steps and software components suitable for performing removal, shading, cardiac shading, nodule image quality enhancement, as well as other components.

Diverse matrices such as Hessian and other types of filters and masks, frames of angiographic data, trace markers, generate centerlines, and are superimposed and registered with OCT, IVUS, or other images or data. It can be applied to improve the image quality of frames of angiographic data before being exposed to additional processing for the purpose. One or more image processing steps can be used, for example, to preprocess frames of angiographic data received from angiographic systems such as system 22 or server 50 or workstations and 87 shown in FIG.

FIG. 5A shows a process flow 160 associated with some additional specific typical pretreatment steps or pretreatment steps. As illustrated, angiographic images can be processed in parallel at various stages. In one embodiment, LoG filtering is performed on multiple scales 160a. Each scale matches the size of the elements in the image acted on by the filter. LoG multiscale-based filters can, in one embodiment, be used to improve the image quality of the small chunks corresponding to the movement markers on the imaging probe. Different scales are used for markers of different sizes. In one embodiment, the LoG operator is calculated on several scales because it is sensitive to small chunks of different sizes and less sensitive to noise. An example of a LoG filter is shown in FIG. 6A. An example of a small mass is shown in FIG. 6B as corresponding to a marker (a set of pixels from an angiographic image) whose image quality is improved by applying LoG of FIG. 6A as part of the image quality improvement of the imaging processing software. In one embodiment, background subtraction is performed to reduce the effect of static features based on the average of several frames of the angiographic image.

Further, in one embodiment, a bottom hat filter or bottom hat conversion 160c can be applied to the angiographic data to enhance the visibility of the guide wire in the image. In one embodiment, the bottom hat filter is configured to eliminate features larger than the size of a particular structural element in a given angiographic map, such as a diaphragm, skeletal features, etc. An example of a bottom hat filter or bottom hat operator applied to angiographic images is shown in FIG. 6E. In one embodiment, multiple image averaging is used for background subtraction. Further, in one embodiment, hesian filtering on a scale, such as scale 1, is performed following a bottom hat filter or bottom hat transformation 160e. Such a Hessian filter at scale 1 is performed to improve the image quality of the wire while smoothing the noisy image after applying the bottom hat operator. An example of a scale 1 hesian filter applied to an image is shown in FIG. 6F.

In one embodiment, a morphology close operation is performed on the image data. The morphology close operation is mainly used to fill the possible gaps that may be obtained in the step of applying the bottom hat transformation. The bottom hat transformation is applied with a small filter kernel to improve the image quality of narrow features such as guide wires.

Binary Image Map Features Applying a set of pretreatment steps to each angiographic image creates a binary map used to determine where the contrast agent is located. In one embodiment, the binary map refers to an image of the same size as the original angiographic image, where the pixels are either black or white, i.e. the pixels with dye are black and the pixels without dye are white. Or vice versa. The binary map may have areas of vascular pixels separated by the inherent imperfections of the binary map. The distance map can then be calculated based on the binary map. An exemplary distance map calculated using the FMM algorithm is shown in FIG. 6H.

The distance map is an image of the same size, and the value of each pixel is determined according to the distance from the nearest "black" pixel in the binary map. Clearly, the pixels where the dye is determined to be present in the binary map (the "black" pixel-the distance from the dye area is 0) remain black and surround the immediate vicinity of the black pixel area (from the dye area). Pixels have a brightness as low as "1". The next layer of pixel brightness is reduced by "2" and so on. As shown in FIG. 6H, the various luminance values are mapped to pixels arranged along the x-axis and y-axis with respect to pixel location. Scales encoded by color or other markings can be used to map luminance values for each pixel location. In one embodiment, the scale is a color scale. Various typical brightness values on the scale are shown in the figure. The central region has the lowest luminance value corresponding to B. The T-luminance value increases with reference to the B value. The Y luminance value increases with reference to the T value, and the R value increases with reference to the Y luminance value.

In the resulting distance map, the area of the dye / contrast agent in the original binary map looks like a ridge, with slopes declining on both sides. If the two such ridges are close enough (small distance in the binary map), they will look like connected ridges in the distance map. The dark center spot with the minimum value of the distance map belongs to the user hint point where the foreground begins to propagate. Due to the potential configuration, this propagates along the wire. The distal endpoint of the trace has the highest value on the distance map. One application of distance maps is to determine separate segments of dye / contrast agent that are close enough to be connected. In one embodiment, the distance map is a tool used to determine the vascular skeleton from a binary map. Distance maps can be used for a variety of purposes.

Typical anatomical feature detection / a priori data generation embodiments Further, in one embodiment, anatomical feature detection is performed as part of the preprocessing of angiographic images. In one embodiment, this can be done to generate specific a priori information relating to the path the imaging probe takes through the blood vessel. For example, line segment generation such as through a skeleton generation process can be used for feature detection. In one embodiment, the skeleton is a static object, such as one or more line segments, created to help trace the blood vessels of the subject being imaged.

The use of skeleton or line segment-based techniques to generate candidate pathways through blood vessels for data acquisition probes that can be used to signal centerline generation and marker tracking is several to refrain from using such techniques. Provide an advantage. For example, skeleton-based techniques can prevent or eliminate the generation of specific centerline traces that would otherwise pass through the side branch or imaging probe catheter. Skeleton generation provides a method for determining early candidates for the geometry of the vessel being imaged as well as lateral branches and other vessels as a map or framework to facilitate centerline generation. By generating skeletons, target points such as bifurcation points and vessel segments can be extracted to stabilize marker tracking and centerline tracing, and to verify tracking quality over the entire frame of angiographic image data. It is possible.

In one embodiment, the process of generating skeletons to detect anatomical features such as lateral branch and vascular geometry is implemented during pretreatment 160d of angiographic images. Skeletons can be used to detect anatomical features such as major bifurcations (170 l) and extrapolation points (170 m). In addition, the skeleton can be used to detect and generate a smooth vascular centerline (170f). For example, skeletons can be used with Dijkstra's algorithm. Skeletons can be generated based on preprocessed Hessian images. User-selected points on the angiographic image, such as the image of FIG. 7A, which are related to the guidewire position, can be used to reduce noise and facilitate skeleton generation.

In FIG. 7D, the endpoint selected by the user and the endpoint determined by the computer are indicated by X. The binary image generated from the Hessian image can be used to generate the skeleton in the angiographic image shown in FIG. 7B. After creating a skeleton, the skeleton can be eroded to eliminate small branches. For example, small branches of the skeleton can be removed or subtracted from the image until only the main section remains. Thresholds related to branch thickness and other parameters can be used to control skeleton erosion. Removal of small branches of the skeleton can be performed on a pixel-by-pixel basis in one embodiment until the final skeleton occurs as shown in FIG. 7C.

を使用して、図７Ｆに示されているように（２，４）を得ることができる。一実施形態では、例えばダイクストラ最短経路アルゴリズム又はその修正されたバージョン等のグラフ検索方法が、スケルトン内の血管の最良の候補経路を入手する為にグラフに適用される。これは、実際にはダイクストラアルゴリズムの修正バージョンである。選ばれる経路は、最大角度変化が以下によって提供される他の任意選択の経路に関して最小であったノード間の経路である。

In one embodiment, the location of the junction on the skeleton is located by detecting the bifurcation and other gaps indicated by the circled region of FIG. 7D. These joints are used to break down the skeleton into branches as shown by branches 1-13 in FIG. 7E. And each branch of the tree that is too small to represent a vascular branch can be eroded and eliminated. In one embodiment, all branches are eroded equally (same number of pixels in length). As a result, relatively long branches survive and relatively short branches are eliminated. The remaining skeleton branches can then be transformed into connected graphs as shown in FIG. 7F. The distances between graph nodes such as nodes 2 and 4 in FIG. 7F, that is, skeleton branches, are based on angular changes. When i = 2 and j = 4 with respect to the node, the following distance relationships:

Can be used to obtain (2,4) as shown in FIG. 7F. In one embodiment, a graph search method, such as Dijkstra's Shortest Path Algorithm or a modified version thereof, is applied to the graph to obtain the best candidate route for blood vessels within the skeleton. This is actually a modified version of the Dijkstra algorithm. The route chosen is the route between the nodes where the maximum angular variation was minimal with respect to the other optional routes provided by:

8A-8C show the best resulting path detected for the skeleton of FIG. 7E based on the application of the Dijkstra shortest path algorithm to the graph generated in FIG. 7F. FIG. 8A shows a route through nodes 2, 4, 8, 7, 6, 3 and 1. FIG. 8B shows a route through nodes 4, 6, 9, 3 and 1. FIG. 8C shows a route through nodes 2, 6, 8, 9, 7, 5, 3 and 1. The use of angles for distance measurement is useful when considering the three-dimensional nature of how nodes and branches are arranged within a subject.

Typical Catheter Detection Embodiments Further, in one embodiment, catheter detection is performed as part of the pretreatment of angiographic images 160f. The presence of the catheter in the field of view can interfere with the various steps and processing steps of the registration method. The intersection of the catheter and the blood vessel can be interpreted as a false bifurcation that can lead to unstable tracking. Marker and centerline tracking can be negatively affected by the presence of catheters that deliver intravascular imaging devices. Another problem associated with such catheters is that the shortest path between two points along a blood vessel can pass through the catheter instead of the blood vessel. As a result, the catheter can lead to errors and the generation of false centerlines.

Thus, for example, it is desirable to be able to assist centerline generation and remove the catheter from each frame of angiographic data before continuing further processing and detection. For example, as shown in FIG. 9A, for a given input angiographic image, for example, the vector fields shown in FIGS. 9B-9D are sections of the moving image, and the catheter extends to the central portion of the figure and the blood vessels are abbreviated in the figure. It can be superimposed on the image based on the detection of a section of the image showing the directional field as shown in FIG. 9B that intersects the catheter at an angle in the center. FIG. 9C shows a vector field map of the vascular region. FIG. 9D, on the other hand, shows a vector oriented substantially straight and vertically within the catheter area.

The vector of the vector field shown in FIGS. 9C and 9D is an eigenvector corresponding to the eigenvalue of the Hessian matrix calculated by the local secondary analysis. In FIG. 9C, all scales 1-5 were used in the flange filter. Examples of such filters include A.I. F. Flange (AF Flange), W.F. J. Niesen (WJ Niessen), K.K. L. Fincken (KL Vinkken), M.D. A. M.A. Viergever, Multiscale Vessel Enhancement filtering, Computer Medical Imaging and Computer Assisted Therapy (MICCAI), 1998, pp. 130-137. Therefore, the disturbed effects outside the blood vessels are explained. In FIG. 9D, only scale sigma-4 is used, thus using an isolated orientation on the catheter, while in the outer region, the eigenvectors have zero weight. For the sigma parameter, this parameter represents the Gaussian scale used in the convolution calculation. Sigma = 4 reflects the typical width of pixels for catheters observed in angiographic datasets.

In one embodiment, catheter detection is based on the primary assumption of catheter directivity and the fact that the catheter always intersects the lower boundary of the image as shown in FIG. 9D. Although local, catheters and blood vessels are generally indistinguishable from each other given their tubular structure. From the point of view of the shape of the catheter, the catheter can be globally distinguished from the blood vessels because it traverses approximately the entire image and has a substantially straight shape. In one embodiment, the vector orientation is used to distinguish between catheters and blood vessels. Locally, the vessel may have a small area of orientation similar to the orientation of the catheter. The eigenvector orientation of the catheter is generally closed up to 90 degrees, while the eigenvector orientation of the blood vessel is not closed.

In one embodiment, a method of catheter detection is used that incorporates a flange filter for the vascular as well as the shape features. In one embodiment, the method is only one scale (sigma = 4 that reflects the typical width of the catheter in pixels as observed in the angiographic dataset) and is oriented based on the eigenvectors of the vascular reference and Hesian images. Includes determining the image. Catheter in a given image frame of angiographic data can be isolated using a variety of criteria. These criteria include orientation (direction image threshold), length of connected components including the catheter (catheter profile length must be at least half the maximum image dimension of x (or y)). Will not be).

As a limitation of image processing software, it is a fact that the catheter usually intersects approximately the entire image when the catheter is detected so that it appears in a given image. In one embodiment, the system is programmed to assume that the catheter always cuts the lower border of the image. As a result, the bottom boundary can be set for the size of the detected object. In addition, when an area of the angiographic image associated with the catheter is detected, the boundary is slightly extended around the centerline of the catheter or otherwise in order to ensure that sufficiently large features have been detected. It is useful to expand. An example of a catheter detected based on the steps outlined above is shown in FIG. 9E.

Typical Catheter Removal Embodiment As described above, the presence of a catheter in the field of view of a given angiographic image can interfere with the various steps and processing steps described herein. Therefore, it is desirable to remove the catheter once it has been detected, for example, by the software-based methods listed herein. The bordered area shown in FIG. 9A shows the catheter diagonally overlapping the blood vessel. A variety of object exclusion approaches can be used to remove the catheter while striving to maintain image integrity. Based on the catheter mask that can be generated from the output of the catheter detection process used or as the output of the catheter detection process used, the software module can be configured to remove the catheter mask by excluding the catheter.

One advantageous technique for removing catheters uses the principle of superimposing functions to erase and remove when out of phase with each other. In one embodiment, a superposition-based software module is used to perform catheter removal, eg, by estimating its brightness profile and reducing it from the image. The catheter brightness profile can be generated based on sampling the points of the image identified as part of the catheter through the catheter detection software module.

As shown in FIGS. 10A and 10B, a typical cylinder 190 can be represented as a variety of longitudinal slices of thicknesses T0, T1 and T2 as illustrated. The cylinder 190 can be displayed as a model representation of the catheter. To the extent that the catheter and cylinder 190 are filled with contrast medium, the change in brightness caused by the contrast medium is greater in the center along the thickness T0, then diminishes as it moves away from the center T0 to slice T1 and then. It further decreases when the slice T2 is reached. Therefore, because there is less contrast at the thinner edge of the catheter compared to the center of the catheter, a brightness profile for the catheter was generated to remove the catheter from the image and the catheter was detected. Can be added to the area of the image. A typical representation of the catheter removal method involved is shown in FIG. 11A.

Given that the catheter was detected as described herein, the mask associated with the pixels in the image constituting the catheter is by using a mask region, such as the region shown in FIG. 9E. Can be generated. In one embodiment, the image brightness is sampled in the catheter region, for example, on a line perpendicular to the catheter line. These vertical lines gradually diminish from one side of the catheter until they reach the low or relative extremum corresponding to the thickest central part of the catheter, at the edge of the catheter, as shown in FIGS. 10A and 10B. As the cross section of the catheter becomes thinner, it straddles the slope of the contrast medium brightness change that gradually increases again. Each line sampled in the catheter area produces a brightness curve. Various brightness curves can be averaged into a single curve. This brightness curve can be inverted and then superimposed on the vertical lines constituting the catheter region as shown in FIG. 11A to effectively remove the catheter from that region.

Typical Shading Embodiment A standard Hessian-based filter is part of the pretreatment and is based on the Hessian eigenvalues of the image. In one embodiment, hesian is calculated on several discrete scales and then the maximum response between them is taken. In one embodiment of the shading process, scales 1-5 are used. Scale 5 can be selected as a scale representing the typical maximum observed vessel width in the available data. The original image, as well as examples of subsequent processing to remove shadows and other features, are shown in FIGS. 6I-6N.

The shading pretreatment step is applied to transform the original image into a modified image with an improved contrast level. In addition, the modified image is modified by the process of applying hesian so that it is virtually unaffected by the shadows of the heart and diaphragm, which can induce several regions or planes of different contrast. .. Removing these shadows is desirable because such areas or planes can lead to incorrect vascular centerlines. In one embodiment, the deshaping step involves applying the bottom-at operator in a filter kernel that is configured to have a distance parameter that is much larger than the width of a typical blood vessel. 6L and 6J show modified images improved by performing a shading process.

Typical Vascular Center Line (Trace) Generation Embodiment Two anchor points, the distal and proximal points, mark the end and start points of the blood vessel center line. Another point is reflected on the vascular skeleton, and Dijkstra's algorithm is applied to find the shortest path in terms of smoothness. FMM is also applied to find the shortest path in terms of brightness (FMM is performed on a hesian image with improved image quality). The FMM results are combined with the Dijkstra results to produce the best vascular centerline (trace) between the two anchor points. Vascular centerlines in other angiographic frames are generated by applying a conformal map combined with the FMM to the originally generated trace.

In one embodiment, a fast marching technique or fast marching method addresses an efficient calculation of geodetic distances based on potential. In one embodiment, in the presence of contrast agent, the potential may be an image with improved image quality. In one embodiment, the potential constructs a function based on distance transformation when only the guide wire is present (even if it is piecewise visible on the angiographic image), for example in the absence of contrast agent. Adjusted by. One method for calculating the foremost propagating potential function can be performed with guidewire-based potentials by applying the Euclidean distance transformation to the binary image. Once the distance transformation is generated, the transformation can be further modified to a potential function by applying a power exponent to the negative fractional power of the distance transformation. A typical guidewire potential is shown in FIG. 6G.

FIG. 5B shows the process flow 170 involved in vascular centerline generation. In one embodiment, a hesian having a scale of 1 is applied to a frame of angiographic data 170a. This application of Hessian leads to improved image quality of thin ridges in images such as guide wires. In one embodiment, the automatic detection of the guide wire and the selection of anchor points on the guide wire are performed 170c. When the guide wire is detected, in one embodiment, the point with the highest LoG response is identified as the anchor point. Tracking the distal guidewire anchor point to all pullback angiographic frames 170e is then performed. Proximal anchor points are detected in a single frame. Distal anchor points are also detected in a single frame. In one embodiment, each anchor point is a feature that can be easily detected in other frames by tracking. The anchor points are then traced to all frames such that each angiographic frame has two endpoints for vascular centerline generation (trace).

In one embodiment, a user-selected point, such as a guide wire point on the angiographic image, is selected 170j. Similarly, a scale hesian (up to about 5) can be applied to angiographic images to improve the image quality of blood vessels at 170 k. The image modified as a result of the Hessian application can then be used to perform 170 l of detection of the nearest branch anchor point. This detection step can use a point selected by the user, that is, a hint point, as input. Next, 170 m where extrapolation detection of anchor points is performed. Clarify which anchor points are being detected. Next, 170n, where tracking of anchor points is performed up to all pullback angiography frames.

In one embodiment, the system then uses a graph search software module such as Dijkstra's shortest path solution for the graph. Applying the Dijkstra algorithm or other shortest path algorithm combined with the FMM to select the best initial vascular centerline can then be performed with respect to the angiographic pullback frame 170f. Tracking the vessel centerline with an angiographic pullback frame using FMM in a narrow band based on conformal mapping is then performed 170 g. In this context, narrowband means building a narrowband region around the desired trace. This narrow band is intended to increase the efficiency of the FMM algorithm for the calculation of geodetic distances in a limited area of the image. These centerlines can be stored in one or more tables and displayed in an applicable angiographic image.

Typical Marker Detection and Co-Registration Embodiment FIG. 5C shows a process flow 180 related to marker detection and co-registration. As used herein, term traces can be replaced with centerlines. Initially, as an input, the centerline (trace) from the pullback frame is provided as an input for the sampling orientation 180a. 180c in which LoG is further applied to the image from the pullback frame. 180e where sampling of LoG images perpendicular to the trace is performed. In one embodiment, 180 g where dynamic programming or iteration is performed at different start or end points to find marker positions in all frames. In one embodiment, dynamic programming or iterative processes can be implemented using the Viterbi algorithm. Next, 180h, the selection of the most possible solution for the marker on a frame-by-frame basis is performed. In all frames, the calculation of the marker position along with the marker normalized arc length position along the vessel centerline is performed 180 l.

Next, it is possible to superimpose and register all combinations of the OCT frame and the angiography frame based on the marker positions calculated in terms of the arc length. Since all vascular centerlines start and end with the same anatomical features of all angiographic frames, each centerline coincides with other centerlines of other frames. Therefore, the centerline length or arc length can be used as the basis for registration. Marker positions in terms of arc length are preserved (up to some error) in every frame.

One challenge encountered when attempting to resolve the opaque marker band of a sensor or data acquisition probe is the use of contrast medium as part of the OCT pullback. In one embodiment, it is useful to process frames of angiographic data prior to introduction of the contrast solution so that a guide wire and an imaging catheter can be used to provide an initial route through the blood vessel. This initial dataset can be iteratively improved using other information and parameters as described herein.

The Viterbi-based algorithm automatically detects X-ray opaque markers in each pullback image. This algorithm can be used to obtain a global solution based on small chunk brightness and location predictions (constant velocities along the trace). A prerequisite for this algorithm is the process of detecting and tracking the blood vessel centerline (trace). Traces are used to create a continuous registration between OCT and angiography. These curves are calculated using the fast marching method. The fast marching method allows efficient calculation of the path (trace) between the proximal point and the distal rest marker (which may be a user-selected point, i.e. a hint point) at each frame. .. Rest markers are detected on the frame (with and / or without contrast agent / dye). Template matching techniques are used to track the proximal and distal points over subsequent sequences.

The Viterbi algorithm is configured to balance exogenous and intrinsic factors. The extrinsic factor (marker band display) is derived from the marker band LoG map by resample the map with discrete strips perpendicular to the trace for each angiography frame. The intrinsic factor is the progression of arc length over time. This intrinsic factor models the advance of the marker band along the arc length of the pullback. The basic idea is that while the average pace is determined by the pullback rate, there is a penalty for deviating from this pace. This factor takes into account the natural "saw" profile by penalizing forward / backward movements differently.

FIG. 12 shows a data acquisition and registration system 300 that includes a variety of software and hardware components suitable for processing intravascular image data and angiography data. In one embodiment, after one or more frames of the OCT image data and the angiographic image data are superimposed and registered, the output is a registration table. In one embodiment, the frame of the OCT data can be monitored to check for a clear frame display state, and this clear frame display will allow the cine to capture the frame of the angiographic data. Can be used to trigger. In one embodiment, for a predetermined pullback procedure in which the probe is pulled back through the vessel while probe data and angiography data are being collected, frame time stamping, registration table addition, and image processing features and other processes are performed. May be done.

The user interface (UI) 308 communicates with the OCT adapter 320. The image processing module 330 communicates with the OCT adapter 320. In one embodiment, the image processing module 330 performs an operator or transformation, or angiography, for example for shading, guidewire detection, catheter removal, and other image processing steps outlined herein. Applies to frames of data. The optical coherence tomography system 310 communicates with the OCT adapter 320. The optical coherence tomography system 310 may include or communicate with the frame grabber 302. Angiographic frames are acquired using a frame grabber and fetched by a software module.

The OCT frame table 315 contains blood vessel information and images obtained during pulling back of the imaging probe through the blood vessel. The role of the OCT adapter 320 is to provide a software interface between the angiography system and the OCT system.

Software-based systems such as servers or workstations described herein, and software modules configured to automatically execute, capture, and tag each image by its acquisition time, include acquisition times. Supports registration of intravascular data tagged with. The image processing module 330, which may include a registration software module, automatically detects an X-ray opacity marker on each angiographic image corresponding to the intravascular acquisition. A single user input may be required to assist in detection as shown in FIG. 5B. The registration software module calculates the path of the intravascular imaging catheter on all angiographic images corresponding to the intravascular image acquisition during pullback of the probe through the vessel being imaged. The registration software module provides accurate intravascular images, the location of X-ray opaque markers on each angiographic image, the location of each intravascular image / data point on each angiographic image, and the accuracy of the results. It gives rise to a registration table of external images containing FOM associated with each registration result, which is a measure of the level of confidence in the vessel.

The user provides a graphical representation of the intravascular and angiographic images, and a correspondence between the two, such as a particular intravascular image on the angiographic image, as part of the user interface when the registration is completed in one embodiment. Is presented. If the FOM or confidence score is unacceptable during the registration procedure, additional user input or other parameters from the OCT system may be requested or obtained automatically.

Typical Confidence Score / Figure of Merrit Each time a probe marker is detected, a confidence score, also called FOM, is assigned to each detected probe marker. The score is based on one or more of the mass brightness, the number of dark masses near the predicted area of the marker, the marker arc length along the trace, the mass movement and the stability of the trace. The FOM / score reflects the confidence criteria of the returned results. In one embodiment, the score is in the range [0,1], where 0 reflects the lowest confidence and 1 reflects the highest confidence.

Software modules related to angiography, such as one or more modules described herein, typically evaluate images generated using an imaging device located outside the subject's body. .. In contrast, data acquisition probes such as OCT, IVUS, FFR, pressure, or other data acquisition modality can be placed inside the patient's blood vessels. As a result, the data obtained from the data collection probe involved during pullback, or the parameters known to be related to the data collection probe, are angiographic software to improve the behavior of the methods and steps described herein. Can be used by. The adapter software module or other software module can be used to provide OCT information to the angiographic image frame processing wear module and vice versa.

For example, the following parameters related to the data obtained with respect to the blood vessel as part of the intravascular data collection may be used for analysis or for the evaluation of the subject, or for different datasets, pull-back lengths in mm, pull-back. Start, end of pullback, eg display of bifurcation points such as side branches from collected OCT data, data collected for frames prior to introduction of contrast agent or dye, synchronized frame time tags for OCT and angiography, pullback rate, The distance between the distal and proximal markers of the catheter, as well as, for example, longitudinal angiographic data, pressure data, EKG data, cardiac contraction state during pullback, cardiac diastolic state during pullback, and subject. It can be sent to an angiography software module or other software module to correlate other factors and parameters obtained with respect to a given data acquisition modality such as other available conditions.

Angiography table An angiography table as shown in FIG. 14 contains not only each acquired angiography frame, but also information explaining the angiography pullback. The angiography table is created by the angiography software module at the time of acquisition and the time stamp data is partially added. This table is extracted and stored by the OCT module when the acquisition is complete. The table is then provided to the angiography software module at the time of registration when the registration dependent field is added.

Co-registration table The co-registration table contains the results of successful co-registration, as shown in FIG. The registration table contains all of the OCT / angiography cross-reference information required to drive the registration GUI toolset. This table contains entries per OCT frame, including a list with entries per angiography frame containing the acquisition time stamp of that frame and OCT marker location information. In one embodiment, the registration table associates the OCT frame index with the registered angiographic frame index. In addition, the table may contain entries that associate OCT and angiographic frames.

Additional Multimodal Collaboration Features and Embodiments In one embodiment, the registration may synchronize frames from two or more data collection modality, or combine information from two or more data collection modality. Point to. For example, the branch detected in the OCT image can be used as an anchor for the branch detected in the angiographic image. The registration features listed here are not limited to OCT. Alternatively, the features described herein relating to imaging data acquisition modality or other data acquisition modality relating to the vasculature and individual vessels can be extended to other intravascular imaging modality. In one embodiment of the invention, the centerline of the blood vessel is determined from the path of the guidewire or catheter that is tracked by a tracking system such as the Medical Position System of Medicine during the advancement of the guidewire or catheter through the blood vessel.

In one embodiment, the side branch detected in the OCT frame of data using OCT image processing can be used as an input to improve co-registration with the angiographic data. For example, in one embodiment, each OCT frame includes a flag (yes / no) indicating whether a side branch is present. In addition, once registration is obtained, stents, calcium deposits, lipid deposits, thrombi, asymptomatic ruptured plaques (TCFA or "vulnerable plaques"), vascular normalization, lateral branch detection, (based on OCT imaging data) Angiography of FFR values (which may be calculated as VRR) values, lumen size values, stents, and a variety of other data described herein, taking into account registrations between datasets. It can be overlaid on an image or OCT image.

Live Stent Placement Guidance Embodiments and Features In one embodiment, following OCT / angiography registration, a guide wire is retained after pulling back for stent particles via another catheter. The process of imaging the blood vessel that was the subject of pullback continues via continuous fluoroscopic imaging, which is superimposed and registered in OCT. Moving along the OCT frame or angiography frame allows you to see side branches and other information. In one embodiment, various processing steps are performed on OCT data such as pre-stent, 3-D superposition registered virtual tissue structure, lumen detection, guidewire detection, stent incomplete adhesion, plaque detection and the like. Since the OCT frame and the angiography frame are registered, the information detected in the OCT frame can be overlaid on the angiography image used by the operator to place the stent. If the side branches can be shown on the angiography on the user interface, this may help avoid unwanted confinement of the side branches during stent deployment.

In addition, various types of overlays related to previously deployed or candidate stents for deployment can be displayed on one or both OCT and angiographic images following registration. For example, a bioabsorbable scaffold (BVS), a new type of stent that is X-ray permeable, can be detected in the OCT frame using OCT imaging. This OCT image data can be used to provide a particular type of stent overlay that is important in the context of angiographic data, as the stent is not visible to X-rays. As another special case of data overlay, in one embodiment, the information of stent incomplete adhesion from a given OCT frame modifies the stent image on the X-ray image by color coding or generating other markings. It can then be used to indicate areas of incomplete adhesion.

Further, considering that the markers on the stent delivery probe can be traced, the stimulated stent can be indicated with respect to the markers in OCT longitudinal mode or L mode. Angiography / OCT registration shows cross-references of tissue features, lumen features, and movement features such as, for example, balloon or stent insertion, with overlays and with display of elements such as stent cross sections in L mode. Can be done. Diameter and length are used to more accurately display the stent in L mode or angiography when the stent scan is obtained as a wireframe model or selected from the drop-down menu prior to stenting. it can.

In one embodiment, bands on OCT images and / or angiographic images showing sites for avoiding stenting, such as side branches and target deployment sites based on stenosis / MLA calculations, can be used. Angiographic and OCT displays can be used to show higher levels of particle size with overlays to help the user properly position the stent within the target area. In addition, given the stent's wireframe model and the lumen area calculated from the OCT frame overlaid and registered with the location of the stent in the angiography system, it provides visual guidance for the stent expansion target. Can be displayed. In one embodiment, this can be performed using a stent-stimulated wireframe and a magnifying balloon used to selectively magnify one or both ends of the stent. These types of studies using OCT and angiography can be used before, after, or as part of future follow-up.

In one embodiment, OCT and angiographic data are stored when pullback is performed. This stored data can be used to generate images or models of arteries. Using such a model enhances live stent placement during subsequent pullbacks. In this way, the pre-existing OCT / angiography registration information can be used as a baseline.

Angiographic data can also be used to notify, improve or correct OCT image display features or detection algorithms. One correction of OCT from angiographic data is to respace OCT frames in L mode to show the actual physical separation between frames measured by the registration tool. This compensates for the spacing error that results from assuming a constant pullback rate relative to the vessel lumen. In reality, the pullback speed changes significantly due to the movement of the heart, and our frames are not evenly spaced. Once the overlay-registered OCT and angiography datasets are generated, software modules can be used to accurately measure frame-to-frame spacing. The frame-by-frame correction can be applied to respace the L-mode views in a given user interface. This will also apply to 3D OCT rendering and will provide a more accurate visual representation of blood vessels.

In general, a variety of additional advantages can be considered by having a superposed set of frames and bidirectional communication between the angiography system and the OCT system. Angiographic information includes traces generated for different blood vessels. The junctions of these branches can be mapped to specific frames to signal OCT side branch detection. In one embodiment, by storing the angiography data and OCT obtained during angiography, the OCT images at different time points are superimposed on the angiography data that acts as a bridge or linker between two different OCT datasets. You can build records over time that you can use to register.

In one embodiment, a pressure probe or other data acquisition modality can be used to collect data and use another imaging modality or parameter to improve vascular representation. In one embodiment, the VRR can be used to calculate the percentage contribution of each stenosis to the overall FFR value and display the percentage relative to the angiographic data. In addition, side branch location information from OCT or angiographic images can be used to improve VRR calculations by identifying additional junctions and flow points in the region near the vessel being imaged.

In one embodiment, the system and method can be used to monitor the thrombectomy catheter in OCTL mode. That is, it can be used with the stimulated stents described herein and inductive stenting using alignment data. In general, in part, the present invention tracks any therapeutic device with an X-ray opaque marker band, and locates it in OCTL mode images and previously acquired, superimposed registered X-ray images. Regarding displaying. The therapeutic device may be a stent or thrombectomy catheter, or a balloon device such as an angiographic balloon or drug-eluting balloon, or a resection device such as a rotary porridge resection probe (rotablator).

Non-limiting Software for Implementing Angiography and Intravascular Data Collection Methods and Systems Features and Embodiments The following description is device hardware suitable for performing the methods of the invention described herein. And other behavioral components are intended to provide an overview. This description is not intended to limit the applicable embodiments or scope of the invention. Similarly, hardware and other operating components may be suitable as parts of the equipment described herein. The present invention can be practiced in personal computers, multiprocessor systems, microprocessor-based electronic devices or programmable electronic devices, network PCs, minicomputers, mainframe computers and the like.

Some parts of the embodiment for carrying out the invention are presented in terms of algorithms and symbolic representations of their behavior on data bits in computer memory. Descriptions and representations of these algorithms are available to those skilled in the art related to computers and software. In one embodiment, the algorithm is here and generally considered to be a coherent sequence of operations leading to the desired result. Methods, performed as steps, or other actions described herein are operations that require a physical amount of physical manipulation. Usually, but not necessarily, these quantities take the form of electrical or magnetic signals that can be stored, transferred, combined, converted, compared and otherwise manipulated.

Unless otherwise stated clearly from the following description, "process" or "calculate" or "calculate" or "compare" or "measure arc length" or "detect" or "detect" through the description. Descriptions using terms such as "trace" or "mask" or "sample" or "work" or "generate" or "determine" or "display" are in the register and memory of the computer system. Manipulating data represented as a physical (electronic) quantity of data, and other data similarly represented as a physical quantity within a computer system memory or register or other such information storage device, transmission device, or display device. Refers to the actions and processes of a computer system or similar electronic computing device that translate into.

The present invention also relates to, in some embodiments, a device for performing the operations herein. The device may be specifically constructed for the required purpose, or it may include a general purpose computer that is selectively activated or reconfigured by a computer program stored in the computer.

The algorithms and displays presented herein are essentially unrelated to any particular computer or other device. A variety of general purpose systems may be used with the program according to the teachings herein, or it may prove convenient to build more specialized equipment to perform the required method steps. The structures required for these various systems emerge from the description below.

Embodiments of the invention are for use with processors (eg, microprocessors, microcontrollers, digital signal processors, or general purpose computers), programmable logic circuits (eg, field programmable gate arrays (FPGAs) or other PLDs). Includes, but never includes, programmable logic, discrete components, integrated circuits (eg, application specific integrated circuits (ASICs)), or computer program logic for use with any other means, including any combination thereof. It may be implemented in many different forms without limitation. In a typical embodiment of the invention, some or all of the processing of data collected using OCT probes, FFR probes, angiography systems, and other imaging and subject monitoring devices, as well as processor-based systems. Is converted into a computer executable format, stored on a computer readable medium, for example, and implemented as a set of computer program instructions executed by a microprocessor under the control of an operating system. Thus, for example, user interface instructions and triggers based on the completion of a pullback or registration request to generate OCT data and perform image processing using the various features and other features and embodiments described above. A suitable processor is translated into an understandable instruction.

Computer program logic that implements all or part of the functionality described herein is generated by source code format, computer executable format, and various intermediate formats (eg, assembler, compiler, linker, or locator). It may be embodied in a variety of formats, including, but by no means limited to. The source code is of a high standard of various programming languages (eg, object code, assembly language, or eg Fortran, C, C ++, JAVA®, or HTML, for use with a variety of operating systems or operating environments. It may include a set of computer program instructions implemented in any of the languages). The source code may define and use a variety of data structures and communication messages. The source code may be in computer executable format (eg, via an interpreter), or the source code may be converted to computer executable format (eg, via a translator, assembler, or compiler).

Computer programs include, for example, semiconductor storage devices (eg RAM, ROM, PROM, EEPROM, or flash programmable RAM), magnetic storage devices (eg diskettes or fixed disks), optical storage devices (eg CD-ROM), PC cards. It may be permanently or temporarily fixed to a tangible storage medium (eg, a PCMCIA card) or other storage device in any format (eg, soft code format, computer executable format, or intermediate format). Computer programs use any of a variety of communication technologies, including, but by no means limited to, analog, digital, optical, wireless (eg Bluetooth®), networking, and internetworking technologies. It may be fixed in any format with a signal that can be transmitted to the computer. Computer programs are attached printed documents that are preloaded on a computer system (eg, on a system ROM or fixed disk) or distributed from a server or electronic bulletin board on a communication system (eg, the Internet or the Worldwide Web). Alternatively, it may be distributed in any format as a removable storage medium together with an electronic document (for example, packaged software).

Hardware logic (including programmable logic for use with programmable logic circuits) that implements all or part of the functionality described herein may be designed using conventional manual methods or may be designed. Electronically designed, captured and simulated using a variety of tools such as computer-aided design (CAD), hardware description language (eg VHDL or AHDL), or PLD programming language (eg PALASM, ABEL, or CUPL). , Or may be documented.

Programmable logic includes, for example, semiconductor storage devices (eg RAM, ROM, PROM, EEPROM, or flash programmable RAM), magnetic storage devices (eg diskettes or fixed disks), optical storage devices (eg CD-ROMs), or the like. It may be permanently or temporarily fixed to a tangible storage medium such as a storage device of the above. Programmable logic can be transmitted to a computer using any of a variety of communication technologies, including but not limited to analog, digital, optical, wireless (eg Bluetooth), networking and internetworking technologies. It may be fixed at a certain signal. Programmable logic is an attached printed document preloaded in a computer system (eg, on a system ROM or fixed disk) or distributed from a server or electronic bulletin board on a communication system (eg, the Internet or the World Wide Web). Alternatively, it may be distributed in any format as a removable storage medium together with an electronic document (for example, packaged software).

Various examples of suitable processing modules are described in more detail below. As described herein, a module refers to software, hardware or firmware suitable for performing a particular data processing or data transmission task. In one embodiment, the module is an instruction, or eg, angiography data, OCT data, FFR data, IVUS data, registration table data, centerlines, shading, pixels, brightness patterns, and other as described herein. Refers to software routines, programs, or other memory-resident applications suitable for receiving, converting, sending, and processing various types of data, such as information of interest.

Computers and computer systems described herein use operably coupled computer-readable media, such as memory for storing software applications used in obtaining, processing, storing and / or communicating data. May include. It can be seen that such memory may be internal, external, remote, or locally present with respect to the operably coupled computer or computer system.

In addition, the memory is, for example, without limitation, hard disk, optical disk, floppy (registered trademark) disk, DVD (digital versatile disk), CD (compact disk), memory stick, flash memory, ROM (read-only memory), RAM (random). To store software instructions or other instructions, including access memory), DRAM (Dynamic Random Access Memory), PROM (Programmable ROM), EEPROM (Electrically Erasable PROM) and / or other similar computer readable media. Any means of may be included.

In general, the computer-readable memory medium applied in connection with the embodiments of the present invention described herein may include any memory medium capable of storing instructions executed by a programmable device. If desired, the method steps described herein may be embodied or executed as instructions stored on one or more computer-readable memory media. These instructions are various programming languages such as C ++, C, Java®, and / or various other types of software programming languages that may be applied to create instructions according to embodiments of the present invention. It may be software embodied in.

Aspects, embodiments, features and examples of the invention should be considered exemplary in all respects and are not intended to limit the invention and the scope of the invention is in the claims. Only defined. Other embodiments, modifications, and uses will be apparent to those skilled in the art without departing from the claimed spirit and scope of the invention.

The use of the headings and sections of the present application is not intended to limit the invention, and each section may apply to any aspect, embodiment or feature of the invention.

Throughout this application, when a component is described as having, including, or compiling a particular component, or when a process is described as having, including, or comprising a particular process step. It is intended that the components of this teaching also consist of or substantially consist of the listed components, and that the processes of this teaching consist of or substantially consist of the listed process steps.

In the present application, when an element or component is said to be included in and / or selected from the list of listed elements or components, the element or component is listed as an element or component. It should be understood that it may be any one of the above and can be selected from a group consisting of two or more of the listed elements or components. Moreover, the elements and / or features of the components, devices or methods described herein deviate from the spirit and scope of this teaching, whether explicit or implied herein. It should be understood that they can be combined in various ways without.

The use of the terms "include, includes", "include", "have, has" or "have" is generally open-ended and unrestricted unless expressly stated otherwise. It should be interpreted as a target.

The use of the singular form herein includes the plural form (and vice versa) unless otherwise stated. In addition, the singular forms "a, an" and "the" include the plural unless the context clearly indicates otherwise. Further, if the use of the term "about" precedes a quantitative value, the teachings also include the particular quantitative value itself, unless otherwise stated.

It should be understood that the order of steps or the order in which a particular action is performed is not important as long as this teaching is operable. In addition, two or more steps or actions may be performed simultaneously.

If a range or list of values is provided, each intermediate value between the upper and lower limits of the range or list of values is considered individually, as if each value is explicitly listed herein. , Included in the present invention. Further, between the upper and lower limits of the predetermined range, and smaller ranges including the upper and lower limits of the predetermined range are considered and are included in the present invention. A typical enumeration or range of values is not a negation of other values or ranges, including between the upper and lower limits of a given range, and the upper and lower limits of a given range.

It should be understood that the various aspects of the claimed invention cover a subset and substeps of the techniques disclosed herein. Moreover, the terms and expressions used herein are used as descriptive terms rather than as limiting terms, and the use of such terms and expressions is the use of the features illustrated and described or parts thereof. It is not intended to exclude any of the equivalents, but it is recognized that various modifications are possible within the scope of the claimed invention. Accordingly, what is desired to be guaranteed by a patent certificate is the invention as defined and distinguished within the claims below, which includes all equivalents.

[Appendix 1]
A processor-based method for displaying angiographic and intravascular representations of blood vessels:
To generate a set of optical coherence tomography image data in response to a distance measurement of the vessel obtained during pulling back of a probe through the vessel using an optical coherence tomography system, the optical coherence tomography. Generate a set of tomographic image data, including multiple cross-sectional images at multiple locations along the blood vessel;
The angiography system is used to generate a set of angiographic image data during the pullback of the probe through the blood vessel using the optical coherence stromography system. , Including multiple two-dimensional images obtained at different time points during the pullback;
Displaying a first panel containing a first longitudinal view of the blood vessel generated using the OCT image data; and a first panel containing a frame of the angiographic image data identifying the blood vessel. , A method comprising displaying using one or more points in the frame and a blood vessel centerline passing through the one or more points.
[Appendix 2]
Further comprising superimposing and registering the OCT image data and the angiographic data using the vessel centerline to generate continuous alignment of the markers to be tracked.
The method of Appendix 1, wherein the tracked marker is placed on an OCT data acquisition probe.
[Appendix 3]
The OCT image data and the angiography data are superimposed and registered so that the frame identifier in the first longitudinal view is changed by selecting a point along the blood vessel centerline through the user interface. The method according to Appendix 1, further comprising.
[Appendix 4]
By performing an iterative search using the pullback speed or pullback length, candidates for the tracked marker are eliminated based on the pullback length and / or the position that the marker can take based on the pullback speed. The method according to Appendix 2, further comprising the above.
[Appendix 5]
The method of Appendix 1, wherein the vessel centerline is generated using a plurality of processing steps from the shortest path technique and Dijkstra's algorithm.
[Appendix 6]
The method of Appendix 1, further comprising the step of removing the guide catheter image from one or more frames of the angiographic data using the superposition of the luminance profiles.
[Appendix 7]
The method of Appendix 1, wherein the blood vessel centerline is generated using pathway information generated from one or more angiographic frames in the substantial absence of a contrast agent solution.
[Appendix 8]
It is described in Appendix 1, further comprising detecting the angiography data and the optical coherence stromography data, and generating a confidence score for each of the overlay registrations between the angiography data and the optical coherence stromography data. Method.
[Appendix 9]
A method for detecting intravascular probe markers:
Obtaining a first frame of angiographic image data that is substantially free of contrast agent and contains the intravascular probe marker;
Obtaining a second frame of the angiographic image data, including contrast agent image data in the vicinity of the intravascular probe marker; and detecting the intravascular probe marker in the first frame and the second frame. Including methods.
[Appendix 10]
A step of applying an image processing transformation to the second frame to remove or modify features within the second frame; and a step of increasing the brightness of the plurality of pixels, wherein the plurality of pixels are: 9. The method of Appendix 9, further comprising an ascending step, comprising a guide wire image in the second frame.
[Appendix 11]
9. The method of Appendix 9, further comprising generating an average luminance value for a plurality of images and subtracting the average luminance from the first or second frame.
[Appendix 12]
The method according to Appendix 9, further comprising applying the bottom hat operator to the second frame and applying the morphology close operation.
[Appendix 13]
To detect the intravascular probe marker is:
Filtering candidate markers containing pixels in the first frame and the second frame by applying the Multiscale LoG (Laplacian of Gaussian) operator to the first and second frames; The method according to Appendix 9, further comprising performing a non-maximum suppression process to identify a small mass having a maximum value in the vicinity of a pixel.
[Appendix 14]
9. The method of Appendix 9, further comprising the step of generating a guidewire-based potential by applying a Euclidean distance transformation to a binary image and applying a power exponent to the negative fractional power of the distance transformation.
[Appendix 15]
14. The method of Appendix 14, further comprising determining a plurality of geodesic distances based on the potential of the guidewire base using a high speed marching method.
[Appendix 16]
Removing shadows from the first frame and the second frame;
9. The method of Appendix 9, further comprising increasing the contrast level of the guidewire of one of the first frame or the second frame; and performing a morphological image reconstruction on each marker candidate. ..
[Appendix 17]
Processing multiple pullback frames with a Hesian-based vascular filter; and using template matching from either the first frame or the second frame through the multiple pullback frames. 9. The method of Appendix 9, further comprising tracking the intravascular probe marker up to all the pullback frames.
[Appendix 18]
9. The method of Appendix 9, further comprising tracking said intravascular probe markers through multiple frames obtained during pullback using the Viterbi dynamic programming method.
[Appendix 19]
A processor-based method of superimposing and registering angiographic image data and intravascular image data obtained during pullback through a blood vessel:
Storage of multiple frames of optical coherence tomography data in memory;
Store a plurality of frames of the angiographic image data in the memory;
Processing the plurality of frames of the angiographic image data so that one or more shadows are substantially reduced;
Detecting a catheter in the plurality of frames of the angiographic image data;
Removing the detected catheter within the plurality of frames of the angiographic image data;
Generating blood vessel centerlines for the plurality of frames of the angiographic image data;
Detecting probe markers in the plurality of frames of the angiographic image data;
Tracking the position of the probe marker along one or more of the blood vessel centerlines; and the plurality of frames of the angiographic image data and the plurality of frames of the optical coherence tomography data. A method comprising overlay registration using the tracked location.
[Appendix 20]
19. The method of Appendix 19, further comprising generating a score indicating the level of confidence in the overlay registration between the frame with the angiographic image data and the frame with the optical coherence tomography data.
[Appendix 21]
The method of Appendix 19, wherein the step of removing the detected catheter is performed using a superposition of luminance profiles generated based on sampling of the area of the detected catheter.
[Appendix 22]
The step of superimposing and registering the plurality of frames of the angiographic image data and the plurality of frames of the optical coherence tomography data includes generating a registration table using a computing device.
The method according to Appendix 19, wherein the registration table includes an angiographic image frame, a plurality of OCT time stamps per frame, a plurality of angiography time stamps per frame, and an optical coherence tomography image frame.
[Appendix 23]
22. The method of Appendix 22, further comprising displaying a stent representation on OCT and angiographic images in a user interface using the registration table and computing device.
[Appendix 24]
22. Addendum 22, further comprising identifying the side branch in one or more of the OCT and angiographic images using the registration table and a user interface configured to display the side branches. the method of.
[Appendix 25]
22. The appendix 22 further comprises adjusting the pullback speed change by setting the frame spacing of the OCT data based on the registration table and displaying a longitudinal view on the user interface based on the spacing. the method of.

Claims (1)

A method of operating a processor device that displays angiographic and intravascular representations of blood vessels:
To generate a set of optical coherence tomography image data in response to a distance measurement of the vessel obtained during pulling back of a probe through the vessel using an optical coherence tomography system, the optical coherence tomography. Generate a set of tomographic image data, including multiple cross-sectional images at multiple locations along the blood vessel;
Including methods.

Images