JP2007296341A - System and method for determining distal end of catheter with x-ray base - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve a method for detecting the 3D position of a catheter. <P>SOLUTION: The method has the step of segmenting one or more of catheter regions in a pair of corresponding images by comparing the threshold values of image intensity histogram gradients, the step of removing pixels present in the catheter regions of pixels to form a catheter contour, the step of matching the catheter contour to the 2D template of the catheter to select a plurality of candidate catheter front end pixels, the step of matching at least some of the catheter front end pixels of the candidate catheter front end pixels in the pair of the images to the 2D projecting image of the 3D template of the catheter front end to select the most coinciding candidate pixel as the catheter front end, and the step of detecting the 3D position of the catheter front end from the intersecting point of two projecting lines. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  This application claims priority to "X-Ray Based Catheter Tip Localization" of US Provisional Application No. 60/79658, filed April 27, 2006 by Balda, et al. The content of this document is included in the disclosure content of the present application by reference.

  The present disclosure relates to an algorithm for detecting and locating a catheter tip in three dimensions from two fluoroscopic images.

  Catheter positioning has become a popular technique in heart related applications. Traditionally, catheter manipulation is primarily based on fluoroscopic images, which are two-dimensional X-ray images acquired during the procedure. Fluoroscopy is an imaging technique commonly used by physicians to obtain real-time images of a patient's internal structure by using a fluoroscope. A typical fluoroscopic device includes an x-ray source and a monitor between which a patient is placed, such a typical fluoroscopic device in an x-ray image intensifier and a CCD video camera. Are combined. A fluoroscopic image does not provide as much anatomical information as a computed tomography image (CT) or magnetic resonance (MR) image, nor does it provide depth information. It is not possible to know the exact 3D position of the catheter tip only by looking at one perspective image. Therefore, to operate using a fluoroscopic image, knowledge of anatomy and experience of interpretation of the fluoroscopic image are required.

  With the catheter positioning technique, a computer-aided operating system coupled to the fluoroscope can know where the tip is in 3D space. 3D catheter tip position information is useful in some applications. For example, the catheter tip position can be directly visualized by data acquired from a 3D DynaCT device. The 3D DynaCT apparatus can perform angiographic computed tomography and supply CT-like images during angiographic imaging. If each image volume is registered, the tip can be displayed with volume rendering of CT data or MR data. When CT heart data or MR heart data is segmented, the tip can be seen in individual ventricles. Endoscopic observations that visualize fly-through observations from virtual perspective cameras inside the heart, or clipping views can be used.

  Using 3D catheter coordinates for electrophysiological mapping purposes, a geometric map for each individual ventricle can be generated. Such objectives can also include the objective of obtaining 3D position samples on the ventricular wall. This position can be used to approximate the ventricular wall by constructing a shell. In this way, the doctor roughly knows where he is after the shell is built. Another purpose is to measure the voltage and latency at one location on the ventricular wall so that the physician can find the shortest interval for applying the ablation. In addition, with 3D catheter coordinates, automatic operation can be performed using a remote control system within the 3D volume. The remote control system needs to obtain 3D position feedback in order to apply the appropriate force to move to the desired destination position.

  Existing techniques for locating catheters are based on magnetic or electric fields and require special and expensive catheters. With two x-ray images using only a regular catheter, the catheter position can be determined. FIG. 1 illustrates how the 3D tip position is calculated from the intersection of rays that pass through the tip in a fluoroscopic image. Two X-ray sources 11, 12 emit X-rays 13, 14 and pass through a volume region 10 including a catheter 15 having a tip 16. Images formed by these X-ray sources are displayed on the monitors 17 and 18 as shown in the figure, and a detector may be disposed here. These monitors display a 2D projection image of the catheter similar to that observed by each x-ray source. In this way, tip detection in fluoroscopic images assists in the realization of a 3D catheter position determination system.

  The object of the present invention is to improve the method of detecting the 3D position of a catheter.

The problem is
Providing a pair of two-dimensional fluoroscopic image series of catheters acquired from different viewing angles, each fluoroscopic image having a plurality of intensities defined by a 2D grid of pixels;
Segmenting one or more catheter regions in corresponding image pairs by threshold comparison of image intensity histogram gradients;
Removing a pixel within the catheter region of these pixels to generate a catheter contour;
Matching the catheter contour to a 2D template of the catheter and selecting a plurality of candidate catheter tip pixels;
Matching at least some catheter tip pixels of the candidate catheter tip pixels in the image pair to a 2D projection image of a 3D template of the catheter tip and selecting the best matching candidate pixel as the catheter tip;
The 3D position of the catheter tip is detected from the intersection of two rays, one of the two rays projecting from a selected pixel in one image of the pair of images, and of the two rays The other is solved by a method comprising projecting from related pixels in the other image of the image pair.

  Embodiments of the invention described herein generally include methods and systems for locating a catheter tip using biplane x-ray images in 3D space. The algorithm according to one embodiment of the present invention is based on a classifier cascade to quickly detect a catheter tip in a perspective image pair and reconstruct the tip position in 3D space. First, a simple classifier is applied to quickly erase many background pixels. If the simple classifier fails to select a suitable candidate, then the complex classifier is applied to a small number of pixels. Here, there are two template matching steps. The first template matching step uses a fixed 2D template to select a small number of best candidate pixels. The second template matching step uses a dynamic 2D template that is a projection image of the 3D template onto the image plane. The algorithm according to one embodiment of the present invention is robust against image noise. Image noise is suppressed by a preprocessing step, and moving structures are emphasized by temporary filtering. Such a 3D template matching step uses epipolar constraints to eliminate misdetections. An algorithm according to one embodiment of the invention can handle different gradients of the catheter tip. The appearance of the catheter tip in the two image planes depends on the direction of the catheter tip in 3D space. The 3D template matching step uses customized templates for the two image planes according to the C-arm geometry settings. The algorithm according to one embodiment of the present invention can uniquely detect the catheter tip even if another catheter is present in the image. Both 2D and 3D templates are built using catheter specifications. The catheter contour is detected in a preprocessing step and the catheter tip is found from the contour pixel. This detection process becomes more efficient when the algorithm rejects pixels inside the contour and focuses only on the contour pixels. Catheter positioning is applied in interventional imaging where the catheter tip is fused with pre-operative images and used to guide the catheter.

In one aspect of the present invention, the following method for locating the catheter tip in three-dimensional space is provided. That is,
Providing a pair of two-dimensional fluoroscopic image series of catheters acquired from different viewing angles, each fluoroscopic image having a plurality of intensities defined by a 2D grid of pixels;
Segmenting one or more catheter regions in corresponding image pairs by threshold comparison of image intensity histogram gradients;
Removing a pixel within the catheter region of these pixels to generate a catheter contour;
Matching the catheter contour to a 2D template of the catheter and selecting a plurality of candidate catheter tip pixels;
Matching at least some catheter tip pixels of the candidate catheter tip pixels in the image pair to a 2D projection image of a 3D template of the catheter tip and selecting the best matching candidate pixel as the catheter tip;
The 3D position of the catheter tip is detected from the intersection of two rays, one of the two rays projecting from a selected pixel in one image of the pair of images, and of the two rays The other having a step of projecting from the associated pixels in the other image of the image pair.

  According to another aspect of the invention, the method normalizes the size of the image and homogenizes the image background in each image of the image sequence pair.

In another aspect of the present invention, in the background homogenization, an average image is initialized as a first blurred image of one of the image sequences, and the average image is converted into the image sequence. over all images included _{in, M n = α · M n} -1 + is updated in accordance with _{(1-α) · I n} , E n = I n + min (I n -M n, 0) sensitized image from E Calculate _n . Here, In is the _nth image, _Mn is the nth average image, and α is a weighting factor.

  In another aspect of the invention, a bandpass filter is applied to the image in the background homogenization.

  In another aspect of the invention, the method estimates a normal direction along each catheter profile.

  In another aspect of the present invention, when matching each catheter contour to a 2D catheter template, the region of the image around the catheter contour is binarized, and the 2D catheter template is placed in an estimated normal direction along the catheter contour. Therefore, determine the direction and increase the score if each pixel in the 2D template and the overlapping pixels in the binarized image area are the same, decrease the score if the two pixels are different To calculate the cross-correlation of these two pixels and select the pixel with the best cross-correlation score.

  In another aspect of the invention, the 2D template is applied in a direction close to the estimated normal direction.

  In another aspect of the invention, when matching candidate catheter tip pixels to a 2D projection image of a 3D catheter template, a plurality of pre-computed 2D projection images of the 3D catheter template are combined into one of the images. When the candidate pixel included is applied to the pixel related to the candidate pixel included in the other image, and each pixel included in the 2D projection image and the overlapping pixel included in the candidate tip region are the same Increase the score and decrease the score if the two pixels are different to calculate the cross-correlation between the two pixels and select the pixel with the best cross-correlation score as the catheter tip.

  In another aspect of the present invention, a pixel in the other image associated with the candidate pixel is found by searching a region around the epipolar line of the candidate pixel in the other image.

  In another aspect of the invention, to pre-compute a 2D projection image of a 3D catheter template, the 3D catheter template is rotated 180 ° by a first angle from -y to y, and the 3D catheter template is -z Rotate 360 ° by the second angle starting from and store the 2D projection image in a look-up table.

  In another aspect of the invention, for a candidate pixel in one image, the first and second angles of the 2D projection image having the best cross-correlation score are detected, and the coordinate system of the one image is Converting to the image coordinate system, the corresponding first angle and second angle of the 2D projection image with respect to the other image are detected.

  In another aspect of the invention, the detected catheter tip is fused to an image rendered with volume reconstruction or multi-section reconstruction.

  In another aspect of the invention, if the two rays are not intersecting, the midpoint between these two rays at the location closest to each other is selected as the 3D tip position.

  In another aspect of the present invention, the following method for positioning the catheter tip in a three-dimensional space is proposed. That is, it provides a pair of processed two-dimensional fluoroscopic images of a catheter acquired from different viewing angles, which images have already been processed to segment the catheter at this point, The image area is binarized, the 2D catheter template is oriented according to the estimated normal direction of the catheter, and each pixel included in the 2D template is the same as the overlapping pixel included in the binarized image. Calculate the cross-correlation of these two pixels by increasing the score if they are different and decreasing the score if the two pixels are different, and select the pixel with the highest cross-correlation score as the candidate catheter tip pixel Select as a pre-computed 2D of 3D catheter template The image is applied to a candidate pixel in one of the images and a pixel included in the other image and related to the candidate pixel, and each pixel included in the 2D projection image overlaps a candidate tip region. Method of calculating cross-correlation between pixels, selecting the pixel with the best cross-correlation score as the catheter tip pixel and detecting the 3D position of the catheter tip from the 2D position of the catheter in each of the two images Propose.

  In another aspect of the invention, each image is obtained from an image sequence, each image having a plurality of intensities defined on a 2D grid of pixels.

  In another aspect of the invention, each image is preprocessed to standardize the size of the image and to homogenize the image background in each image in the sequence pair.

  In another aspect of the invention, the image intensity histogram gradient is thresholded during catheter segmentation.

  In another aspect of the invention, pixels within the catheter region are removed to generate a catheter contour.

  In another aspect of the invention, in detecting the 3D position of the catheter tip, the ray projected from the selected pixel in one image of the image pair and the associated pixel in the other of the image pair. Find the intersection with the projected ray.

  In another aspect of the present invention, a computer readable program storage device is provided. The program storage device implements a program of instructions executable by a computer to implement a method for locating a catheter tip in three-dimensional space.

  Embodiments of the invention described herein generally include methods and systems for detecting and locating a catheter tip in three dimensions from a two-dimensional perspective image. Accordingly, while the invention is susceptible to various changes and modifications, specific embodiments thereof are shown by way of example in the drawings and are described in detail below. However, the invention is not limited to the particular forms disclosed, but conversely encompasses all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. It should be understood.

  The term “image” or “image” as used herein refers to multidimensional data made up of discrete image elements (eg, pixels in 2D images, voxels in 3D images). The image may be, for example, a medical image of a subject collected by computed tomography, magnetic resonance imaging, ultrasound, or any other medical imaging system known to those skilled in the art. The images may also be provided from non-medical things such as remote sensing systems, electron microscopes, etc. Although an image can also be viewed as a function from R3 to R, the method of the present invention is not limited to such an image and can be applied to images of any dimension such as a 2D picture or 3D volume. . For 2D or 3D images, the region of the image is typically a 2D or 3D square array, and each pixel or voxel is specified with respect to two or three mutually orthogonal axes. Can do. The terms “digital” and “digitized” as used herein are suitably obtained in digital form or digitized form obtained via a digital acquisition system, or by conversion from an analog image. Refers to an image or volume in digital form or digitized form.

  FIG. 2 is a block diagram of an example system for catheter position determination according to one embodiment of the present invention. Referring to this figure, a pair 22 of fluoroscopic image series acquired from the online fluoroscopic apparatus 21 is input. The data of the fluoroscopic image series can be in DICOM format. A tracking algorithm 23 detects the catheter tip in the 2D fluoroscopic image sequence and uses the built-in 2D template and 3D template library 24 to determine the 3D position and 3D orientation of the catheter tip. The tracking software can batch process a plurality of fluoroscopic image pairs, and the detection results of each pair can be stored in a file. A separate image fusion component 25 can be used to visualize the detection results. This fusion component can be embodied as a plug-in, loading a pair of fluoroscopic image sequences and corresponding result files, and the catheter tip into 3D of volume data 26, for example DYNA CT data, CT data or MR data. It can be displayed together with a volume rendering display and an MPR display.

  The catheter detection algorithm according to one embodiment of the present invention has a cascading structure that is similar to the structure of a degenerate decision tree. Since the catheter tip is only a small part of the fluoroscopic image, the background pixels must be quickly discarded by an appropriate algorithm. To accomplish this, use a classifier series that returns T (true) for possible tip candidates or F (false) for non-catheter pixels. Reduce false positive rates by calling a more complex classifier after rejecting the majority of pixels using a simple classifier.

  An example structure of a detection cascade according to one embodiment of the invention is shown in FIG. The initial classifier 31 erases a large number of negative pixels with very little processing. The next layer 32, 33, 34 further erases the negative pixels, but here further calculations are required. This cascade structure reflects the fact that negative pixels are overwhelmingly predominant in any individual image. This cascade aims to reject as many negative pixels as possible in the very early stages.

  FIG. 4 is a flowchart of an example detection cascade for catheter tracking according to one embodiment of the present invention. As shown in this figure, the method according to this embodiment can be divided into five stages. Referring to this figure, the first stage 41 is a preprocessing stage. At this stage, for a given perspective image pair, the size of each perspective image is standardized to make the background uniform. The next step 42 is catheter area selection. In one pair of corresponding images, the catheter body pixels are segmented by a simple threshold comparison. As a result, on average, 96% or more of the pixels included in the image are removed. The catheter contour is segmented in each image in the image sequence pair, the outer boundary of the catheter body is extracted, and only about 2% of pixels remain until the next process. In order to select a candidate, a two-dimensional template matching is performed in step 43. From the list of candidates from each 2D image, four top candidate pixels are selected in each image of the image series pair by applying a 2D template that matches the catheter tip. In step 44, 3D template matching is performed. The top four candidates of one image of the image pair are compared with the series of 2D projection images of the 3D template, and the corresponding pixel included in the other image is matched along the epipolar line in the most matching 2D projection image. To explore. The best match is considered to be the catheter tip. This match results in only one pixel in each 2D image plane. From the pixels detected in the 2D image pair, the 3D position and 3D orientation of the catheter tip in the image pair can be calculated in step 45 from the biplane C-arm geometry. By locating the 3D tip for each corresponding image pair in the image sequence, the catheter tip can be tracked in 3D as the catheter tip moves. Details of each of the above steps are described below.

FIG. 15 shows the epipolar line referred to in step 44. The epipolar line is the line 158 in the second image formed by the intersection of the planes defined by the pixels 153 contained in the first image and the optical centers 151, 152 of the two images. For pixel p ₁ 153 of the first image with optical center 151, the 3D point 157 that optical center 151 observes is on line 155 that extends indefinitely to the far end, and in the second image There is also a line 156 extending from the corresponding point 154. The meaning of this epipolar line is that when observed in image 2, the 3D point observed at p ₁ should be on the epipolar line. This is known as the epipolar constant. Therefore, in the case of searching from a point on the second image corresponding to p ₁ it may be only the search epipolar instead of searching the entire image.

  Pre-processing: In the first stage 41, the two fluoroscopic image sequences are read from the DICOM file or read from the online device. In one embodiment of the invention, the image is converted to an 8-bit grayscale format and the DICOM image is cropped according to the collimator settings obtained from the DICOM header. The 8-bit format is an example, and other size bit formats are within the scope of embodiments of the present invention. The cropped image is scaled by the following factors:

scale = SID _ref / SID × p / p _ref × 0.5
Here, SID is the distance from the X-ray source to the image plane, p is the pixel spacing on the image plane, and SID _ref and p _ref are reference parameters. For different SID and p values, the size of the catheter tip appears different on different fluoroscopic images. In order to simplify subsequent template matching so that only one standard template is needed, the image size is standardized according to SID and p. A standard template according to one embodiment of the invention is generated at step 44 using the reference parameters SID _ref = 1050 mm and p _ref = 0.184 mm. Although the original image size here is 1024 × 1024 pixels, other embodiments of the invention can support different image sizes. In order to reduce the processing time and reduce the influence on the detection rate, the image is resized by 0.5 times.

In one embodiment of the invention, temporary enhancement is performed in the preprocessing. This temporary enhancement enhances dark structures that move in the sequence of image pairs while keeping other structures unchanged. This is illustrated in FIGS. 5 (a)-(b). FIG. 5A on the left is an original image, and FIG. 5B on the right is a temporarily emphasized image. The weighted average image M _n is updated in all frames included in one sequence. Initially, this average image M ₁ is simply a blurred image of the first frame contained in sequence I ₁ . The enhanced image is not brighter than the original image. If the pixel is darker than the average image, this intensity difference is subtracted from the original image. An example of emphasis is implemented as follows:
M _n = α · M _n−1 + (1−α) × I _n
M ₁ = blur (I ₁ )
E _n = I _n + min (I _n −M _n , 0)
Here, α = 0.8, and _En is an enhanced image. From experiments using phantom sequences, ie experiments using perspective image sequences of non-human phantom objects, the detection rate is slightly improved by temporary enhancement. This is because the moving catheter tip is emphasized.

  Another preprocessing technique is bandpass filtering. By applying a bandpass filter with strong high-pass characteristics, the difference between the background and the emphasized edge can be removed to facilitate catheter detection. A bandpass filter constructed from the difference between two lowpass filters has been experimentally demonstrated to have good performance. 6A and 6B, the original fluoroscopic image is shown in FIG. 6A on the left side, and the result of bandpass filtering is shown in FIG. 6B on the right side. This result has a more homogeneous background. This image filter can be embodied in the frequency domain to achieve good performance.

  Catheter region selection: In step 42, candidate catheter regions are segmented by threshold comparison. This assumes that the catheter appears darker than the bandpass filtered image. 7 (a)-(e) illustrate candidate pixel selection steps according to one embodiment of the present invention. An example of a bandpass filtered image is shown in FIG. 7 (a), and an intensity histogram of the image is shown in FIG. 7 (b). In the histogram of the filtered image shown in FIG. 7B, a clear separation between the catheter and the background cannot be confirmed, and a simple normal distribution can be confirmed. Therefore, standard separation methods that separate the foreground from the background do not appear to be successful. Finding a threshold based on the number of pixels selected is sensitive to the percentage that the catheter occupies the image. In one embodiment of the invention, the threshold can be selected by analyzing the slope of the histogram. FIG. 7C is a graph of the gradient of the histogram of FIG. A range of 5-30 pixels / intensity has been found suitable for selecting a threshold. In the gradient of FIG. 7 (c), a threshold of about 100 is selected corresponding to a gradient value of about 15. FIG. 7D shows a threshold image in which the white area includes three catheters and some background noise. Areas with a size smaller than 15 pixels are removed.

  After applying threshold comparison, in order to further reduce the size of the candidate catheter region, internal pixels can be removed to generate a boundary contour. This is because the catheter tip should be close to the border of the selected area. FIG. 7 (e) shows the contours generated from FIG. 7 (d), which are then analyzed by 2D template matching.

  In order to speed up the 2D template matching performed in the next step, the perpendicular direction of the contour is also estimated in this step. FIGS. 8A to 8C show the normal direction used for two-dimensional template matching according to one embodiment of the present invention. As shown in FIG. 8 (a), if the catheter tip direction on the 2D image is unknown, the 2D template 81 is rotated from 0 ° to 360 ° to show all possible directions for each contour pixel. Must be inspected every time. FIG. 8 (a) shows eight different directions. The normal direction is estimated at each contour pixel. This is indicated by an arrow in FIG. This direction search can be limited to a range in the vicinity of the perpendicular direction. Template matching need only be applied in the vicinity of the estimated normal direction. FIG. 8 (c) shows a template 82 that is tested in three directions with respect to the normal of the candidate catheter tip.

  2D template matching: 2D templates are constructed when the catheter is parallel to the image plane. To find the best match, 2D template matching is performed on each contour pixel in step 43. In order to do this, several directions are examined in the vicinity of the estimated contour normal direction. Depending on the dose of the fluoroscopic image and the anatomical structure being imaged, the background of the catheter tip and its surroundings in a 2D image often have different intensity values. This makes it difficult for 2D template matching to generate matching scores that can be compared across different fluoroscopic image sequences. In one embodiment of the present invention, matching is simplified by binarizing the image patch for one pixel of the image. This binarization is applied to the bandpass filtered image.

An example of a method for finding an optimum local threshold for binarizing an image patch is the Otsu's method. Otsu method is distributed between classes
By choosing, we choose the optimal threshold t ^* that separates the two Gaussians.
Here, L is the number of image gray levels,
Is defined as follows:

Here, mu _T is the mean intensity of the entire image, mu ₁ is the average intensity of one compartment, the mu ₂ is the mean intensity of the different compartments, the omega ₁ relative weight of the first compartment Ω ₂ is the relative weight of the second section. FIG. 9 shows an example of an image patch near the tip of the catheter that is closest to the result binarized by the Otsu method.

  A 2D template having the shape 100 shown in FIG. 10 is rotated around its tip. This is indicated by arrow 110 in the figure. A cross-correlation is calculated between the 2D template and pixels that overlap the 2D template in the binarized image patch in a certain predetermined direction, for example, every 15 ° rotation angle. Specifically, in a pixel included in the 2D template and a pixel overlapping the 2D template included in the binarized image patch, if the two pixels are the same (black or white), increment 1; A template matching score is calculated by decrementing 1 if the two pixels are different.

  For each image plane, four candidate pixels with the highest cross-correlation are selected. If the catheter is tilted in 3D, its tip may look very different from the 2D template. In this case, since the cross-correlation score is low, there is a possibility that the tip does not exist in any of the four candidate pixels. Most often, however, the tip appears vertical in at least one image plane and is present in eight candidate pixels belonging to both planes. The detection of tilted catheters can be handled by 3D template matching as described in the next step. If the catheter has blur due to motion or image noise, the exact catheter tip may not be present in the candidate pixel. FIGS. 11A to 11B show selection of four tip candidates. FIG. 11A shows a plurality of candidate pixels, which are indicated by arrows pointing outward from the contour. FIG. 11 (b) shows only four arrows corresponding to the pixel with the highest cross-correlation score.

  3D template matching: In many cases, the 2D projection image of the catheter tip often looks different from the 2D template. However, according to the projection geometry, the shape of the catheter tip in one image plane is related to the shape in another image plane. 2D template matching does not consider such a relationship. In the 3D template matching step of Step 44, two 2D templates are generated by spatially rotating the 3D template and projecting it onto two image planes. These two customized 2D templates are used to find the best match among the top candidates selected in the 2D template matching step.

  The 3D template can be precomputed before the perspective image is loaded or processed. The 3D template is first constructed based on catheter specifications, such as the diameter, length, etc. of the ablation tip, or is constructed based on image reference parameters as described for the preprocessing step, or between the electrodes. Constructed based on the distance. In addition, in this template construction, it is necessary to take into consideration that features that are not in the specification may appear in the 2D projection image on the perspective image. FIGS. 12A to 12C show 3D template construction images according to the embodiment of the present invention. FIG. 12A shows a 2D projection image of the catheter in a fluoroscopic image, and FIG. 12B shows the catheter as in the specification. The length and diameter of the thin portion between the cutting tip and the second electrode shown in FIG. 12A may not be in the specification. FIG. 12 (c) shows a 3D catheter template containing this information.

  From this 3D template, a series of 2D templates is generated. When a fluoroscopic image sequence is acquired in advance, the template is generated after the fluoroscopic image sequence is loaded. This is because the acquisition angle used to associate two images to perform 3D template matching is obtained from DICOM header information. If the image is acquired online in real time, the acquisition angle is known before the image is acquired. These 2D templates are pre-computed and the generation algorithm requires a 2D template corresponding to a specific 3D direction instead of calculating a projected image at each time point. To pre-calculate the 2D template, the 3D template is rotated and sampled with spherical coordinates.

FIGS. 13A to 13B show the projection geometry of the 3D template. FIG. 13A shows a C-arm system provided with respect to one image plane for the phantom patient 131 on the table. When the primary angle α and the secondary angle γ are 0, the X-ray source tube 130 is below the table and emits X-rays upward. A given 3D position in the world coordinate system indicated by directional arrow 132 is referred to as an isocenter, and the projected image of that 3D position on the image plane is represented as follows:

line; queue; procession; parade
Is a combination of series of transformations.

Here, T _d is a parallel progression moving the local coordinate system from the isocenter to the X-ray source, and the SOD distance on the y-axis is:

R _x is a matrix that rotates the local coordinate system by 90 ° about the x axis.

R _z is a matrix that rotates the local coordinate system 180 ° about the z axis.

P is a projection matrix.

SID is the distance to the image plane from the x-ray source, p is the pixel spacing, o _x and o _y are the pixel location representing the X-rays transmitted through the isocenter.

The image plane A is obtained by rotation R (α, γ) by α and γ, which are arbitrary angles. Here R (α, γ) is as follows:

The entire transformation can be described as follows:
In plane A,
In plane B,

Here TA _{→ B} is a 4 × 4 transformation from the 3D coordinate system of plane A to plane B, which compensates for isocenter shift.

Since the relationship between the A and B coordinate systems is unknown, the least squares method can be used to calculate this relationship. For the landmark projection matrices and pixel positions in the two images belonging to plane A, the 3D position of the landmark in the plane A coordinate system is { _pi }, i = 1, 2,. . . , N. Here, N is the number of landmarks. Similarly, in the plane B, the 3D position of the landmark in the coordinate system of the plane B is represented by {p ′ _i }. Project {p _i } to {p ′ _i } by rigid body transformation of the coordinate system of planes A and B.

p ′ _i = Rp _i + T + n _i ,
R is a rotation matrix of 3 × 3, T is the translation vector 3 × 1, n _i is the noise vector of 3 × 1. R and T can be determined by the least squares problem. Examples of functions that should be minimized include:

The coefficient w _i is a weight for each correspondence. After this is determined for R and T, the matrix TA _{→ B} can be constructed.

The least square is obtained as follows. First, as the centroids of {p _i } and {p ′ _i }
Define These are calculated by the following formula:
and
3 × 3 matrix H is calculated.

The superscript t represents matrix transposition. Next, find the singular value decomposition of H, H = UDVt. Here, U and V are 3 × 3 orthogonal matrices, and D is a diagonal matrix. Thereafter, R is calculated as R = VU _t , and the determinant | R | is calculated. If | R | = 1, the rotation matrix will be calculated correctly. If | R | = −1, then R is a mirrored transformation. If one of the singular values, for example λ ₃ is 0, the correct solution for the rotation matrix should be R = V′U _t . Here, V 'is obtained by changing the sign of the third column of V. If none of the singular values of H is 0, then an appropriate solution cannot be obtained, and additional data is needed to find a better solution. Assuming that a solution exists, T is
Can be calculated as

Note that the image shown on the screen is mirrored in the u-axis direction. The coordinates of the mirror-transformed screen are as follows.

  FIG. 13 (b) shows the geometry of the sampling process. During this sampling process, the angle roll is sampled starting at -y in the xy plane, ending at y, and the angle pitch is rotated 360 ° from -z. In one non-limiting embodiment of the present invention, there are 36 samples from 0 ° to 180 ° with respect to roll. In another non-limiting embodiment of the present invention, on the equator, the pitch is sampled approximately every 15 °, and the sampling rate decreases as it approaches two poles on the y-axis and the −y-axis. For each sample in a particular direction, the angle roll and pitch are stored along with the 3D direction and 3D template projection.

During the actual template matching process, for a given roll and pitch of a sample of image plane A, the corresponding roll and pitch of image plane B can be determined. The image plane A samples are associated with the image plane B samples using a bidirectional look-up table. For the direction of the image plane A (d _x , _dy , d _z ), the roll and pitch in the image plane B can be calculated using the following equations:

First, (d _x , _dy , d _z ) is converted into the world coordinate system.

d _xw = d _x
d _yw = −d _z
d _zw = −d _y
Thereafter, (d _xw , d _yw , d _zw ) is converted into the coordinate system 133 of the image plane B.

(D ′ _xw , d ′ _yw , d ′ _zw ) = R _{A → B} × (d _xw , d _yw , d _zw ),
R _{A → B} is a 3 × 3 rotation matrix for converting the image plane A to the image plane B from the above.

Finally, (d ′ _xw , d ′ _yw , d ′ _zw ) is transformed into the coordinate system defined by the 3D template projection geometry.

d ′ _x = d ′ _xw
d ′ _y = −d ′ _zw
d ′ _z = −d ′ _yw
The angle in the image plane B is obtained by the following mathematical formula.

roll ′ = cos ⁻¹ (−d ′ _y )
pitch ′ = tan ⁻¹ (−d ′ _x / d ′ _z )

  In one embodiment of the present invention, 3D template matching is performed on four top candidates among the eight candidates obtained from the preceding 2D template matching. This number of candidates is an example, and does not limit the present invention. In another embodiment of the present invention, template matching can be performed on a different number of candidates. Each candidate pixel has a corresponding pixel in the other image. This pixel can be found by scanning the area around the epipolar line of the candidate pixel. Four candidates from one image plane can also be used. FIGS. 14A to 14B show how to search for a corresponding pixel in the contour pixel defined by the shaded band along the epipolar line. FIG. 14 (a) shows two candidate pixels, and FIG. 14 (b) shows a corresponding shaded band-like portion concentrated on the epipolar line and corresponding to each candidate pixel. In one embodiment of the invention, the width of the band is 30 pixels at its native resolution, which corresponds to about 15 pixels in the cropped and resized image.

  Template matching is performed on each candidate pixel and the corresponding pixel on the other image plane using templates projected from different directions. Use all template pairs included in the precomputed lookup table. If a candidate pixel has the highest template matching score, it is considered to be the catheter tip. The 3D direction of the catheter tip is determined from the template that has reached the highest score.

  3D reconstruction: In step 45, the 3D position of the tip is determined. FIG. 16 shows the calculation of the 3D position of the catheter tip. Referring now to FIG. 16, after detecting the catheter tip in both images 161, 162, the 3D position of the catheter tip is determined from the intersection 165 of two rays 163, 164 extending infinitely from the pixels in each plane. Can be sought. Due to calibration and detection errors, these rays do not intersect exactly in 3D space, as shown in image 166. In one embodiment of the invention, the catheter position is defined as the midpoint 167 between the two rays where the two rays are closest.

  Image fusion: The detected catheter tip can be fused with 3D volume and MPR rendering. FIG. 17 shows an example of image integration with DynaCT data. The upper two observation planes show biplane perspective images. Arrows 171 and 172 indicate the 2D direction of the catheter tip, and the base point of the arrow is at the catheter tip. The lower left display surface is a superposition of the catheter tip and the MRP image. The lower right display surface displays a 3D model of the catheter tip with volume rendering of DynaCT data. Here, it is assumed that image registration is not necessary because the object does not move between DynaCT reconstruction and fluoroscopic image acquisition. The object moves when the 3D volume is acquired from pre-operative CT or MR data, but if the object moves, image registration must be performed to enable image fusion. I must.

  An algorithm according to another embodiment of the invention can be applied to a monoplane system. By positioning a single plane once at an angle, the catheter can be detected in 3D using the method described above. In this case, the catheter must be kept in a steady position while the monoplane device acquires images from two different angles.

  It should be noted that the parameters described in the present disclosure are useful for specific test images. These parameters are examples and do not limit the invention, and in other embodiments of the invention, the parameter values can be adjusted for different images having different image sizes and contrasts.

  System Configuration: It should be understood that embodiments of the present invention can be implemented in various forms of hardware, software, firmware, special purpose processing, or combinations thereof. In one embodiment, the present invention can also be embodied in software as an application program implemented on a computer readable program storage device. This application program can be uploaded to and executed by a machine with any suitable architecture.

  FIG. 18 is a block diagram of an example computer system for implementing a catheter tip position determination method according to one embodiment of the present invention. Referring now to FIG. 18, a computer system 181 for implementing the present invention includes, among other things, a central processing unit (CPU) 182, a memory 183, and an input / output (I / O) interface 184. The computer system 181 is generally connected to a display 185 and various input devices 186 such as a mouse and keyboard via an I / O interface 184. The auxiliary circuit may include a circuit such as a cache, a power supply, a clock circuit, and a communication bus. Memory 183 may include random access memory (RAM), read only memory (ROM), disk drive, tape drive, etc., or combinations thereof. The present invention can be implemented as a routine 187 stored in memory 183 and executed by CPU 182 that processes signals from signal source 188. Further, although the computing device 181 is a general-purpose computer system, when the routine 187 of the present invention is executed, the computer 181 also becomes a computer system for specific applications.

  Computer system 181 also has an operating system and microinstruction code. The various processes and functions described herein may be part of the microinstruction code or part of an application program (or combination thereof) that is executed via the operating system. . In addition, various other peripheral devices may be connected to the computer platform such as an additional data storage device and a printing device.

  Further, since some of the component system components and method steps depicted in the accompanying drawings can be implemented as software, the actual connection between system components (or process steps) is It can vary depending on how you do it. If the idea of the present invention disclosed in the present specification is provided, those skilled in the art can deduce similarities to the embodiments or configurations of the present invention.

  Although the invention has been described in detail with reference to advantageous embodiments, those skilled in the art will recognize the invention without departing from the spirit and scope of the invention as set forth in the appended claims. It will be understood that various changes and substitutions can be made.

1 is a schematic diagram of an apparatus for performing X-ray based catheter positioning according to one embodiment of the present invention. FIG. 1 is a block diagram of an example system for performing catheter position determination according to one embodiment of the present invention. FIG. FIG. 4 illustrates an example structure of a detection cascade according to one embodiment of the invention. FIG. 4 is a flowchart of an example detection cascade for performing X-ray based catheter tracking according to one embodiment of the present invention. FIG. 6 illustrates the effect of a temporal image that has been enhanced and preprocessed, according to one embodiment of the present invention. Fig. 6 illustrates the effect of bandpass filter pre-processing according to one embodiment of the invention. Fig. 4 illustrates a candidate pixel selection image according to one embodiment of the present invention. Fig. 5 shows a normal direction in two-dimensional template matching according to one embodiment of the present invention. Fig. 4 shows an example of image patches near the catheter tip according to an embodiment of the present invention, and an example of an image obtained by binarizing these image patches. FIG. 6 shows an image of rotation of a 2D template about a catheter tip according to one embodiment of the present invention. Fig. 4 shows an image of selection of top candidates from contour pixels according to one embodiment of the invention. Fig. 3 shows a 3D template construction image according to one embodiment of the invention. Fig. 4 illustrates an example of a 3D template projection geometry according to one embodiment of the present invention. FIG. 4 shows a search for related pixels near the epipolar line according to one embodiment of the invention. FIG. Fig. 2 shows an epipolar line definition according to one embodiment of the invention. FIG. 6 shows positioning of the catheter tip in 3D according to one embodiment of the present invention. FIG. 5 illustrates catheter tip image fusion to DynaCT data according to one embodiment of the present invention. FIG. 1 is a block diagram of an example computer system for implementing a catheter tip position determination method according to one embodiment of the present invention. FIG.

Explanation of symbols

10 volume region 11,12 X-ray source 13,14 X-ray 15 catheter 16 catheter tip 17,18 monitor 130 X-ray source tube 131 phantom patient 132 isocenter 151,152 optical center 161,162 image 167 intermediate between two rays point

Claims (32)

In a method for locating a catheter tip in a three-dimensional space,
Providing a pair of two-dimensional fluoroscopic image series of catheters acquired from different viewing angles, each fluoroscopic image having a plurality of intensities defined by a 2D grid of pixels;
Segmenting one or more catheter regions in corresponding image pairs by threshold comparison of image intensity histogram gradients;
Removing a pixel within the catheter region of these pixels to generate a catheter contour;
Matching the catheter contour to a 2D template of the catheter and selecting a plurality of candidate catheter tip pixels;
Matching at least some candidate catheter tip pixels of the candidate catheter tip pixels in the image pair to a 2D projection image of a 3D template of the catheter tip and selecting the best matching candidate catheter tip pixel as the catheter tip; ,
The 3D position of the catheter tip is detected from the intersection of two rays, one of the two rays projecting from a selected pixel in one image of the image pair, and the other ray is the Projecting from related pixels in the other image of the image pair.   The method of claim 1, wherein the size of the image is standardized and the image background is homogenized in each image of the image series pair. When homogenizing the image background,
Initializing an average image as a blurred first image of one of the image sequences;
The average image over all images in the image sequence,
M _n = α · M _n−1 + (1−α) · I _n
Update according to
Here is the I _n the n-th image, M _n is the n-th average image, alpha is a weighting factor,
Enhanced image _{E n,} calculated from _{E n = I n + min (} I n -M n, 0), The method of claim 2 wherein.   The method of claim 2, wherein a band pass filter is applied to the image in homogenizing the image background.   The method of claim 1, wherein a normal direction along each catheter profile is estimated. When matching each catheter profile to a 2D catheter template,
Binarize the image area around the catheter contour;
Directing the 2D catheter template according to an estimated normal direction along the catheter contour;
Increase the cross-correlation score if each pixel in the 2D catheter template is the same as the overlapping pixel in the binarized image region and decrease the cross-correlation score if both pixels are different 6. The method of claim 5, wherein the cross-correlation of both pixels is calculated to select the pixel with the best cross-correlation score.   The method of claim 6, wherein the 2D template is applied in a direction close to the estimated normal direction. In matching candidate catheter tip pixels to a 2D projection image of a 3D catheter template,
Applying a plurality of pre-computed 2D projection images of the 3D catheter template to the candidate catheter tip pixel in one of the images and to a pixel present in the other image and corresponding to the candidate catheter tip pixel; ,
If each pixel in the 2D projection image is the same as the overlapping pixel that exists in the candidate catheter tip region, the cross-correlation score is increased. The method of claim 1, wherein cross-correlation between pixels is calculated and the pixel with the best cross-correlation score is selected as the catheter tip.   9. The method of claim 8, wherein a pixel in the other image corresponding to the candidate catheter tip pixel is found by searching a region around the epipolar line of the candidate catheter tip pixel in the other image. In order to pre-calculate a 2D projection image of the 3D catheter template,
Rotating the 3D catheter template 180 ° by a first angle from -y to y;
9. The method of claim 8, wherein the 3D catheter template is rotated 360 [deg.] By a second angle starting from -z and the 2D projection image is stored in a look-up table. Determining the first and second angles of the 2D projection image by the best cross-correlation score of the candidate pixels in one image;
The corresponding first angle and second angle of the 2D projection image with respect to the other image are obtained by converting the coordinate system of the one image into the coordinate system of the other image. Method.   The method of claim 1, wherein the detected catheter tip is fused to a volume rendered or multi-tomographic reconstruction rendered image.   The method of claim 1, wherein if the two rays do not intersect, a midpoint between the two rays at a location where the two rays are closest to each other is selected as a 3D tip position. In a method for locating a catheter tip in a three-dimensional space,
Providing a pair of processed two-dimensional fluoroscopic images of the catheter acquired from different viewing angles, each of the fluoroscopic images being already processed to segment the catheter at this point; ,
Binarizing the image area around the catheter and directing a 2D catheter template according to the estimated normal direction of the catheter;
Increase the cross-correlation score if each pixel included in the 2D template is the same as the overlapping pixel included in the binarized image area, decrease the cross-correlation score if both pixels are different Calculating the cross-correlation of both pixels and selecting the pixel with the best cross-correlation score as the candidate catheter tip pixel;
Applying a plurality of pre-computed 2D projection images of a 3D catheter template to candidate pixels in one of the images and to pixels associated with the candidate pixel in the other image;
Calculating a cross-correlation between each pixel included in the 2D projection image and an overlapping pixel included in the candidate tip region and selecting the pixel having the best cross-correlation score as the catheter tip pixel;
Detecting the 3D position of the catheter tip from the 2D position of the catheter in each of the two images. Acquire each image from the image series,
The method of claim 14, wherein each image includes a plurality of intensities defined on a 2D grid of pixels.   The method of claim 15, wherein each image is preprocessed to normalize the size of the image and to homogenize the image background in each image of the image series pair.   15. The method of claim 14, wherein the step of segmenting the catheter compares the image intensity histogram gradient with a threshold.   15. The method of claim 14, wherein pixels within the catheter area are removed to generate a catheter profile. Detecting the 3D position of the catheter tip,
The method of claim 14, wherein an intersection of a ray projected from a selected pixel in one image of the image pair and a ray projected from a corresponding pixel in the other image of the image pair is found. In a computer readable program storage device,
Implementing a computer-executable program of instructions for performing the steps of the method of locating the catheter tip in three-dimensional space;
The method
Providing a series of two-dimensional fluoroscopic image series of catheters acquired from different viewing angles, each fluoroscopic image having a plurality of intensities defined by a 2D grid of pixels;
Segmenting one or more catheter regions in corresponding image pairs by threshold comparison of image intensity histogram gradients;
Removing pixels within the catheter region to generate a catheter contour;
Matching the catheter contour to a 2D template of the catheter and selecting a plurality of candidate catheter tip pixels;
Matching at least some candidate catheter tip pixels of the candidate catheter tip pixels in the image pair to a 2D projection image of a 3D template of the catheter tip and selecting the best matching candidate pixel as a catheter tip;
The 3D position of the catheter tip is detected from the intersection of two rays, one of the two rays projecting from a selected pixel in one image of the image pair, the other ray is the Projecting from an associated pixel in the other image of the image pair.   21. The computer readable program storage device of claim 20, wherein the method normalizes the size of the image and homogenizes the image background in each image of the image series pair. When homogenizing the image background,
Initializing an average image as a blurred first image of one of the image sequences;
The average image over all images in the image sequence,
M _n = α · M _n−1 + (1−α) · I _n
Update according to
Here is the I _n the n-th image, M _n is the n-th average image, alpha is a weighting factor,
An enhanced image _{E n, E n = I n} + min (I n -M n, 0) is calculated from the computer-readable program storage device of claim 21.   The computer readable program storage device of claim 21, wherein a band pass filter is applied to the image when homogenizing the image background.   21. The computer readable program storage device of claim 20, wherein the method estimates a normal direction along each catheter profile. When matching each catheter profile to a 2D catheter template,
Binarize the image area around the catheter contour;
Directing the 2D catheter template according to an estimated normal direction along the catheter contour;
Increase the cross-correlation score if each pixel in the 2D template is the same as the overlapping pixel in the binarized image region, and decrease the cross-correlation score if both pixels are different 25. The computer readable program storage device of claim 24, wherein: calculating a cross-correlation of both pixels and selecting a pixel having the best cross-correlation score.   26. The computer-readable program storage device according to claim 25, wherein in the method, the 2D template is applied in a direction close to the estimated normal direction. In matching candidate catheter tip pixels to a 2D projection image of a 3D catheter template,
Applying a plurality of pre-computed 2D projection images of the 3D catheter template to the candidate catheter tip pixel in one of the images and to a pixel present in the other image and corresponding to the candidate catheter tip pixel; ,
If each pixel in the 2D projection image is the same as the overlapping pixel that exists in the candidate catheter tip region, the cross-correlation score is increased. 21. The computer readable program storage device of claim 20, wherein the cross correlation between pixels is calculated and the pixel with the best cross correlation score is selected as the catheter tip.   28. The computer-readable device of claim 27, wherein a pixel in the other image that corresponds to the candidate catheter tip pixel is found by searching a region around the epipolar line of the candidate catheter tip pixel in the other image. Program storage device.   To pre-compute a 2D projection image of the 3D catheter template, the 3D catheter template is rotated 180 ° by a first angle from -y to y, and a second angle starting from -z 28. The computer readable program storage device of claim 27, wherein the computer stores the 2D projection image in a look-up table rotated 360 °. In the method, the first and second angles of the 2D projection image are determined by the best cross-correlation score of the candidate pixels in one image;
30. The corresponding first angle and second angle of the 2D projection image with respect to the other image are determined by transforming the coordinate system of the one image into the coordinate system of the other image. Computer readable program storage device.   21. The computer readable program storage device of claim 20, wherein in the method, the detected catheter tip is fused to a volume rendered or multi-tomographic reconstruction rendered image.   21. The computer readable program storage of claim 20, wherein if the two rays do not intersect, a midpoint between the two rays at a location where the two rays are closest to each other is selected as a 3D tip position. apparatus.