US20110184238A1 - Image-based global registration system and method applicable to bronchoscopy guidance - Google Patents
Image-based global registration system and method applicable to bronchoscopy guidance Download PDFInfo
- Publication number
- US20110184238A1 US20110184238A1 US13/016,093 US201113016093A US2011184238A1 US 20110184238 A1 US20110184238 A1 US 20110184238A1 US 201113016093 A US201113016093 A US 201113016093A US 2011184238 A1 US2011184238 A1 US 2011184238A1
- Authority
- US
- United States
- Prior art keywords
- branch
- view
- search
- bronchoscope
- airway
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/267—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the respiratory tract, e.g. laryngoscopes, bronchoscopes
- A61B1/2676—Bronchoscopes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/754—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries involving a deformation of the sample pattern or of the reference pattern; Elastic matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
- G06V20/647—Three-dimensional objects by matching two-dimensional images to three-dimensional objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
- G06V20/653—Three-dimensional objects by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
Definitions
- This invention relates generally to bronchoscopy and, in particular, to a global registration system and method useful in image-based bronchoscopy guidance systems and other applications, including other types of endoscopic procedures.
- Bronchoscopy is a medical procedure commonly used in lung-cancer assessment[1].
- Lung-cancer assessment involve two main stages[2,3,4]: 1) three-dimensional (3D) multi-detector computed-tomography (MDCT) image assessment; and 2) live bronchoscopy.
- MDCT assessment the physician uses the two-dimensional (2D) transverse slices obtained from the patient's MDCT scan to identify specific diagnostic regions of interest (ROIs), such as lymph nodes and suspicious nodules[5,6,7].
- ROIs diagnostic regions of interest
- the physician also identifies the closest route to each ROI and mentally plans a 3D route using the 2D slices.
- bronchoscopy the physician maneuvers a flexible bronchoscope through the lung airways towards each ROI along the pre-planned mentally defined route. This is done by identifying the bifurcations along the route on the live endoluminal video feed obtained from the bronchoscope. This manually-based route planning procedure proves to be challenging, resulting in errors in bronchoscopy as early as the second airway generation[8,9].
- bronchoscopy guidance systems enable more accurate bronchoscopy[4,10,11,12,13]. These systems are motivated by virtual bronchoscopy (VB), wherein the 3D MDCT image of a person's chest serves as a “virtual environment”[14,15,16].
- VB virtual bronchoscopy
- a software-defined virtual camera navigates through the lungs in the virtual environment and presents endoluminal renderings of the 3D data, also known as VB images.
- all bronchoscopy guidance systems rely on some method for registration of the real bronchoscope in 3D surgical space to the 3D MDCT virtual space. Based on the type of sensor used for registration, bronchoscopy guidance systems can be either electromagnetic (EM) or image-based[3,4,11,12,13,15,16,17,18].
- An EM-based guidance system consists of the following: 1) an EM field generator; 2) a steerable EM probe; and 3) guidance software[12,13,17].
- the EM field generator generates an EM field around the patient's chest.
- the steerable EM probe is inserted through the working channel of the bronchoscope and tracked in the external EM field. Prior to the start of bronchoscopy, the steerable probe is used to calibrate and synchronize the coordinate system of the external EM field and the MDCT coordinate system. Thus, as the EM probe is tracked during bronchoscopy, its position in the MDCT coordinate system becomes nominally known. Such a system allows for immediate establishment of the global position of the bronchoscope tip within the 3D MDCT coordinate system.
- Image-based bronchoscopy guidance systems rely on volume-rendered[16,23] or surface-rendered[3,4,24,25] endoluminal images of the airway tree from the 3D MDCT scans in order to establish the location of the bronchoscope. This is generally done by comparing the VB images with the real bronchoscopic (RB) video frames. Weighted normalized sum of square difference errors (WNSSD)[24] and normalized mutual information (NMI)[3,4,26] are metrics that are used for comparing the images obtained from the two sources. Registration is carried out using Powell's optimization, simplex or gradient methods. The image-based bronchoscopy guidance methods rely on local registrations at bifurcations and so are less susceptible to patient breathing motion.
- Global registration is used in various fields such as image fusion[27, 28], remote sensing[29,30], object recognition[31], and robotic navigation[32].
- the problem of establishing the global position in robotic navigation is most similar to global registration in the domain of image-based computer-guided bronchoscopy.
- robotic navigation global registration is also referred to as the “robot kidnapping problem,” wherein the position of a robot has to be estimated when it is moved to any arbitrary pose and no motion estimates are available[32].
- Moreno et al. presented a non-linear filter, termed evolutive localization filter, that uses raw sensor data and recursively estimates the current pose[32].
- Other methods utilizing multi-hypothesis Kalman filters[33, 34], grid-based probabilistic filters[35] and Monte-Carlo localization[36] methods have also been used for addressing the problem of global registration in robot navigation.
- Zhang et al. have described an adaptive region-intensity-based ultrasound and computed tomography registration[37].
- Munim et al. used vector distance functions for registering magnetic resonance (MR) images of multiple patients[38].
- Moghari et al. have described a global registration method for aligning multiple bone fractures to a statistical anatomical atlas model[39].
- Principal component analysis and the unscented Kalman filter were used for local and global registration, respectively.
- Fookes et al. have also described a method for registration of multiple MR images from the same patient by formulating the problem as the minimization of a covariance weighted non-linear least square function[40].
- This invention resides in a global registration system and method useful in image-based bronchoscopy guidance systems and other applications, including other types of endoscopic procedures. Synchronization is restored by identifying the current branch position, thereby facilitating global, technician-independent bronchoscopy guidance without the need for any external device such as an electromagnetic sensor.
- Virtual bronchoscopy (VB) renderings of a 3D airway tree are obtained, these including VB views of branch positions within the airway tree.
- At least one real bronchoscopic (RB) video frame is received from a bronchoscope inserted into the airway tree.
- An algorithm according to the invention is executed on a computer to identify the several most likely branch positions having a VB view closest to the received RB view, and the 3D position of the bronchoscope within the airway tree is determined in accordance with the branch position identified in the VB view.
- the algorithm may use various techniques to speed operation, including an intra-branch search followed by an inter-branch search.
- a fast search may be carried out over all the branches in a global airway-bifurcation search space, with a weighted normalized sum of squares distance metric being used to determine the best match.
- the intra-branch search uses pre-computed lumen region enclosing rectangles in conjunction with a fast local registration refinement.
- the intra-branch search is given by:
- ⁇ ⁇ b i arg ⁇ ⁇ max ⁇ ⁇ b i ⁇ C ⁇ ( I V , I CT ⁇ )
- the inter-branch search is given by:
- ⁇ ⁇ o arg ⁇ ⁇ min ⁇ ⁇ K ⁇ ⁇ b i ⁇ D ⁇ ( I V , I CT ⁇ )
- Results show that the algorithm currently needs 3 seconds per search-space branch, with a trend of improving accuracy with reducing search-space being seen.
- FIGS. 1A and 1B show bronchoscopic video frame distortion correction
- FIGS. 2A-2F show examples of the 3D airway tree and the virtual bronchoscope
- FIG. 3 illustrates a pre-computation for intra-branch search
- FIG. 4A shows an input RB frame
- FIGS. 4B-4F show the VB images associated with the best bronchoscope position as found by the intra-branch search in five different branches of the search space;
- FIGS. 5A-5E show input bronchoscopic video frames for airway phantom
- FIGS. 5F-5J show VB images at the bronchoscope position estimated by the algorithm for airway phantom.
- An image-based bronchoscopy guidance system provides discrete guidance at bifurcations.
- multi-detector computed-tomography MDCT co-ordinate system we model the image seen by the bronchoscope by using 3D MDCT data.
- FIGS. 1A and 1B show bronchoscopic video frame distortion correction.
- FIG. 1A shows an example of a bronchoscopic video frame with barrel distortion
- FIG. 1B shows the bronchoscopic video frame after distortion correction.
- the airway tree is automatically segmented using a robust segmentation algorithm [43].
- the marching cubes algorithm is run on the 3D segmentation to obtain polygonal surface representation of the airway tree [44].
- the virtual bronchoscope is designed as a pinhole camera model with a focal length of f and the same image dimensions as the bronchoscope camera.
- the endoluminal airway surfaces are assumed to be Lambertian, with the light source at the focal point. Secondary reflections are disregarded.
- the VB view is rendered using OpenGL[45] as shown in FIGS. 2A-2F . Virtual bronchoscopic views.
- FIGS. 1 Virtual bronchoscopic views.
- FIGS. 2D-2F show examples of the 3D airway tree and the virtual bronchoscope (yellow cylinder+graphical needle) positioned within the trachea, left main bronchus and the right main bronchus (shown in FIGS. 2D-2F ) and the corresponding VB views.
- the virtual bronchoscope yellow cylinder+graphical needle
- Global registration can be defined as establishing the current branch position of the bronchoscope in the 3D airway tree.
- RB bronchoscopic
- VB virtual bronchoscopy
- the global registration algorithm finds the branch that contains the viewpoint with a VB view closest to the given RB view.
- MAP Maximum A Posteriori
- the first part is probability density estimation problem, where we estimate the posterior density over the space of available bronchoscope poses given the incoming bronchoscopic video.
- This problem formulation is similar to that by Moreno et al [32].
- an iterative solution is used for global registration but we present a method for registration using a single frame which is similar to the method by Wei ⁇ et al [46].
- the global registration problem is an optimization problem where we estimate the branch in which the bronchoscope gives a pose that maximizes the posteriori probability density
- ⁇ ⁇ arg ⁇ ⁇ max ⁇ ⁇ K tree ⁇ p ⁇ ( ⁇
- C(•,•) is a similarity function that gives the measure of similarity between the bronchoscopic video frame I V and I CT ⁇ the rendering obtained at pose ⁇ . If the similarity function C(•,•) is replaced by a dissimilarity measure, the global registration problem becomes
- ⁇ ⁇ arg ⁇ ⁇ min ⁇ ⁇ K tree ⁇ D ⁇ ( I V , I CT ⁇ ) ( 5 )
- D(•,•) is a dissimilarity function between the RB image I V and the VB view I CT ⁇ at view point ⁇ and K tree is the search space for the view points.
- the optimum branch is given by
- B(•) is a function that finds the branch containing the view point ⁇ circumflex over ( ⁇ ) ⁇ .
- the proposed algorithm accepts as input one or more live bronchoscopic video frames and outputs the 3D position of the bronchoscope within the airway tree.
- the algorithm is invoked during live bronchoscopy when the position of the bronchoscope is unknown.
- the algorithm will then determine the bronchoscope position within the airway tree.
- the physician positions the bronchoscope to give a good view of an airway bifurcation, so that the lumen region is well represented, as shown in FIG. 4 .
- Multiple runs of the algorithm on either the same bifurcation or at related bifurcations (parent-daughter branches) can be used to improve the overall accuracy of the method.
- the algorithm broadly divides into two major stages: intra-branch search and inter-branch search.
- FIG. 3 illustrates a pre-computation for intra-branch search.
- This figure shows the pre-computation carried out for the branch highlighted by the blue line.
- the red lines are the pre-defined centerline navigation routes through the 3D airway tree.
- the green triangles highlight three of the pre-defined view-sites (A, B, C) along the selected branch.
- the figures to the right of the view-sites are the VB renderings (A 1 , B 1 , C 1 ), the lumen thresholded images (A 2 , B 2 , C 2 ) and the minimum enclosing rectangles (A 3 , B 3 , C 3 ) associated with the respective view-sites.
- the longer length of the minimum enclosing rectangle is saved during pre-computation and later used for localizing the position of the given RB frame in a given branch.
- the intra-branch search is formulated as
- ⁇ ⁇ b i arg ⁇ ⁇ max ⁇ ⁇ b i ⁇ C ⁇ ( I V , I CT ⁇ ) ( 7 )
- the intra-branch search carries out a fast scan through all the pre-defined view sites along the centreline of the branch.
- the second step comprises render position adjustment using fast local registration. Because of uncertainty in the roll angle of the bronchoscope we use a set of four likely positions for each branch using different roll angles.
- the minimum enclosing rectangle of the lumen region has a change in its larger dimension as the render position moves along the centreline towards the bifurcation as shown in FIG. 3 .
- the convex hull of the segmented region is found.
- This convex hull is used to find the minimum enclosing rectangle of the lumen region as shown in FIG. 3 .
- the larger dimension of the enclosing rectangle is saved as Dim Max .
- the same feature extraction step is carried out at all the pre-defined centreline view points of all the branches as a pre-computation step and the larger dimension of the enclosing rectangle is saved.
- the intra-branch search gives the best centreline view-point at each branch that has its feature value closest to Dim Max .
- the best rendering position that is obtained has a fixed roll angle. However, the real image could have any possible roll angle. To address this, roll angles of 90 degrees, 180 degrees and 270 degrees are applied to this rendering position resulting in the four best rendering positions for each branch.
- the real image is obtained from a bronchoscope having multiple degrees of freedom. So the four best rendering positions obtained in the previous step may not give rendering images I CT similar to the real image I V . Therefore, the four rendering positions obtained from the previous step are adjusted further using the inverse compositional method for local registration [24].
- the local registration uses WNSSD metric for image comparison and the optimal Gauss-Newton gradient for parameter update ( ⁇ ⁇ ) as given in (8) and (9).
- ⁇ CT and ⁇ V are the respective weighted-image means and ⁇ CT 2 and ⁇ V 2 are the respective weighted image variances and w u,v are weights that can be used to arbitrarily assign higher importance to pixels in the image based on geometry, graylevel value, gradient strength, or any appropriate confidence measure.
- w u,v are weights that can be used to arbitrarily assign higher importance to pixels in the image based on geometry, graylevel value, gradient strength, or any appropriate confidence measure.
- ⁇ V W(u,v,Z; ⁇ eq )
- ⁇ CT u,v
- Parameter update is run for 200 ms, which is sufficient for convergence and limits the total run-time of the algorithm resulting in four image rendering positions for each branch.
- the rendered images at each of these positions are thresholded to obtain the lumen region.
- These thresholded images are then compared with the lumen thresholded image obtained from the real image using the metric C 1 described in equation (11). Substituting from equation (11) in equation (7) we get the best rendering position of the branch.
- ⁇ ⁇ o arg ⁇ ⁇ min ⁇ ⁇ ⁇ K ⁇ ⁇ b i ⁇ D ⁇ ( I V , I CT ⁇ ) ( 12 )
- WNSSD dissimilarity metric as defined in equation (8). The optimum view point is used to find the current bronchoscope branch position using equation (6).
- FIGS. 4A-4F show the best matches for the different branches of the search space and FIG. 4B , the correctly identified branch after the inter-branch search.
- FIG. 4A shows the input RB frame and
- FIGS. 4B-4F show the VB images associated with the best bronchoscope position as found by the intra-branch search in five different branches of the search space.
- Inter-branch search correctly identified branch with rendering shown in FIG. 4B as the best branch.
- Algorithm 1 Pre-computation Phase: Computation of the feature value for every view site along each branch.
- Use BFP algorithm to find the convex hull of the thresholded lumen region. Find the minimum enclosing rectangle using the convex hull. save the longer dimension of the rectangle by the view-site index j. end end
- the global registration algorithm was evaluated using the three virtual cases derived from the CT data of consented patients as described in Table I. While using virtual bronchoscopic cases, a minimum of 28 branches from the first five airway generations were used. Four different tests were carried out by varying the search-space of the algorithm. In all tests, a virtual bronchoscopic bifurcation view was randomly selected from the branch search space—this served as the unknown “live bronchoscope video” view in the test. This bifurcation view was obtained by randomly moving to any of the view sites along a branch. A random roll angle from 0-360 degrees was used and a perturbation of up to ⁇ 5 mm was applied to the virtual bronchoscopic position to move the virtual bronchoscope off the center line.
- the airway phantom test involved live bronchoscopy on a preconstructed airway-tree phantom [25].
- the phantom consists of 5 different accessible branch bifurcations when using a 5.9 mm diameter bronchoscope. While carrying out the global registration, each incoming video frame from the bronchoscope was used as input and the output of the global-registration algorithm was one of the five branches from the search space. In order to carry out the tests, the bronchoscope was moved to each of the five different branches and the video collected from this bronchoscope maneuver was used in the testing. From the collected video, only those frames that gave good bifurcation views were used in the study. A total of 836 such frames were obtained.
- FIGS. 5A-5J show global registration results for airway phantom.
- the top row ( FIGS. 5A-5E ) are the input bronchoscopic video frames and the bottom row ( FIGS. 5F-5J ) are the VB images at the bronchoscope position estimated by the algorithm.
Abstract
Description
- This application claims priority from U.S. Provisional Patent Application Ser. Nos. 61/299,194, filed Jan. 28, 2010 and 61/300,969, filed Feb. 3, 2010, the entire content of both being incorporated herein by reference.
- This invention was made with government support under Grant No. R01-CA074325. awarded by NIH National Cancer Institute. The government has certain rights in the invention.
- This invention relates generally to bronchoscopy and, in particular, to a global registration system and method useful in image-based bronchoscopy guidance systems and other applications, including other types of endoscopic procedures.
- Bronchoscopy is a medical procedure commonly used in lung-cancer assessment[1]. Lung-cancer assessment involve two main stages[2,3,4]: 1) three-dimensional (3D) multi-detector computed-tomography (MDCT) image assessment; and 2) live bronchoscopy. During MDCT assessment, the physician uses the two-dimensional (2D) transverse slices obtained from the patient's MDCT scan to identify specific diagnostic regions of interest (ROIs), such as lymph nodes and suspicious nodules[5,6,7]. In this step, the physician also identifies the closest route to each ROI and mentally plans a 3D route using the 2D slices. During bronchoscopy, the physician maneuvers a flexible bronchoscope through the lung airways towards each ROI along the pre-planned mentally defined route. This is done by identifying the bifurcations along the route on the live endoluminal video feed obtained from the bronchoscope. This manually-based route planning procedure proves to be challenging, resulting in errors in bronchoscopy as early as the second airway generation[8,9].
- Image-guided bronchoscopy guidance systems enable more accurate bronchoscopy[4,10,11,12,13]. These systems are motivated by virtual bronchoscopy (VB), wherein the 3D MDCT image of a person's chest serves as a “virtual environment”[14,15,16]. A software-defined virtual camera navigates through the lungs in the virtual environment and presents endoluminal renderings of the 3D data, also known as VB images. To facilitate guidance during bronchoscopy, all bronchoscopy guidance systems rely on some method for registration of the real bronchoscope in 3D surgical space to the 3D MDCT virtual space. Based on the type of sensor used for registration, bronchoscopy guidance systems can be either electromagnetic (EM) or image-based[3,4,11,12,13,15,16,17,18].
- An EM-based guidance system consists of the following: 1) an EM field generator; 2) a steerable EM probe; and 3) guidance software[12,13,17]. The EM field generator generates an EM field around the patient's chest. The steerable EM probe is inserted through the working channel of the bronchoscope and tracked in the external EM field. Prior to the start of bronchoscopy, the steerable probe is used to calibrate and synchronize the coordinate system of the external EM field and the MDCT coordinate system. Thus, as the EM probe is tracked during bronchoscopy, its position in the MDCT coordinate system becomes nominally known. Such a system allows for immediate establishment of the global position of the bronchoscope tip within the 3D MDCT coordinate system. However, metallic objects in the vicinity induce ferromagnetic device interference, leading to distortions in the external EM field[19]. Moreover, the patient's breathing causes chest movement that leads to registration errors[20]. These errors are magnified in the peripheral airways, as the airway branches become smaller and move with the patient's breathing. Furthermore, once the bronchoscope is guided to the ROI, the steerable probe has to be retracted from the working channel of the bronchoscope so that the biopsy tools can be inserted to collect ROI tissue samples. Thus, EM-based bronchoscopy guidance systems implicitly provide global registration, but suffer in local registration. There has been ongoing research to combine the EM and image-based guidance methods in an attempt to mitigate these problems[21,22].
- Image-based bronchoscopy guidance systems rely on volume-rendered[16,23] or surface-rendered[3,4,24,25] endoluminal images of the airway tree from the 3D MDCT scans in order to establish the location of the bronchoscope. This is generally done by comparing the VB images with the real bronchoscopic (RB) video frames. Weighted normalized sum of square difference errors (WNSSD)[24] and normalized mutual information (NMI)[3,4,26] are metrics that are used for comparing the images obtained from the two sources. Registration is carried out using Powell's optimization, simplex or gradient methods. The image-based bronchoscopy guidance methods rely on local registrations at bifurcations and so are less susceptible to patient breathing motion. However, as these methods rely on the bronchoscope video, they are affected by artifacts in bronchoscope video caused by patient coughing or mucous obstruction. Also, most of the available systems rely on manual registration for initialization of the bronchoscope position. During a live bronchoscopic procedure, the absence of a global registration algorithm leads to increased procedure time and some uncertainty in the bronchoscope position. This in turn leads to guidance errors. Thus, image-based bronchoscopy guidance methods implicitly provide excellent local registration, but no global registration.
- Global registration is used in various fields such as image fusion[27, 28], remote sensing[29,30], object recognition[31], and robotic navigation[32]. The problem of establishing the global position in robotic navigation is most similar to global registration in the domain of image-based computer-guided bronchoscopy. In robotic navigation, global registration is also referred to as the “robot kidnapping problem,” wherein the position of a robot has to be estimated when it is moved to any arbitrary pose and no motion estimates are available[32]. Moreno et al. presented a non-linear filter, termed evolutive localization filter, that uses raw sensor data and recursively estimates the current pose[32]. Other methods utilizing multi-hypothesis Kalman filters[33, 34], grid-based probabilistic filters[35] and Monte-Carlo localization[36] methods have also been used for addressing the problem of global registration in robot navigation.
- In the domain of medical imaging, global registration has been primarily used for multi-modal registration. Zhang et al. have described an adaptive region-intensity-based ultrasound and computed tomography registration[37]. Munim et al. used vector distance functions for registering magnetic resonance (MR) images of multiple patients[38]. Moghari et al. have described a global registration method for aligning multiple bone fractures to a statistical anatomical atlas model[39]. Principal component analysis and the unscented Kalman filter were used for local and global registration, respectively. Fookes et al. have also described a method for registration of multiple MR images from the same patient by formulating the problem as the minimization of a covariance weighted non-linear least square function[40].
- In image-based bronchoscopy guidance, researchers have focused on the problem of local registration. However, few have worked on the problem of global registration, whereby the branch location of the bronchoscope is established. Bricault et al. have proposed a multi-level strategy for registration[23]. In this work, the relative position change of the sub-division wall from one bifurcation to the next was used to identify the branch position of the bronchoscope. Shinohara et al. described a branch identification method using eigenspace-image matching[41]. However, this method addresses bronchoscope tracking and cannot be used for global registration. Moreover, it requires manual initialization.
- This invention resides in a global registration system and method useful in image-based bronchoscopy guidance systems and other applications, including other types of endoscopic procedures. Synchronization is restored by identifying the current branch position, thereby facilitating global, technician-independent bronchoscopy guidance without the need for any external device such as an electromagnetic sensor.
- Virtual bronchoscopy (VB) renderings of a 3D airway tree are obtained, these including VB views of branch positions within the airway tree. At least one real bronchoscopic (RB) video frame is received from a bronchoscope inserted into the airway tree. An algorithm according to the invention is executed on a computer to identify the several most likely branch positions having a VB view closest to the received RB view, and the 3D position of the bronchoscope within the airway tree is determined in accordance with the branch position identified in the VB view.
- The algorithm may use various techniques to speed operation, including an intra-branch search followed by an inter-branch search. In particular, a fast search may be carried out over all the branches in a global airway-bifurcation search space, with a weighted normalized sum of squares distance metric being used to determine the best match. In a preferred embodiment the intra-branch search uses pre-computed lumen region enclosing rectangles in conjunction with a fast local registration refinement.
- The intra-branch search is given by:
-
- where C(•,•) is a similarity function, {circumflex over (θ)}b
i is the optimum view point in branch i for the given RB video frame IV and bi is a subset of Ktree and contains all the view points in branch I. - The inter-branch search is given by:
-
- where K{circumflex over (θ)} b
i ={{circumflex over (θ)}b1 , {circumflex over (θ)}b2 , . . . , {circumflex over (θ)}bn } is the set of view points obtained from the intra-branch search. - Results show that the algorithm currently needs 3 seconds per search-space branch, with a trend of improving accuracy with reducing search-space being seen. The algorithm gives accuracy rates=90% when using multiple views of the same bifurcation. When using the airway phantom, where a lighting model of the test frames was different from that used for the VB frames, a global registration accuracy of 89% was obtained.
-
FIGS. 1A and 1B show bronchoscopic video frame distortion correction; -
FIGS. 2A-2F show examples of the 3D airway tree and the virtual bronchoscope; -
FIG. 3 illustrates a pre-computation for intra-branch search; -
FIG. 4A shows an input RB frame; -
FIGS. 4B-4F show the VB images associated with the best bronchoscope position as found by the intra-branch search in five different branches of the search space; -
FIGS. 5A-5E show input bronchoscopic video frames for airway phantom; and -
FIGS. 5F-5J show VB images at the bronchoscope position estimated by the algorithm for airway phantom. - During bronchoscopy, the physician maneuvers a bronchoscope through the airway tree. An image-based bronchoscopy guidance system provides discrete guidance at bifurcations. In order to register the view seen in the bronchoscope video with multi-detector computed-tomography MDCT co-ordinate system, we model the image seen by the bronchoscope by using 3D MDCT data.
- To model the VB views, the bronchoscope camera is first calibrated by using an off-line procedure [42]. Parameters such as the focal length f and the FOV angle of the camera are computed. Coefficients are also found in order to correct the barrel distortion of the video frames as seen in
FIG. 1 .FIGS. 1A and 1B show bronchoscopic video frame distortion correction. In particular,FIG. 1A shows an example of a bronchoscopic video frame with barrel distortion, andFIG. 1B shows the bronchoscopic video frame after distortion correction. - To create VB views, the airway tree is automatically segmented using a robust segmentation algorithm [43]. The marching cubes algorithm is run on the 3D segmentation to obtain polygonal surface representation of the airway tree [44]. The virtual bronchoscope is designed as a pinhole camera model with a focal length of f and the same image dimensions as the bronchoscope camera. The endoluminal airway surfaces are assumed to be Lambertian, with the light source at the focal point. Secondary reflections are disregarded. Using these assumptions, the VB view is rendered using OpenGL[45] as shown in
FIGS. 2A-2F . Virtual bronchoscopic views.FIGS. 2A-2C show examples of the 3D airway tree and the virtual bronchoscope (yellow cylinder+graphical needle) positioned within the trachea, left main bronchus and the right main bronchus (shown inFIGS. 2D-2F ) and the corresponding VB views. - Global registration can be defined as establishing the current branch position of the bronchoscope in the 3D airway tree. To formulate the global registration problem, we consider the scenario where the bronchoscope is “blindly” inserted into the airway so that it lies at some bifurcation. We denote the real bronchoscopic (RB) video frame from the bronchoscope as IV(x, v). The virtual bronchoscopy (VB) renderings obtained from the virtual bronchoscope are denoted by ICT(x, y). The current unknown viewpoint of the bronchoscope is denoted by θ=(x, y, z, α, β, γ), where (x, y, z) gives the 3D spatial position and (α, β, γ) specifies the Euler angles. The global registration algorithm finds the branch that contains the viewpoint with a VB view closest to the given RB view.
- This problem can be posed as a Maximum A Posteriori (MAP) problem. The first part is probability density estimation problem, where we estimate the posterior density over the space of available bronchoscope poses given the incoming bronchoscopic video. This problem formulation is similar to that by Moreno et al [32]. In that paper, an iterative solution is used for global registration but we present a method for registration using a single frame which is similar to the method by Wei β et al [46]. From MAP point of view, the global registration problem is an optimization problem where we estimate the branch in which the bronchoscope gives a pose that maximizes the posteriori probability density
-
- where {circumflex over (θ)} is the estimated optimum view point, χ is one of the view points in Ktree, the search space for the view points and p(χ|IV) is the posterior density over the space of available bronchoscope poses given the incoming bronchoscopic video. Using Bayes theorem, we get
-
- In the above expression, if all bronchoscope poses are considered equi-likely p(χ) can be considered as a constant and p(IV) is a constant with respect to χ. Thus, we get
-
argmax p(χ|I V)=argmax p(I V|χ) (3) - The term p(IV|χ) can be estimated using a similarity function that finds the similarity between IV and the VB image rendered at χ, given by ICT χ. Hence, (3) becomes
-
argmax p(I V|χ)=argmax C(I V ,I CT χ) (4) - where C(•,•) is a similarity function that gives the measure of similarity between the bronchoscopic video frame IV and ICT χ the rendering obtained at pose χ. If the similarity function C(•,•) is replaced by a dissimilarity measure, the global registration problem becomes
-
- here D(•,•) is a dissimilarity function between the RB image IV and the VB view ICT χ at view point χ and Ktree is the search space for the view points. The optimum branch is given by
-
b o =B({circumflex over (θ)}) (6) - where B(•) is a function that finds the branch containing the view point {circumflex over (θ)}.
- The proposed algorithm accepts as input one or more live bronchoscopic video frames and outputs the 3D position of the bronchoscope within the airway tree. The algorithm is invoked during live bronchoscopy when the position of the bronchoscope is unknown. The algorithm will then determine the bronchoscope position within the airway tree. Before invoking the algorithm, the physician positions the bronchoscope to give a good view of an airway bifurcation, so that the lumen region is well represented, as shown in
FIG. 4 . Multiple runs of the algorithm on either the same bifurcation or at related bifurcations (parent-daughter branches) can be used to improve the overall accuracy of the method. The algorithm broadly divides into two major stages: intra-branch search and inter-branch search. -
FIG. 3 illustrates a pre-computation for intra-branch search. This figure shows the pre-computation carried out for the branch highlighted by the blue line. The red lines are the pre-defined centerline navigation routes through the 3D airway tree. The green triangles highlight three of the pre-defined view-sites (A, B, C) along the selected branch. The figures to the right of the view-sites are the VB renderings (A1, B1, C1), the lumen thresholded images (A2, B2, C2) and the minimum enclosing rectangles (A3, B3, C3) associated with the respective view-sites. The longer length of the minimum enclosing rectangle is saved during pre-computation and later used for localizing the position of the given RB frame in a given branch. - The intra-branch search is formulated as
-
- where C(•,•) is a similarity function, {circumflex over (θ)}b
i is the optimum view point in branch i for the given RB video frame IV and bi is a subset of Ktree and contains all the view points in branch i. - This optimization process needs spanning through all possible view positions in a given branch. This is not practical in a real-time algorithm due to the excessive time involved. Hence, we use a novel intra-branch search which comprises of two steps. In the first step, the intra-branch search carries out a fast scan through all the pre-defined view sites along the centreline of the branch. The second step comprises render position adjustment using fast local registration. Because of uncertainty in the roll angle of the bronchoscope we use a set of four likely positions for each branch using different roll angles.
- The minimum enclosing rectangle of the lumen region has a change in its larger dimension as the render position moves along the centreline towards the bifurcation as shown in
FIG. 3 . This is the underlying idea of the fast-scan search along the centreline. In this search, first the input real image IV is thresholded to segment out the lumen region. The p-tile thresholding with a value of p=10 has been empirically found to give good results for segmenting out the lumen. Once the lumen region is segmented, the convex hull of the segmented region is found. We use the Bentley-Faust-Preparata (BFP) fast approximate 2D convex hull algorithm for this purpose [47]. This convex hull is used to find the minimum enclosing rectangle of the lumen region as shown inFIG. 3 . The larger dimension of the enclosing rectangle is saved as DimMax. The same feature extraction step is carried out at all the pre-defined centreline view points of all the branches as a pre-computation step and the larger dimension of the enclosing rectangle is saved. During live global registration, the intra-branch search gives the best centreline view-point at each branch that has its feature value closest to DimMax. The best rendering position that is obtained has a fixed roll angle. However, the real image could have any possible roll angle. To address this, roll angles of 90 degrees, 180 degrees and 270 degrees are applied to this rendering position resulting in the four best rendering positions for each branch. - The real image is obtained from a bronchoscope having multiple degrees of freedom. So the four best rendering positions obtained in the previous step may not give rendering images ICT similar to the real image IV. Therefore, the four rendering positions obtained from the previous step are adjusted further using the inverse compositional method for local registration [24]. The local registration uses WNSSD metric for image comparison and the optimal Gauss-Newton gradient for parameter update (ΘΔ) as given in (8) and (9).
-
- where μCT and μV are the respective weighted-image means and θCT 2 and σV 2 are the respective weighted image variances and wu,v are weights that can be used to arbitrarily assign higher importance to pixels in the image based on geometry, graylevel value, gradient strength, or any appropriate confidence measure. Here, we use the trivially weighted case
-
- where
-
- is a 6D row vector of steepest-descent images, ĪV(W(u,v,Z;Θeq)) and ĪCT(u,v) are the normalized images and H, the Gauss-Newton Hessian is computed as
-
- Parameter update is run for 200 ms, which is sufficient for convergence and limits the total run-time of the algorithm resulting in four image rendering positions for each branch. The rendered images at each of these positions are thresholded to obtain the lumen region. These thresholded images are then compared with the lumen thresholded image obtained from the real image using the metric C1 described in equation (11). Substituting from equation (11) in equation (7) we get the best rendering position of the branch.
-
- The inter-branch search is given by
-
- where K{circumflex over (θ)} b
i ={{circumflex over (θ)}b1 , {circumflex over (θ)}b2 , . . . , {circumflex over (θ)}bn } is the set of view points obtained from the intra-branch search. In (12) we use the WNSSD dissimilarity metric as defined in equation (8). The optimum view point is used to find the current bronchoscope branch position using equation (6). - The above global registration method has been summarized in equation (12).
FIGS. 4A-4F show the best matches for the different branches of the search space andFIG. 4B , the correctly identified branch after the inter-branch search.FIG. 4A shows the input RB frame andFIGS. 4B-4F show the VB images associated with the best bronchoscope position as found by the intra-branch search in five different branches of the search space. Inter-branch search correctly identified branch with rendering shown inFIG. 4B as the best branch. -
Algorithm 1: Pre-computation Phase: Computation of the feature value for every view site along each branch. Initialization: Computing features for every view-site along a branch for ∀ branch i ε search-space do for ∀ view-site j ε branch i do Generate a virtual image ICT. Threshold the image ICT using p-tile thresholding (p = 10). Use BFP algorithm to find the convex hull of the thresholded lumen region. Find the minimum enclosing rectangle using the convex hull. save the longer dimension of the rectangle by the view-site index j. end end -
Algorithm 2: Global Registration for each new real image IV do Capture real image IV into buffer Threshold the real image IV Find convex hull of the thresholded region Find minimum enclosing rectangle using the convex hull Set bestValue=∞ and bestBranch=0 for every branch i in the search-space do Find closest branch view-site j using the pre-computed features for branch i From view-site j, obtain 4 viewing positions ( roll angles of θ = [90, 180, 270] ). Use (11) to find the VB image at the best render position for the branch ICT θ b 1 Use (8) to find D1(i) ( ICT = ICT θ b 1 ). if D1(i) ≦ bv then bestValue = D1(i) bestBranch = i end end end - To evaluate the accuracy of the global registration algorithm, we carried out three sets of tests. In the first set, we evaluated the global registration algorithm using virtual case studies. The second set of tests evaluated the accuracy of the global registration algorithm using RB frames obtained from the bronchoscopic exploration of an airway phantom. In the third set, we evaluated the improvement in the accuracy of the algorithm when multiple test frames were used.
- The global registration algorithm was evaluated using the three virtual cases derived from the CT data of consented patients as described in Table I. While using virtual bronchoscopic cases, a minimum of 28 branches from the first five airway generations were used. Four different tests were carried out by varying the search-space of the algorithm. In all tests, a virtual bronchoscopic bifurcation view was randomly selected from the branch search space—this served as the unknown “live bronchoscope video” view in the test. This bifurcation view was obtained by randomly moving to any of the view sites along a branch. A random roll angle from 0-360 degrees was used and a perturbation of up to ±5 mm was applied to the virtual bronchoscopic position to move the virtual bronchoscope off the center line. In the first test set, all branches were used for the search space. In the second set, the search space was divided into two by using branches either in the left lung or right lung only. In the third set, the search space was divided into five different parts based on the lung lobar regions. In the fourth set, only branches that were daughters of the same branch were used in the search space. The results of these tests are summarized in Table II. Global registration accuracy ranged from 71% to 92%. Note that a random selection from among 28 airway branches would only give an “accuracy” of 3.6%.
-
TABLE I Summary of cases used for testing. (ΔX, ΔY) in Total Total Case number Scanner mm slices branches 20349.3.47 Siemens Sensation 16 (0.55, 0.55) 517 227 20349.3.49 Siemens Sensation 40 (0.64, 0.64) 673 273 20349.3.61 Siemens Sensation 40 (0.6, 0.6) 588 219 -
TABLE II Global registration accuracy in percent for different test sets for virtual bronchoscopic cases. Case Full-tree Half-tree Lobar Branch pairs number (set 1) (set 2) (set 3) (set 4) 47 73% 76% 80% 92% 49 75% 77% 83% 91% 61 71% 73% 80% 83%
Evaluation using a Phantom Case - The airway phantom test involved live bronchoscopy on a preconstructed airway-tree phantom [25]. The phantom consists of 5 different accessible branch bifurcations when using a 5.9 mm diameter bronchoscope. While carrying out the global registration, each incoming video frame from the bronchoscope was used as input and the output of the global-registration algorithm was one of the five branches from the search space. In order to carry out the tests, the bronchoscope was moved to each of the five different branches and the video collected from this bronchoscope maneuver was used in the testing. From the collected video, only those frames that gave good bifurcation views were used in the study. A total of 836 such frames were obtained. The global registration algorithm gave an accuracy of 89% for the 836 frames (see
FIG. 5A-5J ).FIGS. 5A-5J show global registration results for airway phantom. The top row (FIGS. 5A-5E ) are the input bronchoscopic video frames and the bottom row (FIGS. 5F-5J ) are the VB images at the bronchoscope position estimated by the algorithm. - To evaluate the performance of the global registration algorithm using multiple frames, the virtual cases described in Table I were used. This test setup was the same as that used for evaluation using virtual cases. We carried out two sets of tests. In the first set, multiple “random views” of the same bifurcation were used for testing. In this testing methodology, each of the frames was independently evaluated by the global registration algorithm. The frame that gave the lowest WNSSD metric (bestValue from (12)) decided the branch location for the frames. In the second set, test frames from two consecutive bifurcations were used. Both the branches were independently evaluated. Of the two test frames, the frame giving the lower WNSSD metric determined the branch pair. The results for this set of testing has been summarized in Table III. The global registration accuracy ranges from 82% to 99%. In all of the above tests, the algorithm was found to run in an average time of 2-3 seconds per search space branch.
-
TABLE III Global registration accuracy in percent when using multiple test frames for virtual bronchoscopic cases. Two consecutive Case 3 frame 5 frame bifurcations number (set 1) (set 1) (set 2) 47 94% 98% 89% 49 95% 99% 87% 61 85% 86% 82% -
- 1. K. P. Wang, A. C. Mehta, and J. F. Turner, eds., Flexible Bronchoscopy, 2nd. Ed., Blackwell Science, Cambridge, Mass., 2003.
- 2. A. D. Sihoe and A. P. Yim, “Lung cancer staging,” J. Surgical Research, vol. 117, no. 1, pp. 92-106, March 2004.
- 3. J. P. Helferty, A. J. Sherbondy, A. P. Kiraly, and W. E. Higgins, “Computer-based system for the virtual-endoscopic guidance of bronchoscopy,” Comput. Vis. Image Underst., vol. 108, no. 1-2, pp. 171-187, October-November 2007.
- 4. W. E. Higgins, J. P. Helferty, K. Lu, S. A. Merritt, L. Rai, and K. C. Yu, “3D CT-video fusion for image-guided bronchoscopy,” Comput. Med. Imaging Graph., vol. 32, no. 3, pp. 159-173, April 2008.
- 5. E. A. Kazerooni, “High resolution CT of the lungs,” Am. J. Roentgenology, vol. 177, no. 3, pp. 501-519, September 2001.
- 6. N. C. Dalrymple, S. R. Prasad, M. W. Freckleton, and K. N. Chintapalli, “Introduction to the language of three-dimensional imaging with multidetector CT,” Radiographics, vol. 25, no. 5, pp. 1409-1428, September-October 2005.
- 7. J. Ueno, T. Murase, K. Yoneda, T. Tsujikawa, S. Sakiyama, and K. Kondoh, “Three-dimensional imaging of thoracic diseases with multi-detector row CT,” J. Med. Invest., vol. 51, no. 3-4, pp. 163-170, August 2004.
- 8. D. Osborne, P. Vock, J. Godwin, and P. Silverman, “CT identification of bronchopulmonary segments: 50 normal subjects,” AJR, vol. 142, no. 1, pp. 47-52, January 1984.
- 9. M. Y. Dolina, D. C. Cornish, S. A. Merritt, L. Rai, R. Mahraj, W. E. Higgins, and R. Bascom, “Interbronchoscopist variability in endobronchial path selection: a simulation study,” Chest, vol. 133, no. 4, pp. 897-905, April 2008.
- 10. F. Asano, Y. Matsuno, A. Tsuzuku, M. Anzai, N. Shinagawa, H. Moriya, et al., “Diagnosis of peripheral pulmonary lesions using a bronchoscope insertion guidance system combined with endobronchial ultrasonography with a guide sheath,” Lung Cancer, vol. 60, no. 3, pp. 366-373, June 2008.
- 11. S. B. Solomon, P. White, Jr., C. M. Wiener, J. B. Orens, and K. P. Wang, “Three-dimensionsal CT-guided bronchoscopy with a real-time electromagnetic position sensor: a comparison of two image registration methods,” Chest, vol. 118, no. 6, pp. 1783-1787, December 2000.
- 12. T. R. Gildea, P. J. Mazzone, D. Karnak, M. Meziane, and A. C. Mehta, “Electromagnetic navigation diagnostic bronchoscopy: a prospective study,” Am. J. Resp. Crit. Care Med., vol. 174, no. 9, pp. 982-989, 1 Nov. 2006.
- 13. Y. Schwarz, J. Greif, H. D. Becker, A. Ernst, and A. Mehta, “Real-time electromagnetic navigation bronchoscopy to peripheral lung lesions using overlaid CT images: the first human study,” Chest, vol. 129, no. 4, pp. 988-994, April 2006.
- 14. W. E. Higgins, K. Ramaswamy, R. Swift, G. McLennan, and E. A. Hoffman, “Virtual bronchoscopy for 3D pulmonary image assessment: State of the art and future needs,” Radiographics, vol. 18, no. 3, pp. 761-778, May-June 1998.
- 15. H. P. McAdams, P. C. Goodman, and P. Kussin, “Virtual bronchoscopy for directing transbronchial needle aspiration of hilar and mediastinal lymph nodes: a pilot study,” Am. J. Roentgenology, vol. 170, no. 5, pp. 13614364, May 1998.
- 16. K. Hopper, T. Lucas, K. Gleeson, J. Stauffer, R. Bascom, D. Mauger, and R. Mahraj, “Transbronchial biopsy with virtual CT bronchoscopy and nodal highlighting,” Radiology, vol. 221, no. 2, pp. 531-536, November 2001.
- 17. H. D. Becker, F. Herth, A. Ernst, and Y. Schwarz, “Bronchoscopic biopsy of peripheral lung lesions under electromagnetic guidance: a pilot study,” J. Bronchology, vol. 12, no. 1, pp. 9, January 2005.
- 18. N. Shinagawa, K. Yamazaki, Y. Onodera, K. Miyasaka, E. Kikuchi, H. Dosaka-Akita, and M. Nishimura, “CT-guided transbronchial biopsy using an ultrathin bronchoscope with virtual bronchoscopic navigation,” Chest, vol. 125, no. 3, pp. 1138-1143, March 2004.
- 19. K. Mori, K. Ishitani, D. Deguchi., T. Kitasaka, Y. Suenaga, H. Takabatake, M. Mod, and H. Natori, “Compensation of electromagnetic tracking system using an optical tracker and its application to bronchoscopy navigation system,” 2007, vol. 6509, p. 65090M,
- 20. I. Wegner, J. Biederer, R. Tetzlaff, I. Wolf, and H.-P. Meinzer, “Evaluation and extension of a navigation system for bronchoscopy inside human lungs,” in SPIE Medical Imaging 2007: Visualization and Image-Guided Procedures, Kevin R. Cleary and Michael I. Miga, Eds., 2007, vol. 6509, pp. 65091H1-65091H12.
- 21. T. D. Soper, D. R. Haynor, R. W. Glenny, and E. J. Seibel, “Validation of CT-video registration for guiding a novel ultrathin bronchoscope to peripheral lung nodules using electromagnetic tracking,” in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, February 2009, vol. 7261 of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series.
- 22. K. Mori, D. Deguchi, K. Akiyama, T. Kitasaka, C. R. Maurer, Y. Suenaga, H. Takabatake, M. Mod, and H. Natori, “Hybrid bronchoscope tracking using a magnetic tracking sensor and image registration,” in Medical Image Computing and Computer Assisted Intervention 2005, J. Duncan and G. Gerig, Eds., 2005, vol. LNCS 3750, pp. 543-550.
- 23. I. Bricault, G. Ferretti, and P. Cinquin, “Registration of real and CT-derived virtual bronchoscopic images to assist transbronchial biopsy,” IEEE Transactions on Medical Imaging, vol. 17, no. 5, pp. 703-714, October 1998.
- 24. S. A. Merritt, L. Rai, and W. E. Higgins, “Real-time CT-video registration for continuous endoscopic guidance,” in SPIE Medical Imaging 2006: Physiology, Function, and Structure from Medical images, A. Manduca and A. A. Amini, Eds., 2006, vol. 6143, pp. 370-384.
- 25. S. A. Merritt, J. D. Gibbs, K. C. Yu, V. Patel, L. Rai, D. C. Cornish, R. Bascom, and W. E. Higgins, “Real-time image-guided bronchoscopy for peripheral lung lesions: A phantom study,” Chest, vol. 134, no. 5, pp. 1017-1026, November 2008.
- 26. J. P. Helferty and W. E. Higgins, “Technique for registering 3D virtual CT images to endoscopic video,” IEEE Int. Conf. image Processing, vol. II, pp. 893-896, Oct. 7-10 2001.
- 27. E-Y Kang, I. Cohen, and G. Medioni, “A graph-based global registration for 2D mosaics,” in ICPR, 2000, pp. 1257-1260.
- 28. Y. Wang and Lu-ping L. Xu, “A global optimized registration algorithm for image stitching,” in Image and Signal Processing, 2008. CISP '08. Congress on, May 2008, vol. 3, pp. 525-529.
- 29. Y. Li and C. Davis, “A combined global and local approach for automated registration of high-resolution satellite images using optimum extrema points,” in Geoscience and Remote Sensing Symposium, 2008. IGARSS 2008. IEEE International, July 2008, vol. 2, pp. II-1032-II-1035.
- 30. A. Wong and D. Clausi, “ARRSI: Automatic registration of remote-sensing images,” Geoscience and Remote Sensing, IEEE Transactions on, vol. 45, no. 5, pp. 1483-1493, May 2007.
- 31. N. Gelfand, N. J. Mitra, L. J. Guibas, and H. Pottmann, “Robust global registration,” in SGP 2005: Third Eurographics Symposium on Geometry processing, Matthieu Desbrun and Helmut Pottmann, Eds. 2005, pp. 197-206, Eurographics Association.
- 32. L. Moreno, S. Gamido, and D. Blanco, “Mobile robot global localization using an evolutionary MAP filter,” J. of Global Optimization, vol. 37, no. 3, pp. 381-403, 2007.
- 33. P. Jensfelt and S. Kristensen, “Active global localisation for a mobile robot using multiple hypothesis tracking,” IEEE Transactions on Robotics and Automation, vol. 17, no. 5, pp. 748-760, October 2001.
- 34. K. O. Arras, J. A. Castellanos, M. Schilt, and R. Siegwart, “Feature-based multi-hypothesis localization and tracking using geometric constraints,” Robotics and Autonomous Systems, vol. 44, no. 1, pp. 41-53, 2003.
- 35. W. Burgard, D. Fox, D. Hennig, and T. Schmidt, “Estimating the absolute position of a mobile robot using position probability grids,” in AAAI/IAAI, Vol. 2, 1996.
- 36. F. Dellaert, D. Fox, W. Burgard, and S. Thrun, “Monte Carlo localization for mobile robots,” in IEEE International Conference on Robotics and Automation (ICRA99), May 1999.
- 37. Z. Zhang, “Adaptive region intensity based rigid ultrasound and CT image registration,” in Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, June 2008, PP. 1-7.
- 38. H. Munim and A. Farag, “A new global registration approach of medical imaging using vector maps,” in Biomedical Imaging: From Nano to Macro, 2007, ISBI 2007. 4th IEEE International Symposium on, April 2007, pp. 584-587.
- 39. M. H. Moghari and P. Abolmaesumi, “Global registration of multiple bone fragments using statistical atlas models: Feasibility experiments,” in Engineering in Medicine and Biology Society, 2008. EMBS 2008. 30th Annual international Conference of the IEEE, August 2008, pp. 5374-5377.
- 40. C. Fookes, J. Williams, and M. Bennamoun, “Global 3D rigid registration of medical images,” in Image Processing, 2000. Proceedings. 2000 International Conference on, 2000, vol. 2, pp. 447-450.
- 41. R. Shinohara, K. Mori, D. Deguchi, T. Kitasaka, Y. Suenaga, H. Takabatake, M. Mori, and H. Natori, “Branch identification method for CT-Guided bronchoscopy based on eigenspace image matching between real and virtual bronchoscopic images,” in SPIE Medical Imaging 2006: Physiology, Function, and Structure from Medical Images, A. Manduca and A. A. Amini, Eds., March 2006, vol. 6143, pp. 385-396.
- 42. J. P. Helferty, C. Zhang, G. McLennan, and W. E. Higgins, “Videoendoscopic distortion correction and its application to virtual guidance of endoscopy,” IEEE Trans. Med. Imaging, vol. 20, no. 7, pp. 605-617, July 2001.
- 43. M. W. Graham, J. D. Gibbs, D. C. Cornish, M. Khan, R. Bascom, and W. E. Higgins, “Image-guided peripheral bronchoscopy: A pilot human study,” Chest, under review, 2009.
- 44. W. E. Lorensen and H. E. Cline, “Marching cubes: A
high resolution 3D surface construction algorithm,” Computer Graphics, vol. 21, no. 4, pp. 163-169, July 1987. - 45. R. S. Wright, Jr., and B. Lipchak, OpenGL Super Bible, 3rd. Ed., SAMS Publishing, 2005.
- 46. G. Weifi, C. Wetzler, and E. V. Puttkamer, “Keeping track of position and orientation of moving indoor systems by correlation of range-finder scans,” in In Proc. 1994 IEEE Int. Conf: on Intelligent Robots and Systems IROS '94, 1994, pp. 595-601.
- 47. J. L. Bentley, F. P. Preparata, and M. G. Faust, “Approximation algorithms for convex hulls,” Commun. ACM, vol. 25, no. 1, pp. 64-68, 1982.
Claims (28)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/016,093 US20110184238A1 (en) | 2010-01-28 | 2011-01-28 | Image-based global registration system and method applicable to bronchoscopy guidance |
US15/928,974 US10667679B2 (en) | 2010-01-28 | 2018-03-22 | Image-based global registration system and method applicable to bronchoscopy guidance |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US29919410P | 2010-01-28 | 2010-01-28 | |
US30096910P | 2010-02-03 | 2010-02-03 | |
US13/016,093 US20110184238A1 (en) | 2010-01-28 | 2011-01-28 | Image-based global registration system and method applicable to bronchoscopy guidance |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/928,974 Continuation US10667679B2 (en) | 2010-01-28 | 2018-03-22 | Image-based global registration system and method applicable to bronchoscopy guidance |
Publications (1)
Publication Number | Publication Date |
---|---|
US20110184238A1 true US20110184238A1 (en) | 2011-07-28 |
Family
ID=44309460
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/016,093 Abandoned US20110184238A1 (en) | 2010-01-28 | 2011-01-28 | Image-based global registration system and method applicable to bronchoscopy guidance |
US15/928,974 Active US10667679B2 (en) | 2010-01-28 | 2018-03-22 | Image-based global registration system and method applicable to bronchoscopy guidance |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/928,974 Active US10667679B2 (en) | 2010-01-28 | 2018-03-22 | Image-based global registration system and method applicable to bronchoscopy guidance |
Country Status (5)
Country | Link |
---|---|
US (2) | US20110184238A1 (en) |
EP (1) | EP2528496A4 (en) |
JP (1) | JP2013517909A (en) |
CN (1) | CN102883651B (en) |
WO (1) | WO2011094518A2 (en) |
Cited By (52)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120059221A1 (en) * | 2010-05-31 | 2012-03-08 | Olympus Medical Systems Corp. | Endoscopic form detection device and form detecting method of insertion section of endoscope |
CN102521833A (en) * | 2011-12-08 | 2012-06-27 | 东软集团股份有限公司 | Method for obtaining tracheae tree from chest CT image and apparatus thereof |
US20120289777A1 (en) * | 2011-05-13 | 2012-11-15 | Intuitive Surgical Operations, Inc. | Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery |
WO2013173234A1 (en) | 2012-05-14 | 2013-11-21 | Intuitive Surgical Operations | Systems and methods for registration of a medical device using rapid pose search |
WO2014028394A1 (en) | 2012-08-14 | 2014-02-20 | Intuitive Surgical Operations, Inc. | Systems and methods for registration of multiple vision systems |
US20140180097A1 (en) * | 2011-10-17 | 2014-06-26 | Butterfly Network, Inc. | Volumetric imaging and related apparatus and methods |
US20140232840A1 (en) * | 2010-08-20 | 2014-08-21 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation in endoscopic applications |
US20150057498A1 (en) * | 2013-03-12 | 2015-02-26 | Olympus Medical Systems Corp. | Endoscope system |
US20150073269A1 (en) * | 2013-09-06 | 2015-03-12 | Covidien Lp | System and method for light based lung visualization |
US20150196228A1 (en) * | 2013-04-15 | 2015-07-16 | Olympus Medical Systems Corp. | Endoscope system |
US9516993B2 (en) | 2013-03-27 | 2016-12-13 | Olympus Corporation | Endoscope system |
US20170071504A1 (en) * | 2015-09-16 | 2017-03-16 | Fujifilm Corporation | Endoscope position identifying apparatus, endoscope position identifying method, and recording medium having an endoscope position identifying program recorded therein |
US9757021B2 (en) * | 2011-02-04 | 2017-09-12 | The Penn State Research Foundation | Global and semi-global registration for image-based bronchoscopy guidance |
JPWO2016151711A1 (en) * | 2015-03-20 | 2018-02-15 | オリンパス株式会社 | Image processing apparatus, image processing method, and image processing program |
CN108042092A (en) * | 2012-10-12 | 2018-05-18 | 直观外科手术操作公司 | Determine position of the medical instrument in branch's anatomical structure |
US20180220883A1 (en) * | 2010-01-28 | 2018-08-09 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
US20180263527A1 (en) * | 2017-03-16 | 2018-09-20 | Fujifilm Corporation | Endoscope position specifying device, method, and program |
US20180263712A1 (en) * | 2017-03-16 | 2018-09-20 | Fujifilm Corporation | Endoscope position specifying device, method, and program |
US10098565B2 (en) | 2013-09-06 | 2018-10-16 | Covidien Lp | System and method for lung visualization using ultrasound |
US10201265B2 (en) | 2013-09-06 | 2019-02-12 | Covidien Lp | Microwave ablation catheter, handle, and system |
WO2019112804A1 (en) * | 2017-12-05 | 2019-06-13 | Covidien Lp | Multi-rigid registration of magnetic navigation to a computed tomography volume |
WO2019118677A1 (en) * | 2017-12-13 | 2019-06-20 | Covidien Lp | Systems, methods, and computer-readable media for automatic computed tomography to computed tomography registration |
US10561463B2 (en) | 2013-09-06 | 2020-02-18 | Covidien Lp | Microwave ablation catheter, handle, and system |
US10796432B2 (en) | 2015-09-18 | 2020-10-06 | Auris Health, Inc. | Navigation of tubular networks |
US10806535B2 (en) | 2015-11-30 | 2020-10-20 | Auris Health, Inc. | Robot-assisted driving systems and methods |
US10827913B2 (en) | 2018-03-28 | 2020-11-10 | Auris Health, Inc. | Systems and methods for displaying estimated location of instrument |
US10898286B2 (en) | 2018-05-31 | 2021-01-26 | Auris Health, Inc. | Path-based navigation of tubular networks |
US10898275B2 (en) * | 2018-05-31 | 2021-01-26 | Auris Health, Inc. | Image-based airway analysis and mapping |
US10898277B2 (en) | 2018-03-28 | 2021-01-26 | Auris Health, Inc. | Systems and methods for registration of location sensors |
US10905499B2 (en) | 2018-05-30 | 2021-02-02 | Auris Health, Inc. | Systems and methods for location sensor-based branch prediction |
US11020016B2 (en) | 2013-05-30 | 2021-06-01 | Auris Health, Inc. | System and method for displaying anatomy and devices on a movable display |
US11051681B2 (en) | 2010-06-24 | 2021-07-06 | Auris Health, Inc. | Methods and devices for controlling a shapeable medical device |
US11058493B2 (en) | 2017-10-13 | 2021-07-13 | Auris Health, Inc. | Robotic system configured for navigation path tracing |
US11129602B2 (en) | 2013-03-15 | 2021-09-28 | Auris Health, Inc. | Systems and methods for tracking robotically controlled medical instruments |
US11147633B2 (en) | 2019-08-30 | 2021-10-19 | Auris Health, Inc. | Instrument image reliability systems and methods |
US11160615B2 (en) | 2017-12-18 | 2021-11-02 | Auris Health, Inc. | Methods and systems for instrument tracking and navigation within luminal networks |
US11207141B2 (en) | 2019-08-30 | 2021-12-28 | Auris Health, Inc. | Systems and methods for weight-based registration of location sensors |
US11241203B2 (en) | 2013-03-13 | 2022-02-08 | Auris Health, Inc. | Reducing measurement sensor error |
US11278357B2 (en) | 2017-06-23 | 2022-03-22 | Auris Health, Inc. | Robotic systems for determining an angular degree of freedom of a medical device in luminal networks |
US11298195B2 (en) | 2019-12-31 | 2022-04-12 | Auris Health, Inc. | Anatomical feature identification and targeting |
US11304762B2 (en) * | 2017-02-24 | 2022-04-19 | Fujifilm Corporation | Mapping image display control device, method, and program |
US11426095B2 (en) | 2013-03-15 | 2022-08-30 | Auris Health, Inc. | Flexible instrument localization from both remote and elongation sensors |
US11490782B2 (en) | 2017-03-31 | 2022-11-08 | Auris Health, Inc. | Robotic systems for navigation of luminal networks that compensate for physiological noise |
US11504187B2 (en) | 2013-03-15 | 2022-11-22 | Auris Health, Inc. | Systems and methods for localizing, tracking and/or controlling medical instruments |
US11503986B2 (en) | 2018-05-31 | 2022-11-22 | Auris Health, Inc. | Robotic systems and methods for navigation of luminal network that detect physiological noise |
US11510736B2 (en) | 2017-12-14 | 2022-11-29 | Auris Health, Inc. | System and method for estimating instrument location |
US11602372B2 (en) | 2019-12-31 | 2023-03-14 | Auris Health, Inc. | Alignment interfaces for percutaneous access |
US11660147B2 (en) | 2019-12-31 | 2023-05-30 | Auris Health, Inc. | Alignment techniques for percutaneous access |
US11771309B2 (en) | 2016-12-28 | 2023-10-03 | Auris Health, Inc. | Detecting endolumenal buckling of flexible instruments |
US11850008B2 (en) | 2017-10-13 | 2023-12-26 | Auris Health, Inc. | Image-based branch detection and mapping for navigation |
US11957319B2 (en) | 2018-12-06 | 2024-04-16 | Verathon Inc. | Endobronchial ultrasound imaging |
US11969157B2 (en) | 2023-04-28 | 2024-04-30 | Auris Health, Inc. | Systems and methods for tracking robotically controlled medical instruments |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2849670B1 (en) | 2012-05-14 | 2019-03-27 | Intuitive Surgical Operations, Inc. | Systems for registration of a medical device using a reduced search space |
US10039473B2 (en) | 2012-05-14 | 2018-08-07 | Intuitive Surgical Operations, Inc. | Systems and methods for navigation based on ordered sensor records |
EP2904957A4 (en) * | 2013-03-06 | 2016-08-24 | Olympus Corp | Endoscope system |
US9754367B2 (en) * | 2014-07-02 | 2017-09-05 | Covidien Lp | Trachea marking |
CN104306072B (en) * | 2014-11-07 | 2016-08-31 | 常州朗合医疗器械有限公司 | Medical treatment navigation system and method |
US10163262B2 (en) * | 2015-06-19 | 2018-12-25 | Covidien Lp | Systems and methods for navigating through airways in a virtual bronchoscopy view |
US20190290225A1 (en) * | 2016-05-13 | 2019-09-26 | The Regents Of The University Of California | Airway mucus impaction |
KR102051410B1 (en) * | 2018-10-25 | 2019-12-03 | 인하대학교 산학협력단 | Method and Apparatus for Scalable Global View and Depth Generation for Large-scale Multi-view Videos |
US11877806B2 (en) | 2018-12-06 | 2024-01-23 | Covidien Lp | Deformable registration of computer-generated airway models to airway trees |
CN113855242B (en) * | 2021-12-03 | 2022-04-19 | 杭州堃博生物科技有限公司 | Bronchoscope position determination method, device, system, equipment and medium |
CN116416413A (en) * | 2021-12-31 | 2023-07-11 | 杭州堃博生物科技有限公司 | Motion navigation method, device and equipment of bronchoscope |
CN116433874A (en) * | 2021-12-31 | 2023-07-14 | 杭州堃博生物科技有限公司 | Bronchoscope navigation method, device, equipment and storage medium |
CN114041741B (en) * | 2022-01-13 | 2022-04-22 | 杭州堃博生物科技有限公司 | Data processing unit, processing device, surgical system, surgical instrument, and medium |
Citations (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020118278A1 (en) * | 2001-02-28 | 2002-08-29 | Asahi Kogaku Kogyo Kabushiki Kaisha | Organ-region-indication system incorporated in electronic endoscope system |
US20030091967A1 (en) * | 2000-04-12 | 2003-05-15 | Edna Chosack | Endoscopic tutorial system for urology |
US20050020878A1 (en) * | 2002-07-31 | 2005-01-27 | Junichi Ohnishi | Endoscope |
US6892090B2 (en) * | 2002-08-19 | 2005-05-10 | Surgical Navigation Technologies, Inc. | Method and apparatus for virtual endoscopy |
US20050107679A1 (en) * | 2003-07-11 | 2005-05-19 | Bernhard Geiger | System and method for endoscopic path planning |
US20050122343A1 (en) * | 2002-11-27 | 2005-06-09 | Anthony Bailey | User-interface and method for curved multi-planar reformatting of three-dimensional volume data sets |
US20050182295A1 (en) * | 2003-12-12 | 2005-08-18 | University Of Washington | Catheterscope 3D guidance and interface system |
US20070013710A1 (en) * | 2005-05-23 | 2007-01-18 | Higgins William E | Fast 3D-2D image registration method with application to continuously guided endoscopy |
US20070015997A1 (en) * | 2005-05-23 | 2007-01-18 | Higgins William E | Guidance method based on 3D-2D pose estimation and 3D-CT registration with application to live bronchoscopy |
US20070167714A1 (en) * | 2005-12-07 | 2007-07-19 | Siemens Corporate Research, Inc. | System and Method For Bronchoscopic Navigational Assistance |
US20080207997A1 (en) * | 2007-01-31 | 2008-08-28 | The Penn State Research Foundation | Method and apparatus for continuous guidance of endoscopy |
US20090156895A1 (en) * | 2007-01-31 | 2009-06-18 | The Penn State Research Foundation | Precise endoscopic planning and visualization |
US20090161927A1 (en) * | 2006-05-02 | 2009-06-25 | National University Corporation Nagoya University | Medical Image Observation Assisting System |
US7998062B2 (en) * | 2004-03-29 | 2011-08-16 | Superdimension, Ltd. | Endoscope structures and techniques for navigating to a target in branched structure |
US20110282151A1 (en) * | 2008-10-20 | 2011-11-17 | Koninklijke Philips Electronics N.V. | Image-based localization method and system |
US8102416B2 (en) * | 2010-02-22 | 2012-01-24 | Olympus Medical Systems Corp. | Medical apparatus |
US20120130171A1 (en) * | 2010-11-18 | 2012-05-24 | C2Cure Inc. | Endoscope guidance based on image matching |
US8219179B2 (en) * | 2008-03-06 | 2012-07-10 | Vida Diagnostics, Inc. | Systems and methods for navigation within a branched structure of a body |
US20120203067A1 (en) * | 2011-02-04 | 2012-08-09 | The Penn State Research Foundation | Method and device for determining the location of an endoscope |
US20120203065A1 (en) * | 2011-02-04 | 2012-08-09 | The Penn State Research Foundation | Global and semi-global registration for image-based bronchoscopy guidance |
US8280136B2 (en) * | 2005-09-16 | 2012-10-02 | The Ohio State University | Method and apparatus for detecting intraventricular dyssynchrony |
US8298135B2 (en) * | 2008-05-23 | 2012-10-30 | Olympus Medical Systems Corp. | Medical device with endoscope and insertable instrument |
US20120289777A1 (en) * | 2011-05-13 | 2012-11-15 | Intuitive Surgical Operations, Inc. | Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery |
Family Cites Families (37)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AT399647B (en) * | 1992-07-31 | 1995-06-26 | Truppe Michael | ARRANGEMENT FOR DISPLAYING THE INTERIOR OF BODIES |
US5782762A (en) * | 1994-10-27 | 1998-07-21 | Wake Forest University | Method and system for producing interactive, three-dimensional renderings of selected body organs having hollow lumens to enable simulated movement through the lumen |
US6702736B2 (en) * | 1995-07-24 | 2004-03-09 | David T. Chen | Anatomical visualization system |
JP2000135215A (en) * | 1998-10-30 | 2000-05-16 | Ge Yokogawa Medical Systems Ltd | Conduit guiding method and device thereof and radiation tomographic equipment |
EP1095628A3 (en) * | 1999-10-29 | 2001-05-16 | Marconi Medical Systems, Inc. | Planning minimally invasive procedures for in - vivo placement of objects |
JP2003116772A (en) * | 2001-10-18 | 2003-04-22 | Olympus Optical Co Ltd | Endoscope instrument and hood member for endoscope |
JP4171833B2 (en) * | 2002-03-19 | 2008-10-29 | 国立大学法人東京工業大学 | Endoscope guidance device and method |
US20050272971A1 (en) * | 2002-08-30 | 2005-12-08 | Olympus Corporation | Medical treatment system, endoscope system, endoscope insert operation program, and endoscope device |
US20050033117A1 (en) * | 2003-06-02 | 2005-02-10 | Olympus Corporation | Object observation system and method of controlling object observation system |
US7300398B2 (en) * | 2003-08-14 | 2007-11-27 | Siemens Medical Solutions Usa, Inc. | Method and apparatus for registration of virtual endoscopic images |
JP4464640B2 (en) * | 2003-09-01 | 2010-05-19 | オリンパス株式会社 | Industrial endoscope equipment |
US20050085717A1 (en) * | 2003-10-21 | 2005-04-21 | Ramin Shahidi | Systems and methods for intraoperative targetting |
WO2005043319A2 (en) * | 2003-10-21 | 2005-05-12 | The Board Of Trustees Of The Leland Stanford Junior University | Systems and methods for intraoperative targeting |
JP3820244B2 (en) * | 2003-10-29 | 2006-09-13 | オリンパス株式会社 | Insertion support system |
EP2189107A3 (en) * | 2003-10-31 | 2010-09-15 | Olympus Corporation | Insertion support system |
JP3847744B2 (en) * | 2003-11-04 | 2006-11-22 | オリンパス株式会社 | Insertion support system |
US7991232B2 (en) * | 2004-03-03 | 2011-08-02 | Nec Corporation | Image similarity calculation system, image search system, image similarity calculation method, and image similarity calculation program |
US7811294B2 (en) * | 2004-03-08 | 2010-10-12 | Mediguide Ltd. | Automatic guidewire maneuvering system and method |
JP4698966B2 (en) * | 2004-03-29 | 2011-06-08 | オリンパス株式会社 | Procedure support system |
DE602005024159D1 (en) * | 2004-06-23 | 2010-11-25 | Koninkl Philips Electronics Nv | VIRTUAL ENDOSCOPY |
EP1876954A4 (en) * | 2005-05-02 | 2009-12-16 | Agency Science Tech & Res | Method and apparatus for registration of an atlas to an image |
JP5525727B2 (en) * | 2005-05-23 | 2014-06-18 | ザ ペン ステイト リサーチ ファンデーション | 3D-CT registration with guidance method based on 3D-2D pose estimation and application to raw bronchoscopy |
CN1920882A (en) * | 2005-08-24 | 2007-02-28 | 西门子共同研究公司 | System and method for salient region feature based 3d multi modality registration of medical images |
US7623900B2 (en) * | 2005-09-02 | 2009-11-24 | Toshiba Medical Visualization Systems Europe, Ltd. | Method for navigating a virtual camera along a biological object with a lumen |
EP1960966B1 (en) * | 2005-12-08 | 2009-05-27 | Koninklijke Philips Electronics N.V. | System and method for enabling selection of an image registration transformation |
JP5457178B2 (en) * | 2006-06-30 | 2014-04-02 | ブロンカス テクノロジーズ, インコーポレイテッド | Airway bypass site selection and treatment planning |
US9037215B2 (en) * | 2007-01-31 | 2015-05-19 | The Penn State Research Foundation | Methods and apparatus for 3D route planning through hollow organs |
WO2008111070A2 (en) * | 2007-03-12 | 2008-09-18 | David Tolkowsky | Devices and methods for performing medical procedures in tree-like luminal structures |
JP2008253586A (en) * | 2007-04-06 | 2008-10-23 | Hoya Corp | Endoscope support system |
CN100517387C (en) * | 2007-06-18 | 2009-07-22 | 华中科技大学 | Multi likeness measure image registration method |
JP4922107B2 (en) * | 2007-09-03 | 2012-04-25 | オリンパスメディカルシステムズ株式会社 | Endoscope device |
JP5208495B2 (en) * | 2007-12-27 | 2013-06-12 | オリンパスメディカルシステムズ株式会社 | Medical system |
EP3023055B1 (en) * | 2008-02-12 | 2017-12-13 | Covidien LP | Controlled perspective guidance method |
EP2244633A4 (en) * | 2008-02-14 | 2013-10-23 | Penn State Res Found | Medical image reporting system and method |
JP5372406B2 (en) * | 2008-05-23 | 2013-12-18 | オリンパスメディカルシステムズ株式会社 | Medical equipment |
WO2011094518A2 (en) * | 2010-01-28 | 2011-08-04 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
US20130303887A1 (en) * | 2010-08-20 | 2013-11-14 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation |
-
2011
- 2011-01-28 WO PCT/US2011/022891 patent/WO2011094518A2/en active Application Filing
- 2011-01-28 EP EP11737714.3A patent/EP2528496A4/en not_active Withdrawn
- 2011-01-28 US US13/016,093 patent/US20110184238A1/en not_active Abandoned
- 2011-01-28 CN CN201180017623.4A patent/CN102883651B/en active Active
- 2011-01-28 JP JP2012551312A patent/JP2013517909A/en active Pending
-
2018
- 2018-03-22 US US15/928,974 patent/US10667679B2/en active Active
Patent Citations (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030091967A1 (en) * | 2000-04-12 | 2003-05-15 | Edna Chosack | Endoscopic tutorial system for urology |
US6930706B2 (en) * | 2001-02-28 | 2005-08-16 | Pentax Corporation | Organ-region-indication system incorporated in electronic endoscope system |
US20020118278A1 (en) * | 2001-02-28 | 2002-08-29 | Asahi Kogaku Kogyo Kabushiki Kaisha | Organ-region-indication system incorporated in electronic endoscope system |
US20070142705A1 (en) * | 2002-07-31 | 2007-06-21 | Olympus Corporation | Endoscope device |
US20050020878A1 (en) * | 2002-07-31 | 2005-01-27 | Junichi Ohnishi | Endoscope |
US7641609B2 (en) * | 2002-07-31 | 2010-01-05 | Olympus Corporation | Endoscope device and navigation method for endoscope device |
US6892090B2 (en) * | 2002-08-19 | 2005-05-10 | Surgical Navigation Technologies, Inc. | Method and apparatus for virtual endoscopy |
US20050122343A1 (en) * | 2002-11-27 | 2005-06-09 | Anthony Bailey | User-interface and method for curved multi-planar reformatting of three-dimensional volume data sets |
US20050107679A1 (en) * | 2003-07-11 | 2005-05-19 | Bernhard Geiger | System and method for endoscopic path planning |
US20060149134A1 (en) * | 2003-12-12 | 2006-07-06 | University Of Washington | Catheterscope 3D guidance and interface system |
US20050182295A1 (en) * | 2003-12-12 | 2005-08-18 | University Of Washington | Catheterscope 3D guidance and interface system |
US7998062B2 (en) * | 2004-03-29 | 2011-08-16 | Superdimension, Ltd. | Endoscope structures and techniques for navigating to a target in branched structure |
US20070015997A1 (en) * | 2005-05-23 | 2007-01-18 | Higgins William E | Guidance method based on 3D-2D pose estimation and 3D-CT registration with application to live bronchoscopy |
US20070013710A1 (en) * | 2005-05-23 | 2007-01-18 | Higgins William E | Fast 3D-2D image registration method with application to continuously guided endoscopy |
US7889905B2 (en) * | 2005-05-23 | 2011-02-15 | The Penn State Research Foundation | Fast 3D-2D image registration method with application to continuously guided endoscopy |
US8280136B2 (en) * | 2005-09-16 | 2012-10-02 | The Ohio State University | Method and apparatus for detecting intraventricular dyssynchrony |
US20070167714A1 (en) * | 2005-12-07 | 2007-07-19 | Siemens Corporate Research, Inc. | System and Method For Bronchoscopic Navigational Assistance |
US8199984B2 (en) * | 2006-05-02 | 2012-06-12 | National University Corporation Nagoya University | System that assists in observing a luminal organ using the structure of the luminal organ |
US20090161927A1 (en) * | 2006-05-02 | 2009-06-25 | National University Corporation Nagoya University | Medical Image Observation Assisting System |
US20090156895A1 (en) * | 2007-01-31 | 2009-06-18 | The Penn State Research Foundation | Precise endoscopic planning and visualization |
US20080207997A1 (en) * | 2007-01-31 | 2008-08-28 | The Penn State Research Foundation | Method and apparatus for continuous guidance of endoscopy |
US8219179B2 (en) * | 2008-03-06 | 2012-07-10 | Vida Diagnostics, Inc. | Systems and methods for navigation within a branched structure of a body |
US8298135B2 (en) * | 2008-05-23 | 2012-10-30 | Olympus Medical Systems Corp. | Medical device with endoscope and insertable instrument |
US20110282151A1 (en) * | 2008-10-20 | 2011-11-17 | Koninklijke Philips Electronics N.V. | Image-based localization method and system |
US8102416B2 (en) * | 2010-02-22 | 2012-01-24 | Olympus Medical Systems Corp. | Medical apparatus |
US20120130171A1 (en) * | 2010-11-18 | 2012-05-24 | C2Cure Inc. | Endoscope guidance based on image matching |
US20120203067A1 (en) * | 2011-02-04 | 2012-08-09 | The Penn State Research Foundation | Method and device for determining the location of an endoscope |
US20120203065A1 (en) * | 2011-02-04 | 2012-08-09 | The Penn State Research Foundation | Global and semi-global registration for image-based bronchoscopy guidance |
US20120289777A1 (en) * | 2011-05-13 | 2012-11-15 | Intuitive Surgical Operations, Inc. | Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery |
Non-Patent Citations (1)
Title |
---|
Scott A. Merritt ; Lav Rai ; William E. Higgins "Real-time CT-video registratration for continuous endoscopic guidance" SPIE 6143, Medical Imaging 2006, vol. 6143, pp.370-384. * |
Cited By (101)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10667679B2 (en) * | 2010-01-28 | 2020-06-02 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
US20180220883A1 (en) * | 2010-01-28 | 2018-08-09 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
US8585585B2 (en) * | 2010-05-31 | 2013-11-19 | Olympus Medical Systems Corp. | Endoscopic form detection device and form detecting method of insertion section of endoscope |
US20120059221A1 (en) * | 2010-05-31 | 2012-03-08 | Olympus Medical Systems Corp. | Endoscopic form detection device and form detecting method of insertion section of endoscope |
US11051681B2 (en) | 2010-06-24 | 2021-07-06 | Auris Health, Inc. | Methods and devices for controlling a shapeable medical device |
US11857156B2 (en) | 2010-06-24 | 2024-01-02 | Auris Health, Inc. | Methods and devices for controlling a shapeable medical device |
US20140232840A1 (en) * | 2010-08-20 | 2014-08-21 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation in endoscopic applications |
US11690527B2 (en) | 2010-08-20 | 2023-07-04 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation in endoscopic applications |
US10898057B2 (en) | 2010-08-20 | 2021-01-26 | Veran Medical Technologies, Inc. | Apparatus and method for airway registration and navigation |
US11109740B2 (en) | 2010-08-20 | 2021-09-07 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation in endoscopic applications |
US9757021B2 (en) * | 2011-02-04 | 2017-09-12 | The Penn State Research Foundation | Global and semi-global registration for image-based bronchoscopy guidance |
US8900131B2 (en) * | 2011-05-13 | 2014-12-02 | Intuitive Surgical Operations, Inc. | Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery |
US20120289777A1 (en) * | 2011-05-13 | 2012-11-15 | Intuitive Surgical Operations, Inc. | Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery |
US20140180097A1 (en) * | 2011-10-17 | 2014-06-26 | Butterfly Network, Inc. | Volumetric imaging and related apparatus and methods |
CN102521833A (en) * | 2011-12-08 | 2012-06-27 | 东软集团股份有限公司 | Method for obtaining tracheae tree from chest CT image and apparatus thereof |
WO2013173234A1 (en) | 2012-05-14 | 2013-11-21 | Intuitive Surgical Operations | Systems and methods for registration of a medical device using rapid pose search |
EP2849668A4 (en) * | 2012-05-14 | 2016-05-04 | Intuitive Surgical Operations | Systems and methods for registration of a medical device using rapid pose search |
US10376178B2 (en) | 2012-05-14 | 2019-08-13 | Intuitive Surgical Operations, Inc. | Systems and methods for registration of a medical device using rapid pose search |
US11219385B2 (en) | 2012-08-14 | 2022-01-11 | Intuitive Surgical Operations, Inc. | Systems and methods for registration of multiple vision systems |
KR102194463B1 (en) * | 2012-08-14 | 2020-12-23 | 인튜어티브 서지컬 오퍼레이션즈 인코포레이티드 | Systems and methods for registration of multiple vision systems |
CN106562757A (en) * | 2012-08-14 | 2017-04-19 | 直观外科手术操作公司 | System and method for registration of multiple vision systems |
KR20150043245A (en) * | 2012-08-14 | 2015-04-22 | 인튜어티브 서지컬 오퍼레이션즈 인코포레이티드 | Systems and methods for registration of multiple vision systems |
KR102301021B1 (en) * | 2012-08-14 | 2021-09-13 | 인튜어티브 서지컬 오퍼레이션즈 인코포레이티드 | Systems and methods for registration of multiple vision systems |
US10278615B2 (en) * | 2012-08-14 | 2019-05-07 | Intuitive Surgical Operations, Inc. | Systems and methods for registration of multiple vision systems |
US20140051986A1 (en) * | 2012-08-14 | 2014-02-20 | Intuitive Surgical Operations, Inc. | Systems and Methods for Registration of Multiple Vision Systems |
EP2884879A4 (en) * | 2012-08-14 | 2016-04-27 | Intuitive Surgical Operations | Systems and methods for registration of multiple vision systems |
KR20200143518A (en) * | 2012-08-14 | 2020-12-23 | 인튜어티브 서지컬 오퍼레이션즈 인코포레이티드 | Systems and methods for registration of multiple vision systems |
US11896364B2 (en) | 2012-08-14 | 2024-02-13 | Intuitive Surgical Operations, Inc. | Systems and methods for registration of multiple vision systems |
EP3679881A1 (en) * | 2012-08-14 | 2020-07-15 | Intuitive Surgical Operations, Inc. | Systems and methods for registration of multiple vision systems |
WO2014028394A1 (en) | 2012-08-14 | 2014-02-20 | Intuitive Surgical Operations, Inc. | Systems and methods for registration of multiple vision systems |
CN108042092A (en) * | 2012-10-12 | 2018-05-18 | 直观外科手术操作公司 | Determine position of the medical instrument in branch's anatomical structure |
US11903693B2 (en) | 2012-10-12 | 2024-02-20 | Intuitive Surgical Operations, Inc. | Determining position of medical device in branched anatomical structure |
US9326660B2 (en) * | 2013-03-12 | 2016-05-03 | Olympus Corporation | Endoscope system with insertion support apparatus |
US20150057498A1 (en) * | 2013-03-12 | 2015-02-26 | Olympus Medical Systems Corp. | Endoscope system |
US11241203B2 (en) | 2013-03-13 | 2022-02-08 | Auris Health, Inc. | Reducing measurement sensor error |
US11426095B2 (en) | 2013-03-15 | 2022-08-30 | Auris Health, Inc. | Flexible instrument localization from both remote and elongation sensors |
US11504187B2 (en) | 2013-03-15 | 2022-11-22 | Auris Health, Inc. | Systems and methods for localizing, tracking and/or controlling medical instruments |
US11129602B2 (en) | 2013-03-15 | 2021-09-28 | Auris Health, Inc. | Systems and methods for tracking robotically controlled medical instruments |
US9516993B2 (en) | 2013-03-27 | 2016-12-13 | Olympus Corporation | Endoscope system |
US9357945B2 (en) * | 2013-04-15 | 2016-06-07 | Olympus Corporation | Endoscope system having a position and posture calculating portion |
US20150196228A1 (en) * | 2013-04-15 | 2015-07-16 | Olympus Medical Systems Corp. | Endoscope system |
US11020016B2 (en) | 2013-05-30 | 2021-06-01 | Auris Health, Inc. | System and method for displaying anatomy and devices on a movable display |
US11931139B2 (en) | 2013-09-06 | 2024-03-19 | Covidien Lp | System and method for lung visualization using ultrasound |
US11324551B2 (en) | 2013-09-06 | 2022-05-10 | Covidien Lp | Microwave ablation catheter, handle, and system |
US10098565B2 (en) | 2013-09-06 | 2018-10-16 | Covidien Lp | System and method for lung visualization using ultrasound |
US20150073269A1 (en) * | 2013-09-06 | 2015-03-12 | Covidien Lp | System and method for light based lung visualization |
US10098566B2 (en) | 2013-09-06 | 2018-10-16 | Covidien Lp | System and method for lung visualization using ultrasound |
US11864829B2 (en) | 2013-09-06 | 2024-01-09 | Covidien Lp | Microwave ablation catheter, handle, and system |
US11925452B2 (en) | 2013-09-06 | 2024-03-12 | Covidien Lp | System and method for lung visualization using ultrasound |
US10201265B2 (en) | 2013-09-06 | 2019-02-12 | Covidien Lp | Microwave ablation catheter, handle, and system |
US10561463B2 (en) | 2013-09-06 | 2020-02-18 | Covidien Lp | Microwave ablation catheter, handle, and system |
US10448862B2 (en) * | 2013-09-06 | 2019-10-22 | Covidien Lp | System and method for light based lung visualization |
US10448861B2 (en) | 2013-09-06 | 2019-10-22 | Covidien Lp | System and method for light based lung visualization |
JPWO2016151711A1 (en) * | 2015-03-20 | 2018-02-15 | オリンパス株式会社 | Image processing apparatus, image processing method, and image processing program |
US20170071504A1 (en) * | 2015-09-16 | 2017-03-16 | Fujifilm Corporation | Endoscope position identifying apparatus, endoscope position identifying method, and recording medium having an endoscope position identifying program recorded therein |
US10561338B2 (en) * | 2015-09-16 | 2020-02-18 | Fujifilm Corporation | Endoscope position identifying apparatus, endoscope position identifying method, and recording medium having an endoscope position identifying program recorded therein |
US11403759B2 (en) * | 2015-09-18 | 2022-08-02 | Auris Health, Inc. | Navigation of tubular networks |
US10796432B2 (en) | 2015-09-18 | 2020-10-06 | Auris Health, Inc. | Navigation of tubular networks |
US11464591B2 (en) | 2015-11-30 | 2022-10-11 | Auris Health, Inc. | Robot-assisted driving systems and methods |
US10813711B2 (en) | 2015-11-30 | 2020-10-27 | Auris Health, Inc. | Robot-assisted driving systems and methods |
US10806535B2 (en) | 2015-11-30 | 2020-10-20 | Auris Health, Inc. | Robot-assisted driving systems and methods |
US11771309B2 (en) | 2016-12-28 | 2023-10-03 | Auris Health, Inc. | Detecting endolumenal buckling of flexible instruments |
US11304762B2 (en) * | 2017-02-24 | 2022-04-19 | Fujifilm Corporation | Mapping image display control device, method, and program |
US20180263527A1 (en) * | 2017-03-16 | 2018-09-20 | Fujifilm Corporation | Endoscope position specifying device, method, and program |
US20180263712A1 (en) * | 2017-03-16 | 2018-09-20 | Fujifilm Corporation | Endoscope position specifying device, method, and program |
US11490782B2 (en) | 2017-03-31 | 2022-11-08 | Auris Health, Inc. | Robotic systems for navigation of luminal networks that compensate for physiological noise |
US11278357B2 (en) | 2017-06-23 | 2022-03-22 | Auris Health, Inc. | Robotic systems for determining an angular degree of freedom of a medical device in luminal networks |
US11759266B2 (en) | 2017-06-23 | 2023-09-19 | Auris Health, Inc. | Robotic systems for determining a roll of a medical device in luminal networks |
US11850008B2 (en) | 2017-10-13 | 2023-12-26 | Auris Health, Inc. | Image-based branch detection and mapping for navigation |
US11058493B2 (en) | 2017-10-13 | 2021-07-13 | Auris Health, Inc. | Robotic system configured for navigation path tracing |
US11950855B2 (en) | 2017-12-05 | 2024-04-09 | Covidien Lp | Multi-rigid registration of magnetic navigation to a computed tomography volume |
US11213358B2 (en) | 2017-12-05 | 2022-01-04 | Covidien Lp | Multi-rigid registration of magnetic navigation to a computed tomography volume |
WO2019112804A1 (en) * | 2017-12-05 | 2019-06-13 | Covidien Lp | Multi-rigid registration of magnetic navigation to a computed tomography volume |
US10646284B2 (en) | 2017-12-05 | 2020-05-12 | Covidien Lp | Multi-rigid registration of magnetic navigation to a computed tomography volume |
WO2019118677A1 (en) * | 2017-12-13 | 2019-06-20 | Covidien Lp | Systems, methods, and computer-readable media for automatic computed tomography to computed tomography registration |
US10984585B2 (en) | 2017-12-13 | 2021-04-20 | Covidien Lp | Systems, methods, and computer-readable media for automatic computed tomography to computed tomography registration |
US11481969B2 (en) | 2017-12-13 | 2022-10-25 | Covidien Lp | Systems, methods, and computer-readable media for automatic computed tomography to computed tomography registration |
US11510736B2 (en) | 2017-12-14 | 2022-11-29 | Auris Health, Inc. | System and method for estimating instrument location |
US11160615B2 (en) | 2017-12-18 | 2021-11-02 | Auris Health, Inc. | Methods and systems for instrument tracking and navigation within luminal networks |
US11576730B2 (en) | 2018-03-28 | 2023-02-14 | Auris Health, Inc. | Systems and methods for registration of location sensors |
US10827913B2 (en) | 2018-03-28 | 2020-11-10 | Auris Health, Inc. | Systems and methods for displaying estimated location of instrument |
US11712173B2 (en) | 2018-03-28 | 2023-08-01 | Auris Health, Inc. | Systems and methods for displaying estimated location of instrument |
US11950898B2 (en) | 2018-03-28 | 2024-04-09 | Auris Health, Inc. | Systems and methods for displaying estimated location of instrument |
US10898277B2 (en) | 2018-03-28 | 2021-01-26 | Auris Health, Inc. | Systems and methods for registration of location sensors |
US11793580B2 (en) | 2018-05-30 | 2023-10-24 | Auris Health, Inc. | Systems and methods for location sensor-based branch prediction |
US10905499B2 (en) | 2018-05-30 | 2021-02-02 | Auris Health, Inc. | Systems and methods for location sensor-based branch prediction |
US20210169589A1 (en) * | 2018-05-31 | 2021-06-10 | Auris Health, Inc. | Image-based airway analysis and mapping |
US11503986B2 (en) | 2018-05-31 | 2022-11-22 | Auris Health, Inc. | Robotic systems and methods for navigation of luminal network that detect physiological noise |
US10898275B2 (en) * | 2018-05-31 | 2021-01-26 | Auris Health, Inc. | Image-based airway analysis and mapping |
US10898286B2 (en) | 2018-05-31 | 2021-01-26 | Auris Health, Inc. | Path-based navigation of tubular networks |
US11864850B2 (en) | 2018-05-31 | 2024-01-09 | Auris Health, Inc. | Path-based navigation of tubular networks |
US11759090B2 (en) * | 2018-05-31 | 2023-09-19 | Auris Health, Inc. | Image-based airway analysis and mapping |
US11957319B2 (en) | 2018-12-06 | 2024-04-16 | Verathon Inc. | Endobronchial ultrasound imaging |
US11147633B2 (en) | 2019-08-30 | 2021-10-19 | Auris Health, Inc. | Instrument image reliability systems and methods |
US11944422B2 (en) | 2019-08-30 | 2024-04-02 | Auris Health, Inc. | Image reliability determination for instrument localization |
US11207141B2 (en) | 2019-08-30 | 2021-12-28 | Auris Health, Inc. | Systems and methods for weight-based registration of location sensors |
US11602372B2 (en) | 2019-12-31 | 2023-03-14 | Auris Health, Inc. | Alignment interfaces for percutaneous access |
US11660147B2 (en) | 2019-12-31 | 2023-05-30 | Auris Health, Inc. | Alignment techniques for percutaneous access |
US11298195B2 (en) | 2019-12-31 | 2022-04-12 | Auris Health, Inc. | Anatomical feature identification and targeting |
US11969217B2 (en) | 2021-06-02 | 2024-04-30 | Auris Health, Inc. | Robotic system configured for navigation path tracing |
US11969157B2 (en) | 2023-04-28 | 2024-04-30 | Auris Health, Inc. | Systems and methods for tracking robotically controlled medical instruments |
Also Published As
Publication number | Publication date |
---|---|
CN102883651B (en) | 2016-04-27 |
CN102883651A (en) | 2013-01-16 |
WO2011094518A2 (en) | 2011-08-04 |
EP2528496A2 (en) | 2012-12-05 |
JP2013517909A (en) | 2013-05-20 |
US20180220883A1 (en) | 2018-08-09 |
WO2011094518A3 (en) | 2011-11-10 |
US10667679B2 (en) | 2020-06-02 |
EP2528496A4 (en) | 2015-02-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10667679B2 (en) | Image-based global registration system and method applicable to bronchoscopy guidance | |
Mahmoud et al. | Live tracking and dense reconstruction for handheld monocular endoscopy | |
US8108072B2 (en) | Methods and systems for robotic instrument tool tracking with adaptive fusion of kinematics information and image information | |
US8792963B2 (en) | Methods of determining tissue distances using both kinematic robotic tool position information and image-derived position information | |
US8147503B2 (en) | Methods of locating and tracking robotic instruments in robotic surgical systems | |
Mori et al. | Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images | |
US11564748B2 (en) | Registration of a surgical image acquisition device using contour signatures | |
US7889905B2 (en) | Fast 3D-2D image registration method with application to continuously guided endoscopy | |
US7756563B2 (en) | Guidance method based on 3D-2D pose estimation and 3D-CT registration with application to live bronchoscopy | |
US8781167B2 (en) | Apparatus and method for determining a location in a target image | |
US20110282151A1 (en) | Image-based localization method and system | |
EP2433262B1 (en) | Marker-free tracking registration and calibration for em-tracked endoscopic system | |
WO2009045827A2 (en) | Methods and systems for tool locating and tool tracking robotic instruments in robotic surgical systems | |
JP2012525190A (en) | Real-time depth estimation from monocular endoscopic images | |
Allain et al. | Re-localisation of a biopsy site in endoscopic images and characterisation of its uncertainty | |
Khare et al. | Toward image-based global registration for bronchoscopy guidance | |
Serna-Morales et al. | Acquisition of three-dimensional information of brain structures using endoneurosonography | |
Deguchi et al. | Camera motion tracking of real bronchoscope using epipolar geometry analysis and CT-derived bronchoscopic images | |
Thompson et al. | Image Guided Robotic Radical Prostatectomy |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: THE PENN STATE RESEARCH FOUNDATION, PENNSYLVANIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HIGGINS, WILLIAM E.;KHARE, RAHUL;MERRITT, SCOTT A.;REEL/FRAME:025744/0499 Effective date: 20110127 |
|
AS | Assignment |
Owner name: NATIONAL INSTITUTES OF HEALTH (NIH), U.S. DEPT. OF Free format text: CONFIRMATORY LICENSE;ASSIGNOR:THE PENNSYLVANIA STATE UNIVERSITY;REEL/FRAME:025841/0363 Effective date: 20110222 |
|
AS | Assignment |
Owner name: TIP-BRONCUS LIMITED, HONG KONG Free format text: SECURITY AGREEMENT;ASSIGNOR:BRONCUS MEDICAL INC.;REEL/FRAME:031960/0567 Effective date: 20140109 Owner name: DINOVA VENTURE PARTNERS LP II, L.P., CHINA Free format text: SECURITY AGREEMENT;ASSIGNOR:BRONCUS MEDICAL INC.;REEL/FRAME:031960/0567 Effective date: 20140109 Owner name: AETHER CORPORATE LIMITED, CHINA Free format text: SECURITY AGREEMENT;ASSIGNOR:BRONCUS MEDICAL INC.;REEL/FRAME:031960/0567 Effective date: 20140109 Owner name: LIFETECH SCIENTIFIC (HONG KONG) CO., LTD., CHINA Free format text: SECURITY AGREEMENT;ASSIGNOR:BRONCUS MEDICAL INC.;REEL/FRAME:031960/0567 Effective date: 20140109 |
|
AS | Assignment |
Owner name: BRONCUS MEDICAL INC., CALIFORNIA Free format text: RELEASE OF SECURITY INTEREST;ASSIGNORS:LIFETECH SCIENTIFIC (HONG KONG) CO., LTD.;DINOVA VENTURE PARTNERS LP II, L.P.;TIP-BRONCUS LIMITED;AND OTHERS;REEL/FRAME:033012/0784 Effective date: 20140523 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE |