Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/12—Arrangements for detecting or locating foreign bodies
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
  • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B10/00—Other methods or instruments for diagnosis, e.g. instruments for taking a cell sample, for biopsy, for vaccination diagnosis; Sex determination; Ovulation-period determination; Throat striking implements
  • A61B10/02—Instruments for taking cell samples or for biopsy
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
  • A61B6/03—Computed tomography [CT]
  • A61B6/032—Transmission computed tomography [CT]
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/40—Arrangements for generating radiation specially adapted for radiation diagnosis
  • A61B6/4064—Arrangements for generating radiation specially adapted for radiation diagnosis specially adapted for producing a particular type of beam
  • A61B6/4085—Cone-beams
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/44—Constructional features of apparatus for radiation diagnosis
  • A61B6/4417—Constructional features of apparatus for radiation diagnosis related to combined acquisition of different diagnostic modalities
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/46—Arrangements for interfacing with the operator or the patient
  • A61B6/461—Displaying means of special interest
  • A61B6/463—Displaying means of special interest characterised by displaying multiple images or images and diagnostic data on one display
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/46—Arrangements for interfacing with the operator or the patient
  • A61B6/461—Displaying means of special interest
  • A61B6/466—Displaying means of special interest adapted to display 3D data
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/48—Diagnostic techniques
  • A61B6/486—Diagnostic techniques involving generating temporal series of image data
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
  • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
  • A61B6/5229—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image
  • A61B6/5247—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
  • A61B6/5258—Devices using data or image processing specially adapted for radiation diagnosis involving detection or reduction of artifacts or noise
  • A61B6/5264—Devices using data or image processing specially adapted for radiation diagnosis involving detection or reduction of artifacts or noise due to motion
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
  • A61B8/12—Diagnosis using ultrasonic, sonic or infrasonic waves in body cavities or body tracts, e.g. by using catheters
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/11—Region-based segmentation
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/20—Analysis of motion
  • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
  • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
  • A61B2034/107—Visualisation of planned trajectories or target regions
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
  • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
  • A61B2034/2046—Tracking techniques
  • A61B2034/2051—Electromagnetic tracking systems
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
  • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
  • A61B2034/2046—Tracking techniques
  • A61B2034/2065—Tracking using image or pattern recognition
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
  • A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
  • A61B2090/3966—Radiopaque markers visible in an X-ray image
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/48—Diagnostic techniques
  • A61B6/486—Diagnostic techniques involving generating temporal series of image data
  • A61B6/487—Diagnostic techniques involving generating temporal series of image data involving fluoroscopy
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10016—Video; Image sequence
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10072—Tomographic images
  • G06T2207/10081—Computed x-ray tomography [CT]
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10116—X-ray image
  • G06T2207/10121—Fluoroscopy
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10132—Ultrasound image
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30021—Catheter; Guide wire
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30061—Lung
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30096—Tumor; Lesion

Definitions

• the present invention relates to endobronchial and trans-bronchial navigation of devices.
• the present invention relates in particular to a device for facilitating navigation of an intrathoracic device inside a bronchial pathway, to a system for navigation of an intrathoracic device inside a bronchial pathway and to a method for navigation of an intrathoracic device inside a bronchial pathway
• Peripheral pulmonary lesions may be diagnosed by non-surgical biopsy.
• the peripheral pulmonary nodule is localized and identified as needing further diagnosis by pre-operative CT scans.
• An example for an endobronchial approach to performing non-surgical biopsy is being minimally invasive.
• An example of endobronchial approaches is a fluoroscopy guided intrathoracic biopsy. Conventional fluoroscopy (i.e. X-ray based) is used to help and guide transbronchial navigation and biopsy in the diagnostics of peripheral pulmonary lesions. The endoscopist is then guided by 2D fluoroscopy for a coarse approximation of where the diagnostic tools and the to-be-diagnosed lesions should be, and how they are positioned relative to another.
• an endobronchial ultrasound radial mini-probe might be utilized to help guide the physician.
• rEBUS might be utilized alone or in combination with fluoroscopy. It is not necessarily a navigation method, but can provide valuable confirmatory information on if a lesion has been reached. Either one of approaches helps in further guiding accurate endobronchial biopsy. Combining them provides additive information, as 2D fluoroscopy can help guide coarse navigation and localization whilst rEBUS imaging can provide local confirmatory imaging.
• WO 2018215832 A2 relates to methods for using radial endobronchial ultrasound probes for 3D REM construction of images and improved target localization. However, it has been shown that navigation can still become cumbersome for the operator.
• a device for facilitating navigation of an intrathoracic device inside a bronchial pathway comprises an image data provider, a processor and an output data provider.
• the image data provider is configured to receive a sequence of 2D X-ray images of the region of interest of the bronchial structure with a device inserted in a bronchial pathway.
• the intrathoracic device is visible in the X-ray images.
• the processor is configured to track the intrathoracic device in at least a part of the sequence of 2D X-ray images.
• the processor is also configured to assess direction and magnitude of repetitory cardiovascular and respiratory induced motion based on the tracked intrathoracic device.
• the processor is further configured to model the assessed motion and to generate a navigation information indicative of a range of the modelled motion.
• the output data provider is configured to provide the navigation information as a confidence reference to a user operating the intrathoracic device.
• bronchial pathway relates to the lungs, for example, but also other parts of the pathway for breathing in and out.
• intracranial relates to being within the thorax and relates to e.g. intrapulmonary, intrathoracic and also transthoracic. Also, the term comprises transbronchial.
• the intrathoracic device thus relates to an endobronchial or transbronchial device.
• the processor is also configured to track an intrathoracic region of interest in at least a part of the sequence of 2D X-ray images.
• the processor is configured to enable distinction of the repetitory motion from single event motion, such as induced by endobronchial or transbronchial device manipulation or any of its guiding elements.
• the processor is further configured to enable modelling of found device positioning relative to a pre-procedural imaging source.
• a pre-procedural imaging source for motion modelling, in addition to following the device, information is provided how a bronchial tree actually looks like.
• the confidence interval positioning is related to pre-procedural knowledge of bronchial tree positioning.
• the endobronchial device is tracked, then positioned at main carina, and image data is generated. Further, it is positioned at part of or all of the subcarinas of every pulmonary segment (e.g. right upper lobe, right middle lobe, right lower lobe, left upper lobe, left lower lobe), and image data is generated. Further, it is provided to register, e.g. automatically register, pre-procedural anatomical image to fluoroscopy images, as it is known how the pre-procedural segmentation of lungs would fit in the fluoroscopy images.
• register e.g. automatically register, pre-procedural anatomical image to fluoroscopy images, as it is known how the pre-procedural segmentation of lungs would fit in the fluoroscopy images.
• Fluoroscopy imaging is an X-ray imaging that is also referred to as fluoro imaging or fluoro.
• anatomical features such as the lungs are tracked based on features to increase the model’s accuracy.
• This provides essential information to the user when navigating, which hence means relief and improvement in navigation of bronchial and lung structures.
• the indication of a possible space in which the device, like an imaging probe or an extractor, is located provides guidance for the operator in an intuitive way.
• coarse features in the intrathoracic region of interest are taking into account in order to be able to give an ,error‘ or adjust the confidence interval if a dislocation of the coarser pulmonary features is detected. This is for example the case when a catheter is rotated with predefined (passive) angulation. The lung segment or lesion can be seen to move as the catheter rotates.
• the image data provider is configured to receive pre-procedural imaging information of the patient, i.e. PET-CT, CT (or a variation thereof, i.e. cone beam CT) and/or MRI.
• pre-procedural imaging information of the patient i.e. PET-CT, CT (or a variation thereof, i.e. cone beam CT) and/or MRI.
• the confidence reference is provided as an indication of an envelope enclosing the spatial range of the modelled motion.
• the processor is configured to track the intrathoracic device in at least a part of the sequence of 2D X-ray images; and to assess direction and magnitude of non-repetitory lung tissue and device motion as induced by intrathoracic device manipulation or its guiding elements, i.e. endoscope; and to track the motion and generating a navigation information which is adjusted for the non-repetitory motion induced by endobronchial or transbronchial device manipulating or its guiding elements.
• the image data provider is further configured to provide pre-operative 3D image data comprising a segmented bronchial structure of the subject in the region of interest.
• the processor is configured to register the 2D X-ray images with the pre-operative 3D image data.
• the processor is further configured to determine a spatial position of the intrathoracic device in relation to the segmented bronchial structure of the pre-operative 3D image data based on the 2D X-ray projection of the transthoracic device.
• the output data provider comprises a display that is configured to display the confidence reference in a projection of the segmented bronchial structure, which are both overlaid to a current one of the sequences of 2D X-ray images.
• the image data provider is further configured to provide 3D (reconstructed) image data originating from intra operative X-ray fluoroscopy such as Cone beam computed tomography and, as an option, further comprising a segmented bronchial structure of the subject in the region of interest.
• the processor is configured to register the 2D X-ray images with the (reconstructed) 3D image data.
• the processor is also configured to determine a spatial position of the transthoracic device in relation to the segmented bronchial structure of the 3D image data based on the 2D X-ray projection of the transthoracic device.
• the output data provider comprises a display that is configured to display the confidence reference overlaid in 3D form on the display of the 3D image data.
• the image data is originating from a sweep of a C-arm acquiring at least multiple projections under multiple different angles, possibly further providing a segmented bronchial structure of the subject, such as a patient, or 3D tool location in the region of interest.
• the image data provider is configured to receive, as the sequence, a live stream of 2D X-ray images.
• the processor is configured to continuously track the intrathoracic device in the images.
• the processor is further configured to update the navigation information accordingly.
• the output data provider is configured to continuously provide the navigation information as an updated confidence reference to a user operating the intrathoracic device.
• the X-ray information is further updated and/or integrated with electromagnetic tracking information.
• an offline operation from X-ray is provided by using electromagnetic navigation.
• passive impedance based localization of intrathoracic devices is used.
• a system for navigation of an intrathoracic device inside a bronchial pathway comprises an X-ray imaging device for imaging a thorax area of interest of a subject and an intrathoracic device for insertion in a bronchial pathway, such as the lungs of a subject.
• the system also comprises a display and a device for facilitating navigation according to one of the preceding examples.
• the intrathoracic device is at least partly radio-opaque.
• the X-ray imaging device provides a sequence of 2D X-ray images of the region of interest of the bronchial structure with the intrathoracic device inserted in the bronchial pathway.
• the output data provider provides the navigation information, i.e. the visual indication, to the display.
• the display is a monitor or display as used by the operator.
• the display provides at least one of the sequences of the 2D X-ray images and the visual indication as a confidence reference overlaid to the 2D X-ray image.
• the intrathoracic device is an ultrasound imaging probe configured to provide radial endobronchial ultrasonography data from within bronchial structure, e.g. the lungs.
• the intrathoracic device is an ultrasound imaging probe configured to provide radial and/or forward looking endobronchial ultrasonography data from within a bronchial structure.
• the intrathoracic device is an endobronchial ultrasound device, also referred to as EBUS.
• the device may be configured for radial and/or forward looking ultrasound imaging.
• the intrathoracic device is an endobronchial ultrasound mini probe, i.e. radial endobronchial ultrasound mini device or forward looking endobronchial ultrasound mini device.
• the device may be configured for radial and/or forward looking ultrasound imaging.
• the intrathoracic device comprises an ultrasound imaging probe and a tissue or cell extraction device.
• the tissue or cell extraction device is a biopsy device.
• a method for navigation of an intrathoracic device inside a bronchial pathway comprises the following steps:
• potential and/or existing device radio-opaque properties are used for automated tracking under fluoroscopy in bronchoscopy navigation for enhancing navigation of e.g. known catheters, devices and/or radial endobronchial ultrasonography devices to the lesion, providing lesion access confirmation and/or help guide biopsy extraction.
• computer vision and image processing for 2D and/or 3D tracking of the catheter is deployed. Further, it is provided to augment fluoroscopy images and to display with the results of device tracking in 2D and/or 3D.
• results of device tracking in 2D/3D are registered with results of pre procedural computed tomography (CT) planning, 2D fluoroscopy under multiple angles (continuously or at distinct angles) and/or cone beam CT scans and/or electromagnetic navigation bronchoscopy.
• CT computed tomography
• a minimum of 30° wide sweep under continuous fluoroscopy is provided to have a 3D update of lesion and device positioning.
• a minimum of less than a 30° wide sweep is provided, such as a 20° wide sweep.
• the present invention relates to navigating endo- or transbronchially.
• the navigation relates to navigating in bronchial pathways.
• a sequence of 2D X-ray images of a region of interest of a bronchial structure with an intrathoracic device (visible in the X-ray images) inserted in a bronchial pathway is provided.
• the intrathoracic device is tracked in the 2D X-ray images and direction and magnitude of repetitory cardiovascular and respiratory induced motion is assessed based on the tracked intrathoracic device.
• the assessed motion is modelled and a navigation information indicative of the range of modelled motion is generated.
• the navigation information of potential device positioning is presented to the operator, for example an EBUS or rEBUS historical trajectory confidence reference, to a user operating the intrathoracic device.
• an X-ray 2D image of a thorax region is registered and shown overlaid with a static segmentation of the bronchial structure derived from use of cone beam CT, i.e. cone beam fluoroscopy in case of X-ray imaging (i.e. with a C-arm system) normally in combination with pre-operative CT.
• an augmented fluoroscopy 2D image (302) of a thorax region is registered and shown overlaid with segmentation (304) of the bronchial structure and target lesion (312).
• a confidence reference (314) of the current position of the rEBUS catheter may also be shown.
• Fig. 1 schematically shows a device for facilitating navigation of an intrathoracic device inside a bronchial pathway.
• Fig. 2 shows an example of a system for navigation of an intrathoracic device inside a bronchial pathway.
• Fig. 3 schematically shows basic steps of a method for navigation of an intrathoracic device inside a bronchial pathway.
• Fig. 4 schematically shows an example of an augmented live fluoroscopy 2D image of a thorax region with a confidence reference overlaid to the 2D X-ray image around the current catheter position.
• subject may also be referred to as individual.
• the subject may further also be referred to as patient, although it is noted that this term does not indicate whether any illness or disease is actually present with the subject.
• Fig. 1 schematically shows a device 10 for facilitating navigation of an intrathoracic device inside a bronchial pathway.
• the device 10 for facilitating navigation comprises an image data provider 12, a processor 14 and an output data provider 16.
• the image data provider 12 is configured to receive a sequence of 2D X-ray images of the region of interest of the bronchial structure with an intrathoracic device inserted in a bronchial pathway.
• the intrathoracic device is visible in the X-ray images.
• the processor 14 is configured to track the intrathoracic device in at least a part of the sequence of 2D X-ray images; to assess direction and magnitude of repetitory cardiovascular and respiratory induced motion based on the tracked intrathoracic device; and to model the assessed motion and generating a navigation information indicative of a range of the modelled motion.
• the output data provider 16 is configured to provide the navigation information as a confidence reference to a user operating the intrathoracic device.
• the “device for facilitating navigation” is also referred to as “navigation device”.
• the “output data provider” is also referred to as “navigation facilitator”.
• the “navigation information indicative of a range of the modelled motion” is also referred to as “visual indicator of a range of the modelled motion”.
• the sequence of images is a sequence of live images.
• the assessed motion can also be referred to as estimated range of motion.
• the intrathoracic device is an ultrasound imaging probe that provides radial endobronchial ultrasonography data from within the bronchial structure.
• three radio-opaque markers are provided in a predetermined spatial arrangement along the intrathoracic device.
• the location, position and orientation can be tracked based on the X-ray radiation images.
• a plane By tracking the intrathoracic device, a plane can be identified to be shown on or overlaid to the X-ray images such as fluoroscopic images.
• the location of the intrathoracic device is registered with the X- ray images.
• the tip of the intrathoracic device along with radio-opaque markers are used for registering. The intrathoracic device is thus registered within the subject’s anatomy.
• the confidence reference replaces a dynamic model of the anatomy.
• the confidence reference can also be referred to as confidence map.
• the confidence reference is provided as an indication of an envelope enclosing the spatial range of the modelled motion.
• the confidence reference is provided as an indication of an envelope enclosing the spatial range of the modelled motion in relation to the target, e.g. the target lesion and/or the current location of the catheter, or trajectory of the catheter.
• the confidence reference is shown overlaid to a fluoroscopy image.
• the confidence reference is provided as an indication of an envelope enclosing the spatial range of the modelled motion in relation to the subject.
• the confidence reference is provided as an indication of an envelope enclosing the spatial range of the modelled motion in relation to a segmented object such as lesion overlaid to the fluoroscopy image.
• the range of motion is thus also displayed in relation to the X-ray imaging such as fluoroscopy images.
• the range of motion is provided as a motion envelope.
• the output data provider 16 comprises a display 18 that is configured to display the confidence reference overlaid to a current one of the sequences of 2D X-ray images.
• the display 18 is data- connected to the output data provider 16, as indicated with a dashed connection line 22.
• a first arrow 20 is shown in dashed lines to indicate a data input to the image data provider 12.
• a data connection to an imaging system may be provided, e.g. wireless or as wire connection.
• a frame 24 indicates an integration of the data provider 12, the processor 14 and the output data provider 16.
• the processor 14 and the output data provider 16 are provided in a common housing.
• they are combined and integrated as separate modules.
• the current one of the sequences of 2D X-ray images can also be referred to as live image.
• the 2D X-ray images are provided from a C-arm in a single imaging positioning.
• the 2D X-ray images are provided from a C-arm in multiple imaging positionings.
• the image data provider 12 is further configured to provide pre operative 3D image data comprising a segmented bronchial structure of the subject in a region of interest.
• the processor 14 is configured to register the 2D X-ray images with the pre-operative 3D image data; and to determine a spatial position of the intrathoracic device in relation to the segmented bronchial structure of the pre-operative 3D image data based on the 2D X-ray projection of the intrathoracic device.
• the output data provider 16 comprises the display 18 that is configured to display the confidence reference in a projection of the segmented bronchial structure, which are both overlaid to a current one of the sequence of 2D X-ray images. In an example, a consistent position of the 2D X-ray projection is provided, or a moving sweep from which a 3D reconstruction is made.
• the image data provider 12 is further configured to provide 3D image data originating from intra operative X-ray fluoroscopy such as cone beam computed tomography comprising a segmented bronchial structure of the subject in the region of interest.
• the processor 14 is configured to register the 2D X-ray images with the 3D image data; and to determine a spatial position of the intrathoracic device in relation to the segmented bronchial structure of the 3D image data based on the 2D X-ray projection of the intrathoracic device.
• the output data provider 16 comprises the display 18 that is configured to display the confidence reference overlaid in 3D form on the display of the 3D image data.
• Cone beam computed tomography is also referred to as cone beam CT, CBCT or cbCT.
• the fluoroscopy images are augmented by the pre-operative 3D CT image data.
• the image data provider 12 is configured to receive, as the sequence, a live stream of the 2D X-ray images.
• the processor 14 is configured to continuously track the intrathoracic device; and to update the navigation information accordingly.
• the output data provider 16 is configured to continuously provide the navigation information as an updated confidence reference to a user operating the intrathoracic device.
• a single consistent position is provided for the sequence, i.e. the live stream of the 2D X-ray images.
• a multi-X-ray positioning is provided.
• the term “continuously” relates to a rate of a number of tracking steps per second such that in the user’s experience, a real-time tracking is provided.
• the term “operator” refers to a person operating the device for facilitating navigation.
• the operator is a technician, a physician assistance, a staff member, a nurse, a physician or the like.
• Fig. 2 shows an example of a system 100 for navigation of an intrathoracic device inside a bronchial pathway.
• the system 100 comprises an X-ray imaging device 102 for imaging a thorax area of interest of a subject.
• the system 100 also comprises an intrathoracic device 104 for insertion in a bronchial pathway of a subject.
• the system 100 comprises a device 106 for facilitating navigation according to one of the preceding examples and a display 108.
• the intrathoracic device 104 is at least partly radio opaque.
• the X-ray imaging device 102 provides a sequence of 2D X-ray images of the region of interest of the bronchial structure with the intrathoracic device 104 inserted in the bronchial pathway.
• the device 106 for facilitating navigation provides the navigation information to the display 108.
• the display 108 provides at least one of the sequences of the 2D X-ray images and the navigation information as a confidence reference overlaid to the 2D X-ray image.
• radio-opaque refers to at least partly opacity for X-ray radiation. Briefly said, the term refers to being visible e.g. in (X-ray) fluoroscopic images.
• the X-ray imaging device 102 is provided as a C-arm structure with a C-arm 110 mounted to a movable ceiling support 112.
• An X-ray source 114 and an X- ray detector 116 are provided at opposing ends of the C-arm 110.
• a subject support 118 is provided, e.g. a patient table.
• a table-side display and interface module 120 may be arranged.
• adjustable lighting 122 is provided above.
• the device 106 for facilitating navigation is provided as a control station 124, e.g. arranged sideward.
• the control station 124 comprises display and interface components such as a keyboard 126, a mouse 128, a control panel 130 and display units 132, to allow operation and control of the multiple equipment, e.g. in the examination room or intervention room or operation room.
• a subject 134 may be arranged on the subject support 118.
• the intrathoracic device 104 e.g. an ultrasound imaging probe or an extraction device, may be inserted into the bronchial pathway of the subject 134 (not shown in detail).
• the X-ray imaging device 102 provides image data about the situation inside the subject, i.e. inside the body of the subject, which situation is not visible for the operator from outside.
• the image data is provided on the display 108, e.g. as fluoroscopic images, such as shown in Fig. 4.
• the X-ray imaging device 102 is configured to provide the sequence of 2D X-ray images of the region of interest of the bronchial structure with the intrathoracic device inserted in the bronchial pathway to the device for facilitating navigation.
• the device 106 for facilitating navigation is configured to provide the navigation information to the display 108.
• the display 108 is configured to provide at least one of the sequences of the 2D X-ray images and the navigation information as a confidence reference overlaid to the 2D X-ray image.
• distinct and spaced X-ray radiation opaque markers on the intrathoracic device 104 may be provided, to be automatically and continuously tracked under fluoroscopy by computer vision.
• a catheter is used for pulmonary purposes that is visible under fluoroscopy through appearance of three markers, e.g. 10 mm apart from each other, and an ultrasound transducer located distally at a fixed distance from the markers may also be X-ray radiation opaque, i.e. radio-opaque.
• Computer vision can then recognize the RO signature of the catheter and track its movement automatically both in 2D and potentially in 3D thus also providing spatial location and orientation.
• the intrathoracic device 104 is an ultrasound imaging probe configured to provide radial endobronchial ultrasonography data from within a bronchial structure.
• Endobronchial ultrasonography is also referred to as EBUS; as indicated above, radial endobronchial ultrasonography is referred to as rEBUS, REBUS or R-EBUS.
• a miniature ultrasound probe e.g. an EBUS or radial EBUS probe
• a working channel of a flexible bronchoscope or catheter which is also referred to as guide sheath.
• the staff e.g. a clinician, can determine a lesion's location and size.
• the radial endobronchial ultrasonography data from within the bronchial structure is visualized and also shown on the display 108.
• the intrathoracic device 104 comprises an ultrasound imaging probe and a biopsy extraction device.
• Fig. 3 schematically shows basic steps of a method 200 for navigation of an intrathoracic device inside a bronchial pathway. The method 200 comprises the following steps:
• a sequence of live 2D X-ray images of the region of interest of the bronchial structure with an intrathoracic device inserted in a bronchial pathway is received.
• the intrathoracic device is visible in the X-ray images.
• a second step 204 also referred to as step b
• the intrathoracic device is tracked in at least a part of the sequence of live 2D X-ray images.
• step c direction and magnitude of repetitory cardiovascular and respiratory induced motion are assessed based on the tracked intrathoracic device.
• step 208 the assessed motion is modelled, and a navigation information indicative of the modelled motion is generated.
• step e the navigation information is provided as a confidence reference to a user operating the intrathoracic device.
• intra operative X-ray fluoroscopy such as cone beam CT
• Fig. 4 schematically shows a display 300 of an augmented live fluoroscopy 2D image 302 of a thorax region with an overlaid segmented bronchial tree 304 and target lesion.
• the overlay 304 comprises a bronchial structure 306.
• a calculated EBUS/rEBUS pathway 308 after registration is indicated in a dashed line pattern.
• An EBUS/rEBUS historical trajectory confidence reference is shown as an example of the confidence reference 310.
• a current position confidence reference 314 relating to the current position of the EBUS/rEBUS catheter is also shown.
• the confidence references are displayed as 2D overlays but in other examples can also be projected as 3D structures.
• the augmented X-ray image i.e. the fluoroscopy 2D image 302 shows an anatomical structure with slightly visible vertebras 316, ribs 318 and other structures.
• the static overlay 304 is provided as segmented visualization of the bronchial structure 306 and target lesion 312.
• the static overlay 304 may be provided colored in contrast to the more or less greyscale only X-ray image. Further, information about a current C-arm orientation 320 or a subject orientation indicator 322 can be shown overlaid.
• Fig. 4 provides an augmented display of a live 2D Fluoroscopy image with a segmented bronchial pathway and a segmented target lesion derived from use of CbCT normally in combination with pre-operative CT and a rEBUS catheter with its transducer and radio-opaque markers.
• the confidence reference supports in navigating the rEBUS catheter to the target lesion, confirm its location and then e.g. perform biopsy samples with biopsy tools.
• tissue motion confidence references in 2D and/or 3D, indicated in a dotted line can be established around the current presumed location of the ultrasound transducer and can further clarify to the operator from what spatial region the rEBUS imaging (not shown) may originate from.
• this confidence reference can also apply to the lesion as it is also presumed moving in a similar fashion.
• This confidence reference can be calculated for numerous locations of the rEBUS transducer, and if these historical locations are collected, they can be combined and augmented on the fluoroscopy display to provide an overall confidence reference for the route taken as demonstrated in the dotted region outlining the rEBUS pathway to its left and right
• tracking in 3D of the radio-opaque profile of the catheter and registering it with either the pre-planned pathway or latest cone beam CT constructed position/updated pathway can provide the operator with guidance in real time on how to spatially steer the catheter in real time and/or result in a newly constructed calculated pathway in order to arrive at the target lesion with maximal confidence and minimal cone beam CT scans.
• Flexible video bronchoscopes in combination with dedicated video processor can be used during procedures for guiding the navigational and other consecutive tools, such as a brush, TBNA and or biopsy forceps.
• Introduction of the bronchoscope can be through the mouth, laryngeal mask or endotracheal tube into the bronchial tree.
• Navigation may be performed on basis of pulmonary physician pre-operative CT-scan knowledge, videoscopic imaging provided by the bronchoscope and real-time fluoroscopic imaging as provided by cone beam CT. If visualization by rEBUS is found possible and confirmed, further steps are consecutively performed. When confirmation of adequate tissue sample is provided, additional tissue samples for cytopathology and molecular analysis can be collected, after which the procedure can be terminated. If it shows that the target lesion is not reached, a switch to the navigation techniques and dedicated tools is made. A preceding cbCT scan will be used for calibration of the navigation technology, to prevent radiation as much as possible.
• the pulmonary physician can use a steerable or non-steerable catheter or tool as inserted through the endoscope’s working channel for navigation.
• Live updates on 3D position may be provided by intermittent 2D fluoroscopy.
• the fluoroscopy images can be recombined in real time with a CT scan 3D volume made earlier using image fusion and image processing, providing updates on catheter and lesion position.
• the probe After reaching the target by following the reconstructed pathway as visualized on screen, the probe can be inserted into the catheter (that has been inserted through the endoscope’s working channel) for visualization confirmation.
• tissue samples can be obtained. Imaging confirmation can be fluoroscopy based, for example. If the peripheral pulmonary lesion does not show visible on the probe’s image and/or seems misaligned, repositioning of the catheter is performed and navigation accuracy and probe visualization can be re-assessed.
• repeated tissue biopsy may be performed, and yield may be evaluated.
• the probe e.g. a rEBUS probe
• the probe does not always provide a clear and unambiguous view of the lesion even though it should be visualized due to the localization of the peripheral pulmonary lesions in the parenchyma without direct contact with the bronchial tree which need to be accessed using transbronchial and transparenchymal access under cone beam CT 3D-guidance, since there is no tissue interface for ultrasound.
• Another aspect is the lack of tissue contrast for ultrasound in case of so-called ground glass opacities, which are an increasing part of the peripheral pulmonary lesion problem.
• a further advantage is provided concerning the 3D distal tip orientation.
• navigating it is often unclear to the physician if insertion of tools leads to a 1 : 1 translation in tissue.
• out of plane movement may be poorly visualized.
• angulation/orientation relates to bronchus bifurcations / the lesion is thus provided.
• repeated biopsy may be provided.
• lesion sizing might be as small as 5 mm and tools not positioned directly inside the lesion but only adjacent to the lesion, exact positioning is important.
• the confidence reference compensates for a lack of knowledge regarding the exact lesion positioning and angulation to the catheter, as it not visible on fluoroscopy, and repeated cone beam CT is not viable, e.g. may be not preferred to provide in view of adding to radiation dose of subject and also staff members. The confidence reference thus also helps in view of the attempt to avoid to systematically biopsy repeatedly around the area of interest if not really needed.
• Tracking of the catheter can be provided in an automatic manner.
• the tracking may thus provide an assessment of both direction and magnitude of cardiovascular and respiratory induced motion by tracking a rEBUS catheter movement continuously when the catheter is left stationary.
• the catheter should be placed near to the lesion or at an approximated pathway that should be taken. Under fluoroscopy, one respiratory (and thus several cardiac cycles) should be monitored. This motion can be modelled and augmented on the fluoroscopy display as confidence references which can be utilized to improve rEBUS catheter navigation to the lesion and provide further information in what region around the target lesion to perform biopsy in, as the lesion is also continuously moving.
• target lesion location verification and biopsy extraction process is provided while circumventing additional X-ray exposures, like cone beam CT, by augmenting on fluoroscopy the spatial positions visited by the rEBUS catheter and registering them with the pre-procedural CT planning and/or cone beam CT scans and/or electromagnetic navigation.
• real-time ultrasound visualization of the lesion and position confirmation and biopsy extraction are facilitated if e.g. a tracked catheter is combined with a biopsy extracting mechanism.
• a catheter is provided that is able to guide a ‘re-entry’ needle.
• This can be used in the same fashion as in an EBUS transbronchial needle aspiration (TBNA), which is a staging procedure, for puncturing the lesion and consecutive guidance of a guidewire into that lesion.
• TBNA transbronchial needle aspiration
• a secondary step would then be to guide a needle over that guidewire into the lesion directly, giving an improved local confirmation of accurate tissue biopsy. For example, an accuracy of nearly up to 100% is achievable.
• potential and/or existing catheter radio-opaque properties are used for automated tracking under fluoroscopy in bronchoscopy navigation for enhancing rEBUS catheter navigation to the lesion, lesion confirmation and biopsy extraction.
• computer vision and image processing for 2D and/or 3D tracking of the catheter is deployed. Further, it is provided to augment fluoroscopy images and to display with the results of catheter tracking in 2D and/or 3D.
• results of catheter tracking in 2D/3D are registered with results of pre-procedural CT planning and/or cone beam CT scans and/or electromagnetic navigation.
• Automated rEBUS tracking is provided, e.g. as an endobronchial navigational software package. The results are visible on the display.
• the provision of the confidence reference is provided, as example, to enhance existing cone beam CT system functionality and also to provide improvement in the field of lung inspection and treatment, e.g. for endo bronchial navigation.
• the catheter tracking with rEBUS resulting in the confidence reference is provided to enhance fluoroscopy augmented display further.
• results of rEBUS catheter tracking are registered with existing calculated and displayed pathways, e.g. either pre-operative and/or achieved via intra operative cone beam CT scans.
• a still further option is to enhance spatial/3D navigation of the rEBUS catheter by further integration with cone beam CT.
• the provision of the confidence reference also improves bronchial usage of disposables such as ultrasound catheters and their future variations.
• the rEBUS device is integrated with electromagnetic tracking of the device.
• an adjustment in 2D fluoroscopy imaging angle is provided to provide improved accuracy of the visualized model.
• operator induced deformation of the lung/devices is distinguished from repetitory respiratory / cardiac motion.
• a pre-procedural segmentation of a bronchial tree is overlaid on fluoroscopy.
• rEBUS confirmatory of lesion access is provided, its position information is provided as input to the system such to track where and how the rEBUS is located in the model. In an example, it is provided to extract global features like bigger vessels. These can be used for updating the model accuracy, e.g. pre-procedural bronchial tree segmentation combined with vessel segmentation, and local deformation.
• an automatic zoom and collimation function is provided for the navigation.
• a computer program enabling a processor to carry out the method of one of the examples above is provided.
• a computer program or program element for controlling an apparatus according to one of the examples above is provided, which program or program element, when being executed by a processing unit, is adapted to perform the method steps of one of the method examples above.
• a computer readable medium having stored the program element of the above example is provided.
• a computer program or a computer program element is provided that is characterized by being adapted to execute the method steps of the method according to one of the preceding embodiments, on an appropriate system.
• the computer program element might therefore be stored on a computer unit or be distributed over more than one computer units, which might also be part of an embodiment of the present invention.
• This computing unit may be adapted to perform or induce a performing of the steps of the method described above. Moreover, it may be adapted to operate the components of the above described apparatus.
• the computing unit can be adapted to operate automatically and/or to execute the orders of a user.
• a computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method of the invention.
• aspects of the invention may be implemented in a computer program product, which may be a collection of computer program instructions stored on a computer readable storage device which may be executed by a computer.
• the instructions of the present invention may be in any interpretable or executable code mechanism, including but not limited to scripts, interpretable programs, dynamic link libraries (DLLs) or Java classes.
• the instructions can be provided as complete executable programs, partial executable programs, as modifications to existing programs (e.g. updates) or extensions for existing programs (e.g. plugins).
• parts of the processing of the present invention may be distributed over multiple computers or processors.
• the processing unit for instance a controller implements the control method.
• the controller can be implemented in numerous ways, with software and/or hardware, to perform the various functions required.
• a processor is one example of a controller which employs one or more microprocessors that may be programmed using software (e.g., microcode) to perform the required functions.
• a controller may however be implemented with or without employing a processor, and also may be implemented as a combination of dedicated hardware to perform some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions.
• controller components that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs).
• ASICs application specific integrated circuits
• FPGAs field-programmable gate arrays
• This exemplary embodiment of the invention covers both, a computer program that right from the beginning uses the invention and a computer program that by means of an up-date turns an existing program into a program that uses the invention.
• the computer program element might be able to provide all necessary steps to fulfil the procedure of an exemplary embodiment of the method as described above.
• a computer readable medium such as a CD-ROM
• the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section.
• a computer program may be stored and/or distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.
• the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network.
• a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.

Landscapes

• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Engineering & Computer Science (AREA)
• Medical Informatics (AREA)
• Surgery (AREA)
• Physics & Mathematics (AREA)
• Veterinary Medicine (AREA)
• Public Health (AREA)
• Molecular Biology (AREA)
• General Health & Medical Sciences (AREA)
• Animal Behavior & Ethology (AREA)
• Biomedical Technology (AREA)
• Heart & Thoracic Surgery (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• Pathology (AREA)
• Radiology & Medical Imaging (AREA)
• Biophysics (AREA)
• Optics & Photonics (AREA)
• High Energy & Nuclear Physics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Human Computer Interaction (AREA)
• Dentistry (AREA)
• Oral & Maxillofacial Surgery (AREA)
• Pulmonology (AREA)
• Robotics (AREA)
• Multimedia (AREA)
• Apparatus For Radiation Diagnosis (AREA)
• Ultra Sonic Daignosis Equipment (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=68944254

Family Applications (1)

Country Status (5)

Cited By (4)

Citations (3)

Family Cites Families (14)

• 2019
  • 2019-12-17 EP EP19217043.9A patent/EP3838159A1/de not_active Withdrawn
• 2020
  • 2020-12-11 JP JP2022536770A patent/JP2023506079A/ja active Pending
  • 2020-12-11 EP EP20821208.4A patent/EP4076203A1/de active Pending
  • 2020-12-11 WO PCT/EP2020/085661 patent/WO2021122344A1/en unknown
  • 2020-12-11 US US17/786,052 patent/US20230135733A1/en active Pending
  • 2020-12-11 CN CN202080087689.XA patent/CN114845640A/zh active Pending

Patent Citations (3)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events