US10278778B2 - Medical device navigation using a virtual 3D space - Google Patents
Medical device navigation using a virtual 3D space Download PDFInfo
- Publication number
- US10278778B2 US10278778B2 US15/415,398 US201715415398A US10278778B2 US 10278778 B2 US10278778 B2 US 10278778B2 US 201715415398 A US201715415398 A US 201715415398A US 10278778 B2 US10278778 B2 US 10278778B2
- Authority
- US
- United States
- Prior art keywords
- medical device
- emplacement
- medical
- axis
- plane
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
- A61B17/068—Surgical staplers, e.g. containing multiple staples or clamps
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
- A61B17/068—Surgical staplers, e.g. containing multiple staples or clamps
- A61B17/072—Surgical staplers, e.g. containing multiple staples or clamps for applying a row of staples in a single action, e.g. the staples being applied simultaneously
- A61B17/07207—Surgical staplers, e.g. containing multiple staples or clamps for applying a row of staples in a single action, e.g. the staples being applied simultaneously the staples being applied sequentially
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
- A61B17/32—Surgical cutting instruments
- A61B17/320016—Endoscopic cutting instruments, e.g. arthroscopes, resectoscopes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/10—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges for stereotaxic surgery, e.g. frame-based stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/37—Surgical systems with images on a monitor during operation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
- A61B2017/00017—Electrical control of surgical instruments
- A61B2017/00022—Sensing or detecting at the treatment site
- A61B2017/00106—Sensing or detecting at the treatment site ultrasonic
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
- A61B17/068—Surgical staplers, e.g. containing multiple staples or clamps
- A61B17/072—Surgical staplers, e.g. containing multiple staples or clamps for applying a row of staples in a single action, e.g. the staples being applied simultaneously
- A61B2017/07214—Stapler heads
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
- A61B17/068—Surgical staplers, e.g. containing multiple staples or clamps
- A61B17/072—Surgical staplers, e.g. containing multiple staples or clamps for applying a row of staples in a single action, e.g. the staples being applied simultaneously
- A61B2017/07214—Stapler heads
- A61B2017/07271—Stapler heads characterised by its cartridge
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
- A61B17/32—Surgical cutting instruments
- A61B2017/320052—Guides for cutting instruments
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/107—Visualisation of planned trajectories or target regions
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2048—Tracking techniques using an accelerometer or inertia sensor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2051—Electromagnetic tracking systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2055—Optical tracking systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
- A61B2034/302—Surgical robots specifically adapted for manipulations within body cavities, e.g. within abdominal or thoracic cavities
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/361—Image-producing devices, e.g. surgical cameras
- A61B2090/3612—Image-producing devices, e.g. surgical cameras with images taken automatically
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B2090/364—Correlation of different images or relation of image positions in respect to the body
- A61B2090/365—Correlation of different images or relation of image positions in respect to the body augmented reality, i.e. correlating a live optical image with another image
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B2090/364—Correlation of different images or relation of image positions in respect to the body
- A61B2090/367—Correlation of different images or relation of image positions in respect to the body creating a 3D dataset from 2D images using position information
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/37—Surgical systems with images on a monitor during operation
- A61B2090/372—Details of monitor hardware
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/37—Surgical systems with images on a monitor during operation
- A61B2090/378—Surgical systems with images on a monitor during operation using ultrasound
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/37—Surgical systems with images on a monitor during operation
- A61B2090/378—Surgical systems with images on a monitor during operation using ultrasound
- A61B2090/3782—Surgical systems with images on a monitor during operation using ultrasound transmitter or receiver in catheter or minimal invasive instrument
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
- A61B2090/3937—Visible markers
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/50—Supports for surgical instruments, e.g. articulated arms
- A61B2090/502—Headgear, e.g. helmet, spectacles
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/25—User interfaces for surgical systems
Definitions
- Various systems are available to aid a healthcare provider to guide a medical device in a patient or to provide a user viewing an object with additional information.
- the systems can provide image guidance cues to aid the healthcare provider or user and can also provide additional information for the user's benefit.
- FIG. 1A is a diagram illustrating an embodiment of an environment for image-guided medical procedures.
- FIG. 1B is a diagram illustrating embodiments of coordinate systems that can be used by the system.
- FIG. 2 is a diagram illustrating an embodiment of a rendering of image guidance cues and medical display objects on a display.
- FIG. 3 is a diagram illustrating an embodiment of a rendering of image guidance cues and medical display objects, including a ghost medical device.
- FIG. 4 is a diagram illustrating an embodiment of a rendering of image guidance cues and medical display objects, including a ghost medical image.
- FIG. 5 is a diagram illustrating an embodiment of an affected region associated with a medical device.
- FIG. 6 is a diagram illustrating an embodiment of a medical device having visual tracking fiducials.
- FIG. 7 is a diagram illustrating an embodiment of a rendering of a first and second virtual medical device and a medical image.
- FIG. 8 is a diagram illustrating an embodiment of a rendering first and second virtual medical device.
- FIG. 9 is a diagram illustrating an embodiment of a medical device having an integrated imaging device.
- FIG. 10 is a diagram illustrating an embodiment of a medical device having an integrated imaging device.
- FIG. 11 is a diagram illustrating an embodiment of a medical device having an integrated imaging device.
- FIG. 12 is a diagram illustrating an embodiment of a medical device having an integrated imaging device.
- FIG. 13 is a flow diagram illustrative of an embodiment of a routine implemented by the system to determine and cause a display to display one or more intersection indicators corresponding to an intersection between a plane-of-interest and one or more trajectories of a medical device.
- FIG. 14 is a flow diagram illustrative of an embodiment of a routine implemented by the system to project a display object onto a plane-of-interest to display a ghost display object.
- FIG. 15 is a flow diagram illustrative of an embodiment of a routine implemented by the system to display an affected region of a medical device.
- FIG. 16 is a flow diagram illustrative of an embodiment of a routine implemented by the system to determine emplacement of a virtual medical device and a medical image based at least in part on received emplacement data associated with a medical device.
- Implementations disclosed herein provide systems, methods, and apparatus for displaying medical images, such as, but not limited to ultrasound, CT, and/or MRI images, facilitating medical device insertion into tissue by an operator.
- Certain embodiments pertain to a free-hand medical device guidance system.
- the system can provide the healthcare provider manual control over the medical device, while making the spatial relationships between the target, medical device and medical image (also referred to as an image slice or rendered medical image), or image area corresponding to the medical image (also referred to as an image slice area or scan area), more intuitive via a visual display. Using this visual feedback, the operator can adjust the medical device's position, orientation, or trajectory.
- Certain of the contemplated embodiments can be used in conjunction with systems described in greater detail in U.S. patent application Ser. No.
- Medical interventions typically involve using an instrument to resect, cauterize, staple, seal, or otherwise manipulate soft tissue and organs.
- a physician must take great care to minimize blood loss and minimize damage to ancillary tissue while performing these tissue-damaging interventions. This is even more difficult with minimally-invasive surgeries, such as laparoscopic, endoscopic, and robotic surgeries.
- a physician may use ultrasound to image the internal structures of an organ before stapling, transecting, resecting, sealing, or grasping tissue, helping her avoid critical structures such as blood vessels.
- ultrasound imaging there is a significant possibility of inadvertent damage to surrounding tissue and blood vessels during these procedures. This is because it is not obvious in the externally displayed medical image where a given internal structure is located relative to the medical device.
- the system can aid the healthcare provider in guiding one or more medical devices through or around tissue of the patient and/or placing the medical devices.
- the system can be used to aid in stapling, transecting, resecting, sealing, and/or grasping tissue.
- the system can be used for treatment of tumors, fibroids, cysts, damaged blood vessels, or other damages internal structures of a patient.
- the system can be used during open surgery, laparoscopic surgery, endoscopic procedures, robotic surgeries, biopsies, and/or interventional radiology procedures.
- the system can be used in conjunction with live intraoperative ultrasound (U/S), pre-operative CT, or any cross-sectional medical imaging modality (e.g. MRI, OCT, etc.).
- the system can use a variety of techniques to determine the position and/or orientation of one or more medical devices.
- the system can use the NDI Aurora magnetic system, NDI Polaris optical system, etc.
- a position sensor can be embedded inside, or affixed to each medical device, at the tip, along the shaft, and/or on the handle. Sensors can be built into the medical devices or attached after manufacturing, as described in greater detail in U.S. application Ser. No. 14/212,184, filed Mar. 14, 2014, entitled SENSOR MOUNT, incorporated herein in its entirety.
- Each medical device can be associated with one or more sensors, which can continually, or repeatedly, report position and/or orientation, or a single sensor can be used for all the medical devices.
- the healthcare provider can attach the sensor to the particular medical device that she is intentionally repositioning, and then, once she has placed that medical device, she can remove the sensor and attach it to the next medical device she is repositioning.
- the medical devices can be manipulated by the healthcare provider.
- the system can be used with a robotic manipulator, where the robot controls the medical devices.
- visually-detectable fiducials can be used to determine position and/or orientation for one or more of the medical devices.
- the handles of medical devices can have push-button switches, to allow the user to select a medical device, indicate a tissue target, etc.
- the handle can also have an indicator light to indicate to the users which medical device is selected.
- the handle can have an encoder to detect how much length of electrode has been exposed by the user, and report this information to the guidance system and therapeutic generator.
- FIG. 1A is a diagram illustrating an embodiment of a system for image management in image-guided medical procedures.
- the position sensing unit 140 can track medical devices within a tracking area and provide data to the image guidance unit 130 .
- the medical devices can include invasive medical devices, such as, but not limited to, graspers, staplers, vessel sealers, electrocautery devices, resecting devices, transecting devices, scalpels, biopsy needles, ablation needles, surgical needles, nerve-block needles, or other needles, catheters, stents, laparoscopes or laparoscopic cameras, ultrasound transducers, or other instruments that enter a part of the body, and non-invasive medical devices that do not enter the body, such as, but not limited to, ultrasound transducers, probes, or other external imaging devices, etc.
- invasive medical devices such as, but not limited to, graspers, staplers, vessel sealers, electrocautery devices, resecting devices, transecting devices, scalpels, biopsy needles,
- the medical devices can also include medical imaging devices that provide or aid in the selection of medical images for display.
- the medical imaging device can be any device that is used to select a particular medical image for display.
- the medical imaging devices can include invasive medical devices, such as laparoscopic cameras, and non-invasive medical devices, such as external ultrasound transducers.
- the image guidance unit 130 can process or combine the data and show image guidance data on display 120 .
- This image guidance data can be used by a healthcare provider to guide a procedure and improve care.
- system 100 There are numerous other possible embodiments of system 100 . For example, many of the depicted components can be joined together to form a single component and can be implemented in a single computer or machine.
- additional position sensing units can be used in conjunction with position sensing unit 140 to track all relevant medical devices 145 and 155 , as discussed in more detail below.
- Additional imaging units 150 can be included, and combined imaging data from the multiple imaging units 150 can be processed by image guidance unit 130 and shown on display unit 120 . Additionally, two or more surgical systems 149 can also be included.
- Image guidance unit 130 can be processed by image guidance unit 130 and shown on display 120 .
- image guidance unit 130 can be processed by image guidance unit 130 and shown on display 120 .
- image guidance cues e.g., trajectory indicators, intersection indicators, plane indicators, ghost display objects, etc.
- any combination of the display objects can be displayed concurrently, or simultaneously.
- reference to displaying objects “concurrently” and/or “simultaneously” is to be interpreted broadly and may refer to displaying objects in such a way that to a human observer the objects are visible at the same time.
- Imaging unit 150 can be coupled to image guidance unit 130 .
- imaging unit 150 can be coupled to a second display unit (not shown).
- the second display unit can display imaging data from imaging unit 150 .
- the imaging data displayed on display unit 120 and displayed on second display unit can be the same or different.
- the imaging unit 150 is an ultrasound machine
- the movable imaging device 155 is an ultrasound transducer or ultrasound probe
- the second display unit is a display associated with the ultrasound machine 150 that displays the medical images from the ultrasound machine 150 .
- a movable imaging unit 155 can be connected to image guidance unit 130 .
- the movable imaging unit 155 can be useful for allowing a user to indicate what portions of a first set of imaging data are to be displayed.
- the movable imaging unit 155 can be an ultrasound transducer 155 , a stapler or other medical device, for example, and can be used by a user to indicate what portions of imaging data, such as a pre-operative CT scan, to show on a display unit 120 as image 125 . Further, in some embodiments, there can be a third set of pre-operative imaging data that can be displayed with the first set of imaging data.
- system 100 comprises a display unit 120 and a position sensing unit 140 communicatively coupled to image guidance unit 130 .
- position sensing unit 140 , display unit 120 , and image guidance unit 130 are coupled to the stand 170 .
- Image guidance unit 130 can be used to produce images 125 that are displayed on display unit 120 .
- the images 125 produced on display unit 120 by the image guidance unit 130 can be determined based on ultrasound or other visual images from the first medical device 145 and second medical device 155 .
- the images 125 include a 2D viewing area and a 3D viewing area. In the 2D viewing area, some or all of the display objects can be displayed as 2D objects.
- the 2D viewing area can include a 2D view of each of an ultrasound slice, a virtual medical device corresponding to the first medical device, a virtual imaging device corresponding to the second medical device, intersection indicator, trajectory indicators and/or other image guidance cues.
- some or all of the display objects in the 2D viewing area can be displayed as 3D objects.
- the 3D viewing area includes perspective views of each of the medical image 121 , the virtual medical device 122 , a displayed affected region 129 , the virtual imaging device 123 , intersection indicator 126 , trajectory and other image guidance cues 127 A, 127 B (generically referred as image guidance cues 127 and/or trajectory indicators 127 ), and a patient orientation indicator 128 . It will be understood that any combination of the aforementioned display objects can be displayed in the 2D view and/or 3D view as desired.
- images 125 produced on display 120 can include the images, or video, from the ultrasound probe 155 (e.g., medical image 121 ) combined with other medical display objects and image guidance cues, such as display object trajectories (e.g., trajectory indicators 127 ) or affected region 129 (e.g., trajectory of area corresponding to a volume located between the first limb member and the second limb member), determined based on the emplacement of the stapler 145 .
- display object trajectories e.g., trajectory indicators 127
- affected region 129 e.g., trajectory of area corresponding to a volume located between the first limb member and the second limb member
- images 125 produced on display 120 can include the video from the laparoscopic camera 155 combined with ultrasound data superimposed on the laparoscopic image.
- More medical devices can be added to the system.
- the system can include a stapler, ultrasound probe, ablation needle, laparoscopic camera, scalpel, grasper, vessel sealer, electrocauterizer, resecting device, transecting device and/or any other surgical instrument or medical device.
- the system can also process and/or display collected data, such as preoperative CT scans, X-Rays, MRIs, laser scanned 3D surfaces etc.
- the term “emplacement” as used herein is a broad term and may refer to, without limitation, position and/or orientation or any other appropriate location information.
- the term “pose” as used herein is a broad term encompassing its plain and ordinary meaning and may refer to, without limitation, position and orientation or any other appropriate location information.
- the imaging data obtained from one or both of medical devices 145 and 155 can include other modalities such as a CT scan, MRI, open-magnet MRI, optical coherence tomography (“OCT”), positron emission tomography (“PET”) scans, fluoroscopy, ultrasound, or other preoperative, or intraoperative 2D or 3D anatomical imaging data.
- medical devices 145 and 155 can also be scalpels, implantable hardware, or any other device used in surgery. Any appropriate surgical system 149 or imaging unit 150 can be attached to the corresponding medical instruments 145 and 155 .
- images 125 produced can also be generated based on live, intraoperative, or real-time data obtained using the second medical device 155 , which is coupled to second imaging unit 150 .
- the term “real time” as used herein is a broad term and has its ordinary and customary meaning, including without limitation instantaneously or nearly instantaneously. The use of the term real time can also mean that actions are performed or data is obtained with the intention to be used immediately, upon the next cycle of a system or control loop, or any other appropriate meaning.
- real-time data can be data that is obtained at a frequency that would allow a healthcare provider to meaningfully interact with the data during surgery.
- real-time data can be a medical image of a patient that is updated one time per second.
- real-time data can be ultrasound data that is updated multiple times per second.
- the medical devices 145 , 155 can be communicatively coupled to the position sensing unit 140 (e.g., sensors embedded or coupled to the medical devices 145 , 155 can be communicatively coupled with the position sensing unit 140 ).
- the position sensing unit 140 can be part of imaging unit 150 or it can be separate.
- the position sensing unit 140 can be used to determine the emplacement of first medical device 145 and/or the second medical device 155 .
- the position sensing unit 140 can include a magnetic tracker and/or one or more magnetic coils can be coupled to medical devices 145 and/or 155 .
- the position sensing unit 140 can include an optical tracker and/or one or more visually-detectable fiducials can be coupled to medical devices 145 and/or 155 .
- the position sensing unit 140 can be located below the patient.
- the position sensing unit 140 can be located on or below the table 180 .
- the position sensing unit 140 is a magnetic tracker, it can be mounted below the surgical table 180 .
- magnetic tracking coils can be mounted in or on the medical devices 145 and 155 .
- the position sensing unit 140 can be an electromagnetic measurement system (e.g., NDI Aurora system) using sensor coils for tracking units attached to the first and/or second surgical devices 145 and 155 .
- the second position sensing unit 140 can be an optical 3D tracking system using fiducials.
- optical 3D tracking systems can include the NDI Polaris Spectra, Vicra, Certus, PhaseSpace IMPULSE, Vicon MX, InterSense IS-900, NaturalPoint OptiTrack, Polhemus FastTrak, IsoTrak, or Claron MicronTracker2.
- the position sensing unit 140 can each be an inertial 3D tracking system comprising a compass, accelerometer, tilt sensor, and/or gyro, such as the InterSense InertiaCube or the Nintendo Wii controller. In some embodiments, the position sensing unit 140 can be attached to or affixed on the corresponding surgical device 145 and 155 .
- the position sensing units 140 can include sensing devices such as the HiBall tracking system, a GPS device, or signal emitting device that would allow for tracking of the position and/or orientation (e.g., emplacement) of the tracking unit (also referred to as an emplacement sensor).
- a position sensing unit 140 can be affixed to either or both of the surgical devices 145 and 155 .
- the surgical devices 145 or 155 can be tracked by the position sensing unit 140 .
- a room coordinate system reference, such as the display 120 can also be tracked by the position sensing unit 140 in order to determine the emplacements of the surgical devices 145 and 155 with respect to the room coordinate system.
- Devices 145 and 155 can also include or have coupled thereto one or more accelerometers, which can be used to estimate movement, position, and location of the devices.
- the position sensing unit 140 can be an Ascension Flock of Birds, Nest of Birds, driveBAY, medSAFE, trakSTAR, miniBIRD, MotionSTAR, pciBIRD, or Calypso 2D Localization System and tracking units attached to the first and/or second medical devices 145 and 155 can be magnetic tracking coils.
- tracking unit (also referred to as an emplacement sensor), as used herein, is a broad term encompassing its plain and ordinary meaning and includes without limitation all types of magnetic coils or other magnetic field sensing devices for use with magnetic trackers, fiducials or other optically detectable markers for use with optical trackers, such as those discussed above and below.
- the tracking units can be implemented using optical position sensing devices, such as the HiBall tracking system and the position sensing unit 140 can form part of the HiBall tracking system.
- Tracking units can also include a GPS device or signal emitting device that allows for tracking of the position and/or orientation of the tracking unit.
- a signal emitting device might include a radio-frequency identifier (RFID).
- RFID radio-frequency identifier
- the position sensing unit 140 can use the GPS coordinates of the tracking units or can, for example, triangulate the radio frequency signal being emitted by the RFID associated with tracking units.
- the tracking systems can also include one or more 3D mice.
- Images 125 can be produced based on intraoperative or real-time data obtained using first medical device 145 , which is coupled to first surgical system 149 .
- the first surgical system 149 is shown as coupled to image guidance unit 130 .
- the coupling between the first surgical system 149 and image guidance unit 130 may not be present in all embodiments.
- the coupling between first surgical system 149 and image guidance unit 130 can be included where information about first medical device 145 available to first surgical system 149 is useful for the processing performed by image guidance unit 130 .
- the first medical device 145 (for example, a stapler) includes a joint member and two limb members coupled to the joint member at different angles and the first surgical system 149 is a stapler navigation system 149 .
- the first surgical system 149 is not coupled to image guidance unit 130 .
- Example embodiments including images and graphics that can be displayed are included below.
- the display unit 120 displays 3D images to a user, such as a healthcare provider.
- Stereoscopic 3D displays separate the imagery shown to each of the user's eyes. This can be accomplished by a stereoscopic display, a lenticular auto-stereoscopic display, or any other appropriate type of display.
- the display 120 can be an alternating row or alternating column display.
- Example alternating row displays include the Miracube G240S, as well as Zalman Trimon Monitors.
- Alternating column displays include devices manufactured by Sharp, as well as many “auto-stereoscopic” displays (e.g., Philips).
- Sony Panasonic 3D passive displays and LG, Samsung, and/or Vizio 3D TVs can be used as well.
- Display 120 can also be a cathode ray tube.
- Cathode Ray Tube (CRT) based devices can use temporal sequencing, showing imagery for the left and right eye in temporal sequential alternation. This method can also be used for projection-based devices, as well as by liquid crystal display (LCD) devices, light emitting diode (LED) devices, and/or organic LED (OLED) devices.
- LCD liquid crystal display
- LED light emitting diode
- OLED organic LED
- the display unit 120 can be a head mounted display worn by the user in order to receive 3D images from the image guidance unit 130 .
- a separate display such as the pictured display unit 120 , can be omitted.
- the 3D graphics can be produced using underlying data models, stored in the image guidance unit 130 and projected onto one or more 2D planes in order to create left and right eye images for a head mount, lenticular, or other 3D display.
- the underlying 3D model can be updated based on the relative emplacements of the various devices 145 and 155 , as determined by the position sensing unit(s) 140 , and/or based on new data associated with the devices 145 and 155 .
- the underlying data model can be updated to reflect the most recent medical image.
- the first medical device 145 is a stapler, then the underlying model can be updated to reflect any changes related to the jaws, such as information regarding the affected region or angles of the jaws and/or transecting knife.
- Any appropriate 3D graphics processing can be used for rendering including processing based on OpenGL, Direct3D, Java 3D, etc. Whole, partial, or modified 3D graphics packages can also be used, such packages including 3DS Max, SolidWorks, Maya, Form Z, Cybermotion 3D, VTK, Slicer, or any others.
- various parts of the needed rendering can occur on traditional or specialized graphics hardware.
- the rendering can also occur on the general CPU, on programmable hardware, on a separate processor, be distributed over multiple processors, over multiple dedicated graphics cards, or using any other appropriate combination of hardware or technique.
- kits can be packaged and/or distributed as part of a kit.
- a stapler for example, a stapler, one or more tracking units, 3D viewing glasses, and/or a portion of an ultrasound wand can form a kit.
- Other embodiments can have different elements or combinations of elements grouped and/or packaged together. Kits can be sold or distributed separately from or with the other portions of the system.
- FIG. 1B is a diagram illustrating embodiments of coordinate systems that can be used by the system 100 .
- the system 100 can utilize the coordinate systems to track and display the various display objects on the displays, including, but not limited to one or more position sensing coordinate systems 180 A, 180 B (generically referred to as 180 ), one or more room coordinate systems 182 , one or more 3D scene coordinate systems 184 , one or more display coordinate systems 186 , one or more sensor coordinate systems 188 A, 188 B, one or more medical device coordinate systems 190 A, 190 B, 190 C, 190 D, one or more virtual medical device coordinate systems 192 A, 192 B, 192 C, 192 D, one or more medical image coordinate systems 194 , etc.
- one or more position sensing coordinate systems 180 A, 180 B (generically referred to as 180 ), one or more room coordinate systems 182 , one or more 3D scene coordinate systems 184 , one or more display coordinate systems 186 , one or more sensor coordinate systems 188 A, 188 B
- a position sensing coordinate system 180 can be used to determine the emplacement of various objects within a position sensing region.
- the position sensing coordinate system 180 can refer to the coordinate system 180 A used by a magnetic tracker (not shown) tracking objects within a magnetic field volume, or to the coordinate system 180 B used by an optical tracker 196 tracking objects within a volume.
- multiple position sensing coordinate systems 180 can be used together.
- a magnetic position sensing coordinate system 180 A can be used in conjunction with a magnetic tracker tracking sensor coils within a position sensing region that are coupled to medical devices and an optical position sensing coordinate system 180 B can be used in conjunction with an optical tracker tracking a fiducial coupled to a head mounted display (HMD) or a user, or to an optical tracker analyzing an image captured by an image sensor.
- HMD head mounted display
- any combination of the tracker systems and position sensing coordinate systems 180 can be used as desired.
- a single position sensing coordinate system 180 can be used to track tracking sensors associated with one or more medical devices and/or tracking sensors associated with a user or HMD.
- a different position sensing coordinate system 180 can be used for each tracking sensor, or any combination thereof.
- Room coordinate systems 182 can be used to determine the emplacement of objects within a room, such as an operating room.
- the room coordinate system 182 can be used to determine or identify the relative emplacement of the position sensing unit, medical devices, tracking sensors, user, display, etc. relative to each other within a room.
- a 3D scene coordinate system 184 which may also be referred to as a 3D volume or scene graph coordinate system, can be used to determine the emplacement of display objects within a virtual 3D scene.
- the 3D scene coordinate system 184 can identify the relative emplacement of virtual objects within the 3D scene.
- the virtual objects can correspond to real objects, such as to medical devices 145 , 155 and/or to computer-generated objects, such as such as trajectory cues 127 , affected region 129 , etc.
- display objects can correspond to real objects, virtual objects, and/or computer generated objects.
- a display coordinate system 186 can be used to determine the emplacement of display objects for display on the display 102 .
- the display coordinate system 186 can be used to determine the emplacement of virtual medical devices, medical images, image guidance cues, and the like, within a display 102 .
- the display coordinate system 186 can be used to determine how the objects within the 3D scene are to be displayed on the display.
- the display coordinate system 186 can be used to determine the point-of-view location, or eye point, relative to the 3D scene (or 3D volume coordinate system 184 ) or scene graph for viewing the contents of the 3D scene.
- multiple display coordinate systems 186 can be used.
- left-eye, right-eye, and/or center-eye display coordinate systems 186 can be used to display different perspective of the display objects within a 3D scene, such as when a 3D display and/or an HMD is being used.
- a medical image coordinate system 194 can be used in conjunction with medical images used and/or processed by the system.
- the medical images can be ultrasound images, CT image, MRI, images, etc.
- the images can be different sizes or shapes.
- one ultrasound can output an image having one size and shape while a different ultrasound can output an image having a different size and/or shape.
- CT, MRI, and ultrasound images may have different sizes and shapes.
- the medical image coordinate system can be used to identify the particular size and shape of the medical image being used and/or processed by the system 100 .
- the 3D scene coordinate system 184 can be omitted or combined with display coordinate system 186 and/or the position sensing coordinate system 180 .
- one or more tracking sensor coordinate systems 188 A, 188 B, medical device coordinate systems 190 A, 190 B, 190 C, 190 D, virtual medical device coordinate systems 192 A, 192 B, 192 C, 192 D, or other objects etc. can have their own coordinate system.
- the coordinate systems for the tracking sensors, medical devices, and/or virtual medical devices can be used to identify the dimensions of the sensor/device/display object and relationship of the sensor/device/display object to another sensor/device/display object or other coordinate systems.
- a medical device coordinate system (or virtual medical device coordinate system) can identify the dimensions of a corresponding medical device or virtual medical device, as well as the emplacement of a tracking sensor relative to the medical device (or vice versa).
- a medical imaging device coordinate system can identify the dimensions of the corresponding medical imaging device (or virtual medical imaging device) and/or an emplacement of a medical image relative to the medical imaging device (non-limiting example: the emplacement of an ultrasound image relative to the corresponding ultrasound transducer), or vice versa.
- the system 100 can use various coordinate systems to determine the emplacement of a portion or the entire object with respect to each other and with respect to the other coordinate systems.
- the system 100 can use the various coordinate systems to determine emplacement of objects relative to each other and determine how to display the display objects on a display, such as the display 102 .
- a display such as the display 102
- the system 100 can determine the emplacement of a magnetic tracking sensor coupled to the ultrasound transducer 155 within a magnetic position sensing coordinate system 180 .
- the system 100 can determine the location of each portion of the magnetic tracking sensor within the magnetic position sensing coordinate system 180 .
- the system 100 can also determine the emplacement of each portion the ultrasound transducer 155 within the magnetic position sensing coordinate system by mapping the ultrasound transducer coordinate system 190 B to the magnetic tracking sensor coordinate system 188 B (or vice versa) and/or to the magnetic position sensing coordinate system 180 .
- system 100 can map each portion of the ultrasound image 121 corresponding to the ultrasound transducer 155 to the magnetic position sensing coordinate system 180 by mapping an ultrasound image coordinate system 194 to the ultrasound transducer coordinate system 190 B, the magnetic tracking sensor coordinate system 188 B, and/or to the magnetic position sensing coordinate system 180 .
- the system 100 can map the various objects from the magnetic position sensing coordinate system 180 to a room coordinate system 182 , which can identify the relative emplacement of the position sensing coordinate system 180 to a display.
- the system can then map data to the 3D scene coordinate system 194 and/or the display coordinate system 186 .
- the system 100 can map the objects to multiple display coordinate systems 186 , such as left-eye and/or right-eye coordinate systems.
- the system 100 can determine an emplacement of an optical tracking sensor corresponding to a user within an optical position sensing coordinate system 180 B.
- the emplacement of the optical tracking sensor within the optical position sensing coordinate system 180 B can be mapped to the room coordinate system 182 , the 3D scene coordinate system 184 , and/or the display coordinate systems 186 for display.
- the system 100 can determine the emplacement of the ultrasound transducer 155 and ultrasound image slice 121 relative to the user and display a virtual rendering of the ultrasound transducer 123 and ultrasound image slice 121 within the 3D scene relative to the determined emplacement of the user.
- the system 100 can map the various objects from the magnetic position sensing coordinate system 180 to a room coordinate system 182 , which can identify the relative emplacement of the position sensing coordinate system 180 to a display.
- the system can then map data to the 3D scene coordinate system 194 and/or the display coordinate system 186 .
- the system 100 can map the objects to multiple display coordinate systems 186 , such as left-eye and/or right-eye coordinate systems.
- any of the real or virtual objects described herein may be represented, detected, or imaged in any coordinate system, and conversion between the various coordinate systems can be performed in components of the system such as image guidance unit 130 , position sensing unit 140 , imager 150 , the an HMD, or other components.
- references made herein to determining an emplacement of the medical device can also refer to determining an emplacement of a virtual medical device corresponding to the medical device, or vice versa.
- references made herein to determining an emplacement of a display object (non-limiting example: medical image) relative to the medical device can also refer to determining the emplacement of the display object relative to a corresponding virtual medical device.
- various embodiments herein provide image guidance that can help the healthcare provider better understand the scene, relative emplacements or poses of object in the scene and thereby provide improved image guidance.
- FIG. 2 illustrates a perspective view of a virtual rendering 202 of a medical device 242 being displayed on a screen 220 with a perspective view of a medical image 204 .
- the screen 220 can correspond to the screen of a display unit 120 , which can be implemented using a TV, computer screen, head-mounted display, projector, etc.
- the virtual medical device 202 displayed on the screen 220 corresponds to the stapler 242 .
- a wire 246 connecting the stapler 242 to a stapler navigation system is also depicted in FIG. 2 .
- the virtual medical device 202 (which can also be referred to as an avatar) can be displayed in a virtual 3D space with the screen 220 acting as a window into the virtual 3D space.
- the medical device 242 is moved to the right with respect to a point-of-view location (e.g., the location of the point-of-view for viewing the 3D space)
- the virtual medical device 202 also moves to the right.
- the virtual medical device 202 will likewise show the change in orientation, and show the tip of the virtual medical device 202 in the background and the other end of the virtual medical device 202 in the foreground.
- the medical device 242 is rotated 90 degrees so that the tip of the medical device is pointing away from the point-of-view location (e.g., at the screen 220 )
- the virtual medical device 202 will likewise show the change in orientation, and show the tip of the virtual medical device 202 in the background and the other end of the virtual medical device 202 in the foreground.
- the point-of-view location can be a fixed location, such as a predetermined distance/angle from the screen 220 or stand 170 and/or a location configured by the user; or the point-of-view location can by dynamic.
- the system can track a user in real-time and determine the point-of-view location based at least in part on the tracked location of the user.
- Some models of medical devices have markings such as bands around the shaft or jaws to indicate distance along the shaft or jaws. Healthcare providers performing medical device procedures are often familiar with these markings and can use them to help understand the spatial relationship between the medical device and anatomy.
- the make and model of the medical device 242 is known to the image guidance system and the virtual medical device 202 displayed in display 220 can resemble medical device 242 .
- the features of medical devices that can be rendered in the scene include the overall shape (diameter, angles, cross sectional shape, curvature, etc.), color, distance markers, angle of the jaws, visuals or echogenic fiduciary markers, the state of deployable elements such as tines, paddles, anchors, resection loops, stiffening or steerable sleeves, temperature, radiation, light or magnetic field sensors, lens, waveguides, fluid transfer channels, and the like.
- the type of medical device being used can be input into the image guidance system 100 , can be a system default, can be detected by a camera or other device, can be received as data from an attached medical device, such as surgical system 149 in FIG. 1 , or the information can be received in any other appropriate manner.
- Displaying on display 220 a virtual medical device 202 that resembles the medical device 242 can help healthcare providers associate the image guidance data with the real world and can provide more familiar guidance information to a healthcare provider, thereby further aiding the healthcare provider in the guidance task.
- the healthcare provider can see the familiar markings on the medical device 202 being displayed on the display 220 and therefore be familiar with the distance and relative placement of the displayed medical device 202 with respect to other data, such as tissue 212 seen in a medical image 204 , 205 .
- This knowledge of relative placement of items being displayed can help the healthcare provider move the medical device 242 into place.
- the terms medical device and virtual medical device can sometimes be used interchangeably as they can generally relate to the same object, the medical device being the object in the real world and virtual medical device being a representation of the object, such as an avatar, in virtual space. Additionally, in some instances virtual medical device can sometimes be referred to as virtual surgical instrument, surgical instrument, rendered surgical instrument, rendered medical device, avatar, rendered avatar, and virtual avatar.
- the virtual medical device 202 in the display 220 is a virtual stapler depicting the portion of the stapler that will perform the stapling.
- the displayed virtual medical device can include a joint member 211 , a first limb member 213 , and a second limb member 210 .
- the doctor can find the tissue she wishes to staple by moving the ultrasound probe 222 until she identifies the target tissue.
- she will be able to see the displayed ultrasound data and its location relative to the displayed medical device 202 .
- She can then direct the medical device 242 until she sees, on display 220 , that the affected region of the virtual medical device 202 encompasses a desired stapled region of the tissue in the medical image.
- she activates the stapler 242 she can have a higher degree of confidence that she has stapled the correct portion of the tissue. Various embodiments of this are discussed below.
- the image guidance unit can represent these markings in the images displayed in the display.
- certain ultrasound transducers are built with an orientation mark (e.g., a small bump) on one side of the transducing array. That mark can also be shown in the medical image on the scanner's display, to help the healthcare provider understand where the scanned anatomical structures shown on screen are located under the transducer, inside the patient.
- the image guidance system can display a symbolic 3D representation of an orientation mark 214 both next to the motion-tracked ultrasound slice (e.g., moving with the displayed ultrasound slice) and next to the 2D view of the ultrasound slice also displayed by the system.
- An example of this orientation mark is displayed in FIG. 2 , where a small rectilinear volume 214 is shown both in proximity to the ultrasound slice displayed in the 3D view and the ultrasound slice displayed in a 2D view.
- the orientation mark 214 corresponds to a feature, such as a physical marking, of the ultrasound probe.
- the orientation mark 214 is displayed to provide assistance in associating the 3D view and the 2D view.
- a medical image can correspond to image data received from an imaging device, such as an ultrasound transducer.
- the image data can correspond to a cross-section of tissue having a certain thickness.
- the imaging device can compact the image data, and/or treat the image data as 2D data, such that there is no perceived thickness.
- the system when the medical image is displayed in a 3D view, the system can treat the medical image as a 2D or quasi 2D object. In such embodiments, the system can cause the medical image to have little to no perceptible thickness.
- the system when the medical image is oriented orthogonally or perpendicularly with respect to the point-of-view location, the system can cause the display to display nothing or a line having a relatively small thickness, such as a few pixels, etc.
- the number of pixels used to display the relatively small thickness of the medical image can correspond to the size of the display. For example, more pixels can be used for a larger display and fewer pixels can be used for a smaller display, etc.
- a healthcare provider may want to track one or more of a stapler, vessel sealer, grasper, scalpel, a biopsy, a cauterizer (including an electrocauterizer and Bovies), forceps, cutting loops on hysteroscopes, harmonic sheers, lasers (including CO 2 lasers), etc.
- a stapler including an electrocauterizer and Bovies
- a cauterizer including an electrocauterizer and Bovies
- forceps including an electrocauterizer and Bovies
- cutting loops on hysteroscopes including harmonic sheers
- lasers including CO 2 lasers
- the following devices can be tracked and various aspects of their design displayed on display 220 : OlympusTM OES Pro Hystero-Resectoscope, SonoSurg Ultrasonic Surgical System OlympusTM GF-UC 160 Endoscope WallusTM Embryo Transfer Catheter AngioDynamics® NanoKnifeTM, VenaCureTM laser, StarBurst, Uniblade, Habib® Resector BovieTM Electrodes, Covidien EvidentTM, Cool-TipTM Ablation Antennas, Opti4TM Electrodes Microsulis MEA (microwave endometrial ablation), Acculis HaltTM Medical System Optimed BigLumen Aspiration Catheter Optimed Optipure Stent Central venous catheterization introducer medical device (such as those made by Bard and Arrow).
- OlympusTM OES Pro Hystero-Resectoscope SonoSurg Ultrasonic Surgical System
- a healthcare provider is able to see image guidance data on display 220 that will allow her to know the relative pose, location, or emplacement of the tracked instrument(s) with respect to one another or with respect to imaging data and will be able to see, on display 220 , the features of the instrument rendered in the scene.
- the image guidance system can constantly display an additional 2D view 205 of the medical image, simultaneous to the 3D depiction 204 , so that the medical image is always visible, regardless of the emplacement in which the healthcare provider holds the transducer 222 .
- the 2D view 205 of the medical data can be similar to what a healthcare provider is accustomed to seeing with traditional medical displays, such as ultrasound displays. This can be useful to provide the healthcare provider with imaging to which she is accustomed and allows a healthcare provider to see the medical data regardless of the then-current emplacement of the imaging device with respect to the user.
- the 2D view 205 of the medical image is depicted in the upper right corner of the monitor (though it can be placed in any location).
- the guidance system can automatically (and continually) choose a corner in which to render the 2D view 205 of the medical image, based on the 3D position of the medical devices in the rendered scene. For example, in FIG. 2 , stapler 242 can be held in the healthcare provider's left hand and the medical device shaft is to the left of the 3D view of the medical image, so that the 2D view 202 of the medical image in the upper right corner of display 220 does not cover any of the 3D features of the medical device (or vice-versa).
- the system can automatically move it to a corner that would not otherwise be occupied by graphics or data.
- the system attempts to avoid having the 2D view 205 of the medical image quickly moving among corners of the display in order to avoid overlapping with graphics and data in the display.
- a function f can be used to determine which corner is most suitable for the 2D medical image to be drawn in.
- the inputs to f can include the locations, in the screen coordinate system, of the displayed medical device tip, the corners of the 3D view of the medical image, etc.
- f s output for any given point in time is independent of f s output in the previous frames, which can cause the medical image to move among corners of the display rapidly.
- the image guidance system will filter f's output over time.
- the output of a filter g could be the corner, which has been output by f the most number of times over the last n frames, possibly weighting the most recent values for f most heavily.
- the output of the filter g can be used to determine in which corner of display 220 to display the 2D medical image and the temporal filtering provided by g can allow the 2D view 205 of the medical image display to move more smoothly among the corners of the display 220 .
- other appropriate virtual information and/or image guidance cues can be overlaid on the 2D view 205 of the medical image as well as the 3D view 204 .
- Examples include: orientation indicator 214 , an indication of the distance between the medical device's tip (for example, the tip of the first limb member or the tip of the second limb member) and the point in the plane of the medical image that is closest to the medical device tip; ghost affected area (e.g., a cross section or outline of the affected region projected on the ultrasound slice); and/or intersection indicators (e.g., a point, box, outline, etc.) indicating an intersection between one or more axes or trajectories of a display object and a plane-of-interest (e.g., a medical image plane, a medical device plane, etc.).
- orientation indicator 214 an indication of the distance between the medical device's tip (for example, the tip of the first limb member or the tip of the second limb member) and the point in the plane of the medical image that is closest
- the system 100 can generate and/or display graphical indicators that help indicate the spatial relationship between a medical device and an medical image plane (e.g., graphical image plane indicators) or other plane (e.g., graphical plane indicators), indicators to indicate the relative positions of the medical device(s) and medical image(s), features of interest, annotations, plane indicators, plane intersection indicators, other graphical indicators, approximate medical device location indicators, etc.
- an medical image plane e.g., graphical image plane indicators
- other plane e.g., graphical plane indicators
- the various image guidance cues can be generated based at least in part on the emplacement information of the medical devices used with the system 100 .
- one or more of the medical devices or the system 100 can be a multi-axis medical device.
- the multi-axis medical device 203 can be a number of devices including two or more axes or jaws, such as, but not limited to graspers, staplers, vessel sealers, electrocauterizers, resecting devices, transecting devices and scalpels.
- a distal end of the multi-axis medical device 202 includes a joint member 211 , and two limb members 213 , 210 coupled to the joint member 211 at different angles.
- the multi-axis medical device 202 can have a number of associated axes.
- the joint member 211 can have a joint member axis 198 A
- the first limb member 213 can have a first axis 198 B
- the second limb member 210 can have a second axis 198 C that is different from the joint member axis 198 A and the first axis 198 B.
- the joint member axis 198 A is the same as the first axis 198 B.
- the joint member axis 198 A, first axis 198 B, and second axis 198 C can all be different.
- the first member 213 may be angled relative to the joint member 211 and the second member 210 .
- the multi-axis medical device 202 can have one or more primary axis and/or one or more secondary axis.
- the primary axis can correspond to the longitudinal axis of a body of a medical device or portion of a medical device where two or more limbs are joined, such as joint member 211 (or joint member axis 198 A) and the secondary axis or axes can correspond to the longitudinal axis or axes of the limb members of the medical device, such as limb members 213 , 210 (or first axis 198 B and/or second axis 198 C).
- one or more of the secondary axes can be coincident with the primary axis or parallel to the primary axis. In certain cases, one or more of the secondary axes can move relative to the primary axis.
- the one or more limb members 213 , 210 can be movable and/or fixed.
- the first limb member 213 (or lower jaw) is fixed relative to the joint member 211
- the secondary axis corresponding to the limb member 213 is parallel to the primary axis.
- one or more of the limb members can be an extension of the joint member 211 .
- the medical device is a stapler with two jaws
- the primary axis can correspond to the joint member axis 198 A of the stapler or where the two jaws meet and the secondary axes can correspond to the axes of the limb members 198 B, 198 C.
- a second secondary axis can correspond to the axis 198 C of the second limb member 210 (or upper jaw) and is not parallel to the joint member axis 198 A or the first axis 198 B in a first configuration.
- the second limb member 210 can be moveable relative to the joint member 211 and the first limb member 213 .
- the second limb member 210 is hingeably connected to the joint member 211 .
- the range of motion of the second limb member 210 can be an operating parameter input into the system 100 .
- the range of motion of the second limb member 210 is limited by the hinge.
- a limb member e.g., upper or lower jaw
- the second limb member 210 can be parallel or approximately parallel (e.g., within 5, 10, 15, or 20 degrees of parallel (+/ ⁇ a few degrees)) to the joint member 211 .
- the secondary axis 198 C can change relative to the primary axis as the second jaw 210 moves.
- the stapler 202 can be in a first configuration when the stapler in not activated.
- the stapler may be in a second (or stapling) configuration when the stapler in activated (e.g., the staples are being applied).
- the second jaw 210 can be parallel (or approximately parallel) to the joint member 205 .
- the secondary axis 198 C in the first configuration, is not parallel to the primary axis 198 A, and in the second configuration, the secondary axis 198 C is parallel (or approximately parallel) to the primary axis 198 A. Further, while moving from the first configuration to the second configuration, the secondary axis 198 C is not parallel to the primary axis 198 A.
- both the first limb member 213 and the second limb member 210 are affixed to the joint member at different angles such that the primary axis and secondary axis or axes are all different (e.g., the joint member axis 198 A, the first axis 198 B, and the second axis 198 C are all nonparallel).
- the primary axis and secondary axis or axes are all different (e.g., the joint member axis 198 A, the first axis 198 B, and the second axis 198 C are all nonparallel).
- the multi-axis medical device 202 can have more than three axes and/or can have more than one primary axis.
- the joint member 211 can have more than one axis.
- the joint member can include one or more bends or hinges (as depicted in FIG. 10 ) wherefrom the joint member 211 can be articulated relative to a proximal shaft of the medical device.
- the system can provide image prediction information related to the medical devices as image guidance cues.
- the display 220 illustrates a 3D model of a multi-axis medical device 202 , a second medical device 223 , a medical image 204 , and image guidance cues.
- FIG. 2 depicts the multi-axis medical device as a virtual stapler 202
- the virtual medical device 202 could include any two jaw device, including but not limited to a transector, resector, sealer, or grasper.
- the depicted multi-axis device includes three axes, in some embodiments, the multi-axis device can include two, three, four, five or more axes.
- the multi-axis device could include three limb members extending out along nonparallel axes from a joint member.
- the medical device includes a joint member 211 having a joint member axis 198 A, a first limb member 213 having a first axis 198 B, and a second limb member 210 having a second axis 198 C that is different from the joint member axis 198 A and the first axis 198 B.
- the primary axis can correspond to the joint member axis 198 A or longitudinal axis of the joint member 211
- the secondary axes can correspond to the first axis 198 B and the second axis 198 C.
- one secondary axis e.g., first axis 198 B
- another secondary axis e.g., second axis 198 C
- any of the secondary axis can be parallel or not parallel to the primary axis.
- a display object includes more than one primary axis.
- a trajectory (or trajectory indicator) of a display object can be determined and displayed.
- a trajectory indicator can be displayed to indicate the trajectory of an associated display object, if that display object is moved along a drive path.
- the drive path can be dynamic in that it can correlate to the path of the medical device when moved by a user. For instance, the system can determine that the display object (e.g., medical device) is being moved and, in response, can calculate a drive path which corresponding to the path the medical device would take if it continued to move in the same direction.
- the drive path can be predetermined or dynamic.
- the drive path can correspond to a path that is parallel to a primary axis of the display object (akin to moving the medical device forward/backward).
- the drive path can correspond to a path that is perpendicular to a primary axis of the display object (akin to moving the medical device up/down or side-to-side).
- the drive path can be parallel or perpendicular to a secondary axis.
- a display object can have a predetermined drive path when the system determines the display object is not moving and can have a dynamic drive path when the system detects movement.
- the display object can be a multi-axis medical device (e.g., a virtual stapler 202 ) having a joint member 211 , a first limb member 213 , and second limb member 210 .
- a trajectory can be calculated, determined, and displayed for the first limb member 213 , the second limb member 210 and/or the joint member 211 .
- each of the trajectories can be calculated based on a drive path for the medical device.
- the trajectory for each member of the medical device can correspond to the path that that portion of the medical device (i.e., the joint member 211 , the first limb member 213 , and the second limb member) will follow if the medical device is moved along the drive path.
- the system 100 can display a trajectory indicator for a distal portion of each limb member and/or joint member of the medical device.
- a trajectory indicator can be displayed for any and/or all portions of a display object.
- a trajectory indicator may be displayed for some portions (e.g., the first limb member) and not displayed for other portions (e.g., the second limb member).
- trajectory indicators can indicate a trajectory along an axis that is different from and parallel to the primary axis (non-limiting example: an axis corresponding to a distal portion of a limb member along the drive path).
- the trajectory indicators can extend through or to a distal portion (e.g., the tip) of the medical device (e.g., a distal portion of the second limb member 310 ).
- intersection indicators can correspond to, or track, an axis that intersects with a limb member axis (or secondary axis) at a distal portion of the limb member and is parallel to the joint member axis (or primary axis) and/or the drive path.
- the trajectory indicator(s) can be determined based on a trajectory of the medical device along a primary axis (non-limiting example: drive path that is parallel to the primary axis).
- the trajectory of the limb members can be determined based on axes that are different from, but parallel to the primary axis and extend through a distal (or other) portion of the limb members.
- FIG. 2 illustrates a trajectory indicator 208 of a knife path of the first limb member 213
- the system can additionally or alternatively include a trajectory indicator of the second limb member 210 (as shown in FIG. 3 ), joint member 211 , and/or a transecting knife (as shown in FIG. 3 ).
- the trajectory indicator can indicate the trajectory of the medical device along a primary axis and/or the secondary axis.
- the system can display one or more trajectory indicators, which can indicate the trajectory of a display object if the display object is moved along a drive path.
- the trajectory (e.g., trajectory indicator 208 ) of the medical device 202 can be depicted on the display 220 and can show the healthcare provider the projected path that the medical device will take when moved.
- a trajectory indicator can be displayed for any display object (for example, virtual medical devices, affected regions, ghost display objects, image guidance cues, etc.) to indicate the trajectory of the display object if moved along a drive path (e.g., corresponding to a primary axis, a secondary axis, an axis parallel to the primary axis, etc.).
- a trajectory indicator indicates the trajectory of a display object if the display object is moved along an axis parallel to the axis of another display object's axis or plane.
- a display object can be geometrically projected onto a plane-of-interest (i.e., the plane to which that the display object is to be projected) to display a ghost display object (e.g., an outline, cross section, or indication of the display object on the plane-of-interest).
- the ghost display object can be a ghost medical device having a joint member, a first limb member, and second limb member.
- the system can determine and/or calculate a trajectory of the ghost medical device by determining the trajectory of the joint member, first limb member, and/or second limb member if the ghost medical device is moved along a primary or secondary axis of the ghost medical device.
- the trajectory of multiple display objects can be determined and displayed simultaneously on screen 220 , as described in greater detail in the '274 Application.
- the ghost display object can correspond to any display object.
- the ghost display object can correspond a single axis medical device, such as a needle, as described in greater detail in the '274 Application.
- the guidance cues can include a ghost medical device 230 (also referred to as a projected or mapped medical device) on the image plane.
- the system can additionally or alternatively display one or more trajectories (e.g., trajectory indicators) of the ghost medical device, such as trajectory indicator 231 .
- the image guidance system can draw a number of rings about the joint member axis 198 A, the first axis 198 B, the second axis 198 C, or an axis parallel or perpendicular to the joint member axis 198 A, the first axis 198 B, or the second axis 198 C and emanating from a distal (or other) end of a limb member.
- the trajectory indicators can be extrapolated beyond the tip of the first limb member, second limb member or joint member.
- the image guidance system can draw similar trajectory indicators about the one or more axes corresponding to the ghost medical device 230 .
- ghost medical device 330 can include a first and second limb member from which a trajectory indicator can be drawn.
- the image guidance system can draw a number of rings about an axis of another medical device (for instance, the imaging device).
- a healthcare provider can view and manipulate the emplacement of the medical device 242 and its trajectory (via an associated trajectory indicator) before it enters the patient's body along a drive path.
- this is accomplished by the doctor positioning a trajectory indicator (e.g., virtual rings or cues) such that it is coincident (or passes through) an ultrasound representation of a target such as tissue that the doctor has spotted in the ultrasound.
- Positioning the trajectory indicator(s) such that it is properly aimed at the target can provide guidance to the doctor as she directs a medical device 242 into the body until it reaches its desired target or destination.
- the doctor can align the stapler 242 such that the trajectory indicators on display 220 intersect or otherwise indicate that the medical device, if directed along the appropriate axis, will reach the tissue 212 .
- the doctor may be advantageous for the doctor to position the trajectory indicator (e.g., virtual rings, cues, etc.) such that it is not coincident (or does not pass through) the ultrasound representation of a target.
- the trajectory indicator e.g., virtual rings, cues, etc.
- the rings can, in some embodiments, be spaced at regular (e.g., 0.5, 1, or 2 cm) intervals to provide the healthcare provider with visual or guidance cues regarding the distance from the medical device tip to the targeted anatomy.
- the spacing of the rings can indicate other aspects of the data, such as the speed of the medical device, the density of the tissue, the distance to a landmark, such as the ultrasound data, or any other appropriate guidance data or property.
- the rings or other trajectory indicators can extend beyond the medical device tip, by a distance equal to the length of the medical device shaft. This way, the user knows if the medical device is long enough to reach the target, even before the tip(s) enter the patient.
- the trajectory indicator 308 can extend to or even past the image plane.
- Other display markers can be used to show trajectory, such as a dashed, dotted, or solid line, transparent medical device shaft, point cloud, wire frame, etc.
- three-dimensional rings can be used and provide depth cues and obscure little of the medical image.
- Virtual rings or other virtual markers can be displayed semi-transparently, so that they obscure less of the medical image than an opaque marker would.
- connecting lines 225 can also be drawn from a plane-of-interest (e.g., medical image plane) to a display object (e.g., virtual medical device 202 ) or a display object plane (e.g., virtual medical device plane).
- the connecting lines extend orthogonally or perpendicularly from a plane-of-interest to a display object plane or a display object.
- the connecting lines extend orthogonally or perpendicularly from a display object plane or a display object to a plane-of-interest.
- the connecting lines 225 are not orthogonal (or perpendicular) to any plane.
- the connecting lines 225 can be used to determine or display the difference in angle between a plane-of-interest and a display object plane.
- the connecting lines can have an arc. The arc can be determined by treating the intersection of the two display object trajectories as the center of a sphere or circle and treating the connecting line as a perimeter of the circle or sphere.
- Other prediction information can also be displayed as image guidance cues. For example, if a stapler is being tracked by the image guidance system, then a cutting plane corresponding to the trajectory of a transecting knife of the stapler can be displayed. Such a cutting plan can be coplanar with the transecting knife of the stapler and can extend from the transecting knife of the stapler (for instance, along the axis of the first member) to, for instance, a plane-of-interest.
- the trajectory indicator of the transecting knife e.g., cutting plane
- Similar prediction information can be estimable or determinable for graspers, vessel sealers, resecting devices, transecting devices, scalpels cauterizers, lasers, and numerous other medical devices.
- the data from two or more devices can be combined and displayed based on their relative emplacements or poses.
- the system 100 can determine the emplacement of an image plane based on the emplacement information of the ultrasound probe 222 .
- the medical image 204 can be displayed on the image plane with respect to the virtual medical device 202 on the display 220 in a manner that estimates the relative emplacements or poses of an ultrasound probe 222 and the medical device 242 .
- the image guidance cues associated with the virtual medical device 202 including trajectory indicators, are shown spatially located with the medical image 204 on display 220 .
- the system 100 can provide additional guidance cues that facilitate understanding of the relationship between the image plane and the medical device(s).
- display objects e.g., medical display objects (medical devices, medical images, affected regions, etc.) or image guidance cues (trajectory indicators, intersection indicators, plane indicators, etc.)
- display objects can be geometrically projected onto a plane-of-interest such that a shadow, outline (e.g., a dashed, dotted, or solid line), marking, transparent display object, point cloud, wire frame, or other indication of the display object is displayed on the plane-of-interest.
- the projected display objects e.g., projected medical devices, projected medical images, projected image guidance cues, etc.
- ghost display objects e.g., projected medical devices, projected medical images, projected image guidance cues, etc.
- the plane-of-interest is the plane to which that the display object is to be projected.
- the plane-of-interest can be an image plane (e.g., the plane corresponding to a medical image), a medical device plane (e.g., the plane corresponding to a virtual medical device), or any other display object plane or other desired plane.
- the plane-of-interest can include only a portion of a plane (e.g., the medical image portion of the image plane).
- a plane-of-interest can include more than a portion of any plane including but not limited to the image plane, medical device plane, affected region plane, or other display object plane.
- the plane-of-interest can be determined dynamically based on the display object plane. Further, in some cases, the plane-of-interest can be selected to be a plane that is parallel to the display object plane. For example, the plane-of-interest can be selected as the display object plane (as described below) that is parallel to the image plane of a medical image. However, it will be understood that in some cases, the plane-of-interest can be predetermined, such as a display object plane that bisects the display object into halves.
- the display object plane corresponds to any plane that is parallel to a primary or secondary axis of the display object. In some embodiments, the display object plane corresponds to a plane that bisects the display object (e.g., a virtual medical device) into portions (e.g., left and right portions, top and bottom portions, etc.). In some cases, a display object plane can be determined dynamically based on the plane-of-interest. Further, in certain cases, the display object plane can be selected to be a plane that bisects the display object and that is parallel to the plane-of-interest.
- the display object plane can be selected as a plane (e.g., a plane corresponding to an axis of the display object) that is parallel to the plane-of-interest.
- the display object plane can change as the display object moves throughout the system.
- the display object plane can be predetermined.
- a ghost display object (e.g., projected display object) can be projected onto a plane-of-interest using a variety of techniques.
- the ghost display object can be an orthographic projection of the display object onto the plane-of-interest.
- the display object is a virtual medical device and the plane-of-interest is an image plane.
- the system calculates a plurality of lines which are perpendicular to the image plane. For each perpendicular line that intersects the virtual medical device (or a subset thereof), a point is displayed on the image plane at the location corresponding to the intersecting perpendicular line.
- a plurality of points are displayed on the image plane which correspond to the plurality of perpendicular lines (or subset thereof) intersecting the virtual medical device.
- the plurality of points can illustrate an indication of the virtual medical device on the image plane.
- the plurality of points can be referred to as a ghost, a shadow, an outline, a marking, a projection, or an indication of the virtual medical device.
- the number of perpendicular lines (and the number of corresponding points) can vary across embodiments. For example, in some embodiments, there can be almost infinite perpendicular lines such that a complete representation of the virtual medical device is displayed on the image plane.
- the perpendicular lines can be spaced out at predefined intervals such that the ghost medical device includes a dotted outline or dotted representation of the medical device.
- the subset of perpendicular lines used to generate the ghost display object can correspond to lines that intersect with a particular portion of the virtual medical device, such as the edge thereby creating an outline on the plane-of-interest.
- a ghost display object can be projected onto a plane-of-interest using a virtual light source.
- a virtual light source that is at least as large as the display object to be projected or that covers an entire plane (e.g., a light source plane) positioned behind the display object (e.g., distal the plane-of-interest relative to the display object) and parallel to the plane-of-interest, and that emits parallel light waves.
- the ghost display object can be displayed on the plane-on-interest as a shadow of the display object. Because the light source is infinitely wide (and can be infinitely far away), the size of the ghost display object can correspond directly with the size of the display object. Further, it will be understood that in some cases, the system can display an outline of the shadow generated by the virtual light source or other indicator.
- a display object can be projected by identifying points on a display object plane that correspond (e.g., intersect) to the display and mapping (e.g., projecting, reproducing, copying) the identified points onto a plane-of-interest.
- the identified points on the display object plane can include a plurality of points (e.g., coordinates) that intersect with the corresponding display object.
- the plane of a virtual medical device can be a plane that bisects the virtual medical device.
- the plane can include a plurality of points that intersect with the virtual medical device.
- to project the virtual medical device onto an image plane at least some of the plurality of intersecting points can be drawn on the image plane.
- the resulting ghost medical device can include an outline or cross-section of the virtual medical device.
- a display object can be projected by identifying or determining which points (e.g., coordinates) of the plane-of-interest are closest (e.g., nearest) to points (e.g., coordinates) of the display object or points of the display object intersecting the display object plane (as described above).
- Each of the points corresponding to the display object (or the display object plane) can be projected onto the plane-of-interest at the location of the points of the plane-of-interest that were determined to be closest to the identified points of the display object.
- projecting can include determining portions of the plane-of-interest that are closest to the display object.
- the display object can be projected onto those portions of the plane-of-interest that are closest to the display object.
- projecting can include determining coordinates of a cross-sectional outline of the display object that are closest to the plane-of-interest.
- the ghost display object can be a ghost medical device (for example, a ghost stapler) and the plane-of-interest can be an image plane.
- the virtual medical device (for example, the virtual stapler) can include a plurality of points on a stapler plane (e.g., a plane corresponding to an axis of the stapler, a plane that bisects the stapler into approximate halves, a plane that is parallel to the image plane, etc.).
- the system 100 determines which points on the image plane are closest or nearest to the points of the virtual stapler on the stapler plane.
- the ghost medical device is projected by drawing the points of the virtual medical device onto the determined nearest points of the image plane.
- a ghost display object is an outline or other indication of all or a portion of the display object.
- the ghost display object is a cross-sectional outline of the display object or a portion thereof.
- a ghost display object is a projection of a full or partial representation of the display object.
- the ghost medical device can include a representation of a first limb member, the second limb member, and/or joint member on the image plane.
- the ghost display object can be a ghost medical image and the plane-of-interest can be a medical device plane.
- the medical device plane can be a plane that bisects the left and right portions (or halves) or top and bottom portions (or halves) of the virtual medical device or portion thereof, such as the joint member.
- the virtual medical image can include a plurality of points on the image plane.
- the system 100 determines which points on the medical device plane are closest or nearest to the points of the virtual medical image.
- the ghost medical image is projected by drawing the points of the virtual medical image onto the determined nearest points of the medical device plane.
- the ghost display object is a ghost affected region and the plane-of-interest is an image plane.
- the affected region can include a plurality of points on a display object plane (for example, a medical device plane, an affected region plane, etc.).
- the system 100 can determine which points on the image plane are closest or nearest to the points of the affected region on the display object plane.
- the ghost affected region can be projected by drawing the points from the affected region on the display object plane onto the determined nearest points of the image plane.
- a display object can be projected by determining an arc path and projecting the display object on the plane-of-interest along the arc path.
- the system can determine the arc path by first determining an intersection of a display object trajectory and the plane of interest. The system can use the intersection as the center of a sphere and determine an angle between the display object trajectory and plane-of-interest. An arc path can be determined from the angle and each of the points corresponding to the display object (as described above) can then be radially mapped to the plane-of-interest using the arc path.
- a distance between each of the points corresponding to the display object and the point of intersection can be determined. In examples such as these, each of the points can then be projected onto the plane-of-interest by placing the points on the plane-of-interest at the same distances they were on the display object plane.
- the ghost display object is a ghost medical device and the plane-of-interest is an image plane.
- the system 100 determines the intersection of the trajectory of the virtual medical device and the image plane.
- the virtual medical device can include a plurality of points on the medical device plane.
- the system 100 determines the distance between the intersection point (i.e., the center of the circle) and each of the plurality points defining the medical device on the medical device plane.
- the system then projects the points of the medical device onto the image plane at the same distance from the intersection point, but along the image plane.
- FIG. 3 illustrates some embodiments of a medical device projected onto an image plane.
- the ghost medical device 330 can include an outline or other indication of all or a portion of the virtual medical device 302 .
- the system can display a cross-sectional outline of the virtual medical device 302 or a portion thereof.
- the system displays a cross-sectional outline of a side portion of the virtual medical device 302 .
- the system can display more, less, or different parts of the medical device.
- the system can display a projection of a full or partial representation of the first limb member, the second limb member, the limb member, and/or the affected region onto the image plane.
- FIG. 3 depicts the ghost medical device 330 projected across the entire image plane
- the ghost medical device 330 is projected on only a portion of the image plane, such as within the medical image.
- the ghost medical device 330 can be projected onto more than a portion of the image plane.
- only the affected region or a portion of the affected region is projected onto the plane-of-interest, such as an image plane or medical image, thereby allowing a healthcare provider to see the region of tissue that would be affected if the medical device 302 were positioned on the image plane.
- the plane-of-interest such as an image plane or medical image
- the system includes trajectory indicators of multiple display objects (e.g., virtual medical device 302 and ghost medical device 330 .)
- FIG. 3 illustrates trajectory indicator 308 A and trajectory indicator 308 B, which correspond to the first and second limb member, respectively, if the virtual medical device is moved along a projected drive path (e.g., the primary axis.)
- FIG. 3 illustrates trajectory 308 C and trajectory 308 D corresponding to the ghost first and second limb member, respectively, if the ghost medical device is moved along a projected drive path (e.g., the primary axis of the ghost medical device).
- a trajectory can be displayed for any display object, including but not limited to a virtual medical device, a ghost medical device, a medical image, a ghost medical image, an affected area, a ghost affected area, etc.
- the trajectory can be relative to any axis.
- the display can display a trajectory indicator along a primary and/or secondary axis.
- Trajectory indicators for ghost display objects can be calculated, determined, and displayed using a variety of techniques.
- the trajectory indicators 308 C, 308 D can be projected onto the plane-of-interest using techniques as described above with respect to projecting a display object onto a plane-of-interest.
- the trajectory indicators 308 C, 308 D can be determined by calculating a trajectory along an axis of the plane-of-interest.
- the trajectory indicators 308 C, 308 D of the ghost medical device 330 correspond to the trajectory indicators 308 A, 308 B of the medical device 302 such that the trajectory indicators 308 C, 308 D intersect with trajectory indicators 308 A, 308 B.
- the trajectory indicators 308 C, 308 D can indicate a trajectory along an axis different than the trajectory axis of the trajectory indicators 308 A, 308 B such that the trajectory indicators do not intersect.
- trajectory indicators 308 C, 308 D can extend along a secondary axis of the ghost medical device 330 and trajectory indicators 308 A, 308 B can extend along a primary axis of the virtual medical device 302 .
- FIG. 4 illustrates some embodiments of a medical image (e.g., medical image 404 ) projected onto a medical device plane.
- the projected medical image 422 (also referred to as ghost medical image 422 ) can include an outline or other indication of all or a portion of the medical image 404 .
- the system can display a cross-sectional outline of the medical image 404 or a portion thereof.
- the ghost medical image 422 may be projected only within the affected region of the virtual medical device 403 .
- the ghost medical image 422 can be displayed in any location relative to the medical device plane.
- the projected medical image 422 can include a cube-shaped representation or a full or partial representation of the medical image.
- the ghost medical image and/or any other ghost display objects described herein can be displayed in conjunction with any other embodiments described herein.
- the system can display one or more trajectory indicators 231 , 208 .
- the system can display trajectory indicator 208 to indicate a trajectory of the medical device 202 along its projected drive path and the system can display trajectory indicator 231 to indicate a trajectory of the ghost display object 230 (e.g., ghost medical device) if moved along its drive path.
- moving the ghost medical device along its projected drive path means moving the virtual medical device along its projected drive path.
- the system can also display trajectory indicator 429 to indicate a trajectory another medical device 423 (e.g., the ultrasound transducer) if directed along an associated drive path.
- a trajectory of the transecting knife path can also be displayed.
- the system can display a trajectory indicator for any medical device or image and along any axis.
- the display 220 can include another image guidance cue in the form of one or more intersection indicators, such as intersection indicator 209 (or intersection indicators 309 A, 309 B of FIG. 3 ).
- intersection indicators can indicate where one or more portions of the virtual medical device 202 will intersect a plane (for example, the plane corresponding to the medical image 204 ) if the virtual medical device is moved along an associated drive path and/or where one or more portions or axes of the virtual medical device 202 intersect the plane.
- the system can display an outline of a portion of the medical device and/or a box or other shape. For example, the system can display a cross-sectional outline of the medical device 202 or a portion thereof.
- the system displays a cross-sectional outline 209 of the lower portion of the medical device 202 and a cross-sectional outline of a knife path 208 .
- the system can display more, less, or different parts of the virtual medical device 202 .
- the system can display the upper portion at the point of intersection and display a box around it as well, as desired.
- additional intersection indicators can be used to show the intersection of one or more trajectories of the virtual medical device and the image plane if the virtual medical device is moved along a different drive path, such as the secondary axis 198 C.
- trajectory lines can be displayed extending away from the virtual medical device to show where the medical will go if moved forward along its principal or primary axis and/or one or more of the secondary axes.
- connecting lines 232 can be drawn from the medical image 204 to the intersection location.
- the intersection indicator 209 can indicate where the trajectory of the virtual medical device, or trajectory indicator 208 , intersects the trajectory of the ghost medical device 230 or trajectory indicator 231 . It should be noted that an intersection indicator can be displayed for the intersection of any two or more trajectories or trajectory indicators, or for the intersection of any one or more portions of the virtual medical device with the image plane. For instance, in some examples, as depicted in FIG. 4 , the intersection indicator 416 can indicate the intersection between the trajectory of the knife path, or trajectory indicator 408 , and the trajectory 429 of the second virtual medical device 423 , or trajectory indicator 423 .
- an image guidance system will render relative location information.
- the relative location information may be shown with color (e.g., objects may be rendered in brighter colors if they are closer), with rendering techniques (e.g., objects may be rendered with transparency so that one object behind another may be visible, but visually appear behind the closer object), with geometry (e.g., a geometric connector may be shown that will allow the viewer to discern the relative relationships), or with any other appropriate technique.
- the image guidance system can draw geometry, such as a line (or rectangle) in the plane of the slice to indicate the needle's and medical image's relative positions.
- the relative locations could also be represented using vertical and horizontal elements coplanar with the ultrasound or other medical image.
- Embodiments of the system can include image guidance cues as part of the image guidance data to depict information related to the region or regions that will be affected by the use of medical devices.
- an image guidance cue displayed by the image guidance system can include affected region information.
- the illustrated embodiment of FIG. 5 shows a virtual multi-axis medical device 502 and an additional 3D cue showing the affected region 526 for the multi-axis medical device 502 , indicating, for instance, a region of tissue that will be affected when the multi-axis device 502 is activated.
- the affected region can correspond to the inter-jaw area which can be a roughly triangular area with a curved edge, located between the multiple axes.
- the affected region can correspond to the volume located between the two jaws (i.e., the first limb member 513 and the second limb member 510 ). In some cases, the affected region can be calculated based at least in part on a distance from the hinge. Additional affected sub-regions can be calculated and displayed as well. For example, the plane at which the transecting knife cuts 514 (the line shown in the middle portion of the bottom jaw 513 running generally left to right) can be calculated and shown as well as the one or more planes corresponding to the one or more staples (in the illustrated embodiment, the five lines 527 or bands running transverse to the knife plane.
- the affected sub-regions can be transverse or perpendicular to each other.
- an affected knife sub-region can be transverse to an affected staple sub-region.
- any of the cues described herein can be displayed in conjunction with the aforementioned cues.
- the system can display the intersection of the affected region or trajectory of the affected region along a drive path with the medical image (or image plane).
- the system can project the affected region, or portions thereof, onto the image plane.
- the ghost affected region can be a roughly triangular area, which can correspond to the area or volume between the limb members that is projected onto the image plane or medical image.
- the system can display an intersection indicator on the image plane that corresponds to the intersection of the trajectory of the first limb member and the medical image or the intersection of the first limb member with the medical image.
- an intersection indicator can be displayed at the intersection of the trajectory of the second limb member and the medical image or the intersection of the second limb member with the medical image.
- the system can use the operating parameters of the medical device 502 and/or measured parameters to determine the affected region (and display the displayed affected region 526 ).
- the affected region's size e.g., girth and length
- the affected region's size can be dependent on the length of the first limb member 513 , the length of the second limb member 510 , the range of movement of the first limb member, the range of movement of the second limb member, etc.
- These dimensions and operating ranges can be manually entered and/or automatically computed by the guidance system based on device or operating parameters, such as, but not limited to, the medical device make and model, power settings, and the like.
- the system can use measured parameters to determine the affected region, such as, but not limited to, range of motion of the limb member(s), distance between the limb members, available space within the body of the patient, dimensions of tissue to be stapled, etc.
- the stapler may have a normal range of motion (known as default settings) that can be adjusted based at least in part on the size of the area to be stapled.
- the measured parameters can be received in real-time as real-time data. For instance, as distance between the stapler jaws is increased or decreased, the system can display an adjusted affected region.
- the system can use one or more formulas, look-up-tables, fixed or default values, or any other appropriate available information, etc. to determine the affected region.
- the system can determine affected regions prior to operating the medical device and/or during operation of the medical device. For example, prior to operating the medical device, the system can determine one or more affected regions and/or during operation of the medical device, the system can determine one or more dynamic affected regions. In some embodiments, the affected region is static until operation of the medical device and then change as the device is operated. In certain embodiments, the system may rely more on operating parameters of the medical device to determine an initial affected region (i.e., the starting distance between the jaws of the medical device) and measured parameters to determine a dynamic affected region (i.e., as the jaws are opening or closing). However, it will be understood that operating parameters and/or measured parameters can be used to determine the initial affected region and/or the dynamic affected region.
- the medical device 300 is inserted into and/or is moved around inside the patient in a closed or partially closed state in order to reduce the likelihood of causing damage with the medical device.
- the display affected region can correspond to the smaller region located between the first limb member and the second limb member or the displayed affected area can be displayed as if the medical device was fully open.
- the operating parameters, measured parameters, formulas, a look-up-tables, fixed or default values, or other information used to determine the affected regions may include some amount of error or variance.
- the variance may be due, for instance, to uncertainty regarding the distance between the first limb member 313 and the second limb member 310 .
- the system can also determine the emplacement of the affected region.
- the emplacement of the affected region can be based at least in part on the emplacement of some or all of the corresponding medical device (or virtual medical device), such as medical device 242 .
- the system can receive emplacement data from one or more emplacement sensors associated with the medical device 242 (non-limiting examples: coupled to or integrated with the medical device 242 , within an optical path of the medical device, etc.).
- the system can use the emplacement data to determine the emplacement of the tracked medical device and/or the emplacement of the virtual medical device 202 corresponding to the medical device 242 .
- the system can determine the emplacement of the medical device 242 and/or virtual medical device 202 with respect to a point-of-view location.
- the emplacement of the affected region can be based at least in part on the emplacement of the stapler (or its virtual version) or at least a portion of it, such as the location(s) on the stapler where the staples are located or portions of the first or second limb members.
- the affected region can be centered at a location on the medical device, such as the portions on the medical device that move when the medical device is operated.
- the posed can be based at least in part on the emplacement of the medical devices.
- the system can display the affected region in a variety of ways. Furthermore, although the illustrated embodiment of FIG. 2 includes only one displayed affected region 206 , it will be understood that one or more affected regions can be displayed corresponding to each medical device 242 that is displayed on the screen 220 and/or multiple affected regions can be displayed corresponding to a single medical device 242 . In some embodiments, the system can display a perspective view of the affected region and/or non-perspective view, such as by displaying the affected region on or with the medical image displayed in the 2D view. Some or all of the affected regions can be displayed as desired. In some embodiments, the portion of the affected region that is displayed can be referred to as the displayed affected.
- the affected region can correspond to a variety of medical procedures. For example, if a cauterizer is tracked as part of an image guidance system, then the affected region can correspond to a cauterization volume. If a laser is tracked as part of the image guidance system, then the affected region can correspond to a projected laser path. If an ablation needed is tracked as part of the image guidance system, then the affected region can correspond to an ablation volume.
- the affected region can correspond to a biopsy volume, an electroporated volume, cryoablation volume, laser ablation volume, high-frequency focused ultrasound ablation (HIFU) volume, external beam radiation therapy volume, and drilling volume (where the display volume corresponds to the region of bone and other tissue that the manually operated, or computer-controlled drill would remove), depending on the type of medical instrument being used.
- a biopsy volume an electroporated volume
- cryoablation volume laser ablation volume
- high-frequency focused ultrasound ablation (HIFU) volume high-frequency focused ultrasound ablation (HIFU) volume
- HIFU high-frequency focused ultrasound ablation
- external beam radiation therapy volume external beam radiation therapy volume
- drilling volume where the display volume corresponds to the region of bone and other tissue that the manually operated, or computer-controlled drill would remove
- the system 100 projects the affected region (or a portion of the affected region) of the virtual medical device to the image plane or medical image.
- the affected region (or a portion of the affected region) is projected onto the plane-of-interest (for example, the image plane) in addition to the ghost medical device.
- the affected region (or a portion of the affected region) is projected onto the plane-of-interest without projecting a ghost medical device.
- projecting the ghost medical device and the affected region causes the image plane to be crowded. Thus, projecting only the affected region can make it easier for the physician to see exactly what tissue will be affected. However, it will be understood that more than the affected region can be projected onto the plane-of-interest.
- FIG. 5 illustrates a multi-axis medical device having two different axes
- a multi-axis device can have any number of axes.
- the multi-axis device can have three, four, five or more axes.
- the multi-axis medical device can be a three-prong grasper (not shown).
- the three-prong grasper can have an axis corresponding to each prong and an axis corresponding to its body.
- the affected region corresponding to the three-prong grasper can include the region between the three prongs, the region corresponding to the volume that will be affected if the prongs are activated, etc.
- FIG. 6 illustrates an embodiment of a displayed medical device 602 having visual tracking fiducials 632 .
- the position sensing unit 140 can include an optical tracker and/or the one or more visually-detectable fiducials 632 can be coupled to the one or more medical devices.
- These markings 632 can be tracked with the optical tracking system using, for instance, laparoscope video.
- marking 632 may be added to any instruments described herein, including but not limited to a transducer, cauterizer, sealer, stapler, transector, resector, grasper, and any other medical device.
- the medical device 602 can be tracked in a variety of ways. As shown above, visual tracking fiducials 632 can be placed on different portions of the medical device 602 . In some cases, sensors, such as 5 DOF sensors or other sensors can be used. In some embodiments, multiple sensors can be coupled to the medical device 602 . For example, one or more sensors can be located on the medical device proximal to the jaw hinge and/or proximal to a second hinge (e.g., second hinge 1028 shown in FIG. 10 ), and/or one or more sensors can be coupled to a lower portion and/or the upper portion of the first and/or second limb member.
- a second hinge e.g., second hinge 1028 shown in FIG. 10
- a medical device 602 can also have markings such as bands around the shaft or jaws, for example, to indicate distance along the shaft or jaws. Healthcare providers performing medical device procedures are often familiar with these markings and can use them to help understand the spatial relationship between the medical device and anatomy.
- the make and model of the medical device is known to the image guidance system and the virtual medical device 602 displayed in display 220 can resemble medical device 242 .
- the features of medical devices that can be rendered in the scene include the overall shape (diameter, angles, cross sectional shape, curvature, etc.), color, distance markers, angle of the jaws, visuals or echogenic fiduciary markers, the state of deployable elements such as tines, paddles, anchors, resection loops, stiffening or steerable sleeves, temperature, radiation, light or magnetic field sensors, lens, waveguides, fluid transfer channels, and the like.
- the system can use a switch, sensor, or button or other mechanism to determine when the jaw has been closed.
- a switch or sensor can be near the handle or other actuator used to close the jaw.
- the system can update the virtual display to show that the jaw has been closed.
- jaw open or closure and distance between the jaws can be operating parameters of the medical device that are used to display the virtual medical device.
- a first medical device can be used to mark a path for a second medical instrument. Accordingly, the system 100 can track the first medical device such that the system can display the marked path for the second medical device.
- FIG. 7 illustrates an example of tracking a first medical device (e.g., a cauterizing device 702 ) which is used to cauterize (or mark) a path (e.g., a stapling, transecting, resecting, grasping path).
- the navigation software can assist the physician to cauterize (or mark) tissue while inspecting the medical image relative to the tip of the cauterizing device.
- the cauterizing device 702 and/or tip of the cauterizing device can be tracked and subsequently used as guidance for a second medical device (e.g., a stapler).
- any of the navigation features and/or guidance cues can also be used for with the cauterizing device 730 .
- the cauterizing device 730 can have associated trajectory indicators, shadow bars, ghost cauterizing device, projections, tip-to-plane distance markers, and any other guidance cues as described herein.
- the medical image 804 can traverse the affected region 826 of the medical device 802 .
- the system 100 can determine and display a point or area of intersection of the affected region 826 with the medical image 804 .
- the system 100 can determine and display a point of area of intersection between a plane (for example, an image plane) or image area and an affected region 826 .
- an intersection indicator 830 can be projected onto the image plane and/or medical image 804 to illustrate the location on the medical image 804 that the stapler will intersect if the stapler is activated (e.g., closed).
- the depicted intersection indicator can appear like a schematic of the multi-axis device superimposed onto the image plane.
- the intersection indicator 830 can be markings such as the guidance cues described above.
- the healthcare provider can identify where a medical device will intersect a plane (e.g., the image plane) if the stapler is operated.
- an ultrasound transducer for example, a miniature ultrasound transducer
- a multi-axis instrument such as a transecting stapler, grasper, or vessel sealer.
- a physician may use the integrated ultrasound transducer and jawed instrument to ensure that the physician can view, in ultrasound, the internal tissue structures before they are stapled, transected, resected, sealed, or grasped, thereby helping the physician avoid critical structures (such as blood vessels).
- a 3D viewing area can include perspective views of each of a virtual integrated device, a medical image, navigation cues (e.g. affected region as shown in FIG. 5 ) or other cues described herein.
- the integrated device can include one or more sensors located at different locations on the device.
- a sensor can be embedded inside, or affixed to one or more of the joint member, the first limb member, the second limb member, and the ultrasound transducer (such as at the tip, along the shaft, or on a handle).
- the system can use the known configuration of the device to determine the proper location between the affected region and the medical image for display purposes.
- FIG. 9 is a diagram illustrating an embodiment of a transducer mounted at the tip of the multi-axis instrument 922 such that the scan plane coincides with the inter-jaw plane.
- a 3D viewing area can include perspective views of each at least a portion of the medical image 904 , at least a portion of the integrated device 922 (for example, the first limb member 913 , the second limb member 910 , and/or the ultrasound transducer), a displayed affected region (not shown), and other image guidance cues (not shown).
- FIG. 9 illustrates the transducer mounted to the tip of the first limb member
- the transducer can be mounted anywhere on the medical device.
- the transducer can be mounted to the second limb member 910 (such as at the tip, along the body, or near the hinge), the first limb member 913 (such as at the tip, along the body, or near the hinge), or the joint member (such as at the tip, along the body, or near/on the hinge).
- FIG. 10 illustrates an embodiment of an ultrasound transducer mounted to a multi-axis instrument.
- the integrated device 1022 includes a transverse-mounted ultrasound transducer.
- the scanning plane of the transducer can be perpendicular to the path of the transecting knife, the stapler axis (i.e., the joint member axis), and/or the first member axis (i.e., the first axis).
- the scanning plane of the transducer can also be perpendicular or parallel to the affected staple sub-regions (as described above).
- FIG. 11 illustrates an embodiment of an ultrasound transducer mounted to a multi-axis instrument.
- the integrated device 1122 shows an ultrasound transducer mounted parallel to the rigid stapler surface and scanning “backwards.” This design attempts to provide better imaging of the area located exactly between the jaws at the moment of stapling, transecting, resecting, sealing, or grasping.
- FIG. 12 illustrates an embodiment of an ultrasound transducer mounted to a multi-axis instrument.
- a 2D phased array miniature transducer is integrated with a multi-axis instrument.
- the integrated device 1222 can scan a volumetric tissue area between the limb members 1210 , 1213 .
- a volumetric rendering method or conventional volume rendering could be used for real-time visualization during stapler guidance. It should be noted that the 2D phased array miniature transducer can be utilized in any of the previous examples.
- the ultrasound transducer can be integrated with the medical device in a variety of ways and that FIGS. 9-12 provide non-limiting examples. Further, the transducers can cover a variety of angles. They can be attached to any portion of the multi-axis instrument (such as either the first limb member or the second limb member) and angled differently than shown. In addition, 2D phased arrays can be flat and thus integrate more seamlessly with tool designs than shown, avoiding local tissue displacement at the transducing element.
- FIG. 13 is a flow diagram illustrative of an embodiment of a routine implemented by the system to determine and cause a display to display one or more intersection indicators corresponding to an intersection between a plane-of-interest and one or more trajectories of a medical device.
- routine 1300 can be implemented by one or more computing devices/components that are associated with the system 100 , such as the position sensing unit 140 , the image guidance unit 130 , surgical system 149 , an HMD, and/or the imager 150 . Accordingly, routine 1300 has been logically associated as being generally performed by the system 100 . However, the following illustrative embodiment should not be construed as limiting.
- routine 1300 can be implemented in a variety of orders.
- the system 100 can implement some blocks concurrently or change the order as desired.
- fewer, more, or different blocks can be used as part of the routine 1300 .
- the solid blocks are part of routine 1300 and the dotted blocks are not implemented.
- any of the blocks (and more or different blocks) can be implemented as part of routine 1300 .
- the system 100 receives first emplacement data associated with a first emplacement sensor and/or a first medical device.
- the first emplacement data can be generated by the first emplacement sensor and/or by a position sensing unit.
- the first emplacement sensor can be associated with the first medical device.
- the first emplacement sensor can be associated with and/or attached to an ultrasound transducer, as described above.
- the first medical device is a multi-axis medical device.
- the first medical device has a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis.
- one or more emplacement sensors can be associated with a first limb member, a second limb member, and/or a joint member of the associated first medical device.
- the system can receive emplacement data from each of these emplacement sensors.
- the system 100 receives second emplacement data associated with a second emplacement sensor and/or a second medical device.
- the second emplacement data can be generated by the second emplacement sensor and/or by a position sensing unit.
- the second emplacement sensor can be associated with the second medical device.
- the second emplacement sensor can be associated with and/or attached to a stapler, as described above with respect to FIGS. 9-12 .
- the second medical device is an imaging device such as an ultrasound transducer.
- the system 100 can determine an emplacement of a first virtual medical device based at least in part on the received first emplacement data.
- the first virtual medical device can correspond to a multi-axis device such as a transecting stapler, grasper, or vessel sealer.
- the virtual medical device corresponds to a single-axis device such as an ablation probe.
- the system 100 can use the first emplacement data and one or more characteristics of the associated first medical device (or a corresponding virtual medical device) to determine the emplacement of at least a portion of the first medical device.
- the at least a portion of the first medical device can include a portion the first limb member, a portion of the second limb member, or a portion of the joint member.
- the characteristics of the associated first medical device may indicate an emplacement of the second limb member relative to the associated first medical device.
- the system 100 can determine the emplacement of the first medical device, for example, relative to known location of emplacement sensor(s) on the first medical device. For example, the system 100 can use a known relationship between the first emplacement data and the emplacement of the medical device (non-limiting example: the second limb member extends 8 cm. away from the hinge at a 45 degree angle relative to the first limb member) and/or use a known relationship between the emplacement of the emplacement sensor(s) and/or associated medical device (or virtual medical device) and the emplacement of the medical device (non-limiting examples: the first limb member begins 4 cm. from the tip of the first medical device (or first virtual medical device) and ends at the hinge of the first medical device (or first virtual medical device), or the first limb member extends 4 cm. in either direction from the ends of one of the emplacement sensors).
- the first limb member begins 4 cm. from the tip of the first medical device (or first virtual medical device) and ends at the hinge of the first medical device (or first virtual medical device), or the first
- the system 100 can determine the emplacement of the first medical device in one or more coordinate systems by mapping the first emplacement data, from one coordinate system to a second coordinate system.
- the first emplacement data may be received with respect to a first coordinate system, such as a position sensing coordinate system, and then mapped to a second coordinate system, such as a 3D scene coordinate system and/or a screen coordinate system.
- the emplacement of the first medical device can be determined with respect to one or more of the coordinate systems.
- the emplacement of the first medical device can be determined after the first emplacement data has been mapped to the second coordinate system, such as the 3D scene coordinate system and/or the screen coordinate system, or the emplacement of the first medical device can be determined for the first coordinate system, such as the position sensing coordinate system, and then mapped to the 3D scene coordinate system and/or the screen coordinate system.
- the second coordinate system such as the 3D scene coordinate system and/or the screen coordinate system
- the emplacement of the first medical device can be determined for the first coordinate system, such as the position sensing coordinate system, and then mapped to the 3D scene coordinate system and/or the screen coordinate system.
- the system 100 can also use point-of-view location (e.g., the location of the point-of-view for viewing the 3D space) to determine the emplacement of the first medical device for viewing.
- point-of-view location e.g., the location of the point-of-view for viewing the 3D space
- the point-of-view location can be a fixed location, such as a predetermined distance/angle from the screen 220 or stand 170 and or a location configured by the user; or the point-of-view location can by dynamic.
- the system can track a user in real-time and determine the point-of-view location based at least in part on the tracked location of the user.
- the system 100 can also use an offset to determine the emplacement of the first virtual medical device for viewing. For example, the system 100 can determine an initial emplacement of the first medical device in the 3D scene coordinate system and/or the screen coordinate system, and then apply an offset to the initial emplacement and/or the system 100 can determine an initial emplacement of the first medical device in the position sensing coordinate system and apply an offset to the initial emplacement prior to mapping the emplacement of the first medical device in the position sensing coordinate system to the 3D scene coordinate system and/or the screen coordinate system.
- the offset can be made in one, or a combination of, coordinate systems, and/or with respect to one, or a combination of, axes. In certain embodiments, the offset can be made along a y-axis (up/down) of the position sensing coordinate system, the 3D scene coordinate system and/or the screen coordinate system.
- the system 100 can adjust the y-coordinate (up/down coordinate) of the determined emplacement of the first medical device (or the first emplacement sensor(s)) in the position sensing coordinate system by the offset amount.
- the system 100 can use the adjusted emplacement.
- the system 100 can adjust the y-coordinate (up/down coordinate) of the determined emplacement of the first medical device (or the first emplacement sensor(s)) in the 3D scene coordinate system and/or the screen coordinate system by the offset amount. Any combination of the above-referenced examples can be used as desired.
- the offset can be made in any one or any combinations of the coordinate systems and with reference to any one or any combination of the axes. For example, the adjustment can be made along any one or any combination of the x-axis, y-axis, or z-axis.
- the offset can be a predetermined offset and/or a dynamic offset.
- a predetermined offset can be used.
- the system 100 can use a static offset based on an average height of males and/or females or average distance between elbows and hands, the height of the user, a distance between the user's elbow and eyes, expected location of a user with respect to the imaged volume, etc.
- the system 100 can use a dynamic offset, such as a determined emplacement of an HMD relative to one or more emplacement sensors, position sensing region, and/or position sensing unit coordinate system.
- the system 100 can determine the emplacement of an HMD relative to a medical device or imaged area and adjust the offset such that the medical image is always in view on the displays of an HMD.
- the system 100 can determine the change in relative emplacement between the HMD and the medical device or imaged area and adjust the offset such that the medical image remains in view in substantially the same emplacement.
- the offset described herein with reference to the first medical device can be applied to any one or any combination of the objects to be displayed and/or to all contents of the virtual 3D scene. In some embodiments, the offset can be applied to some objects to be displayed but not others.
- the system 100 can determine an emplacement of a second virtual medical device based at least in part on the received second emplacement data.
- the second virtual medical device can correspond to an imaging device such as an ultrasound transducer.
- the emplacement of the second virtual medical device can be determined in a similar manner as described with respect to the first medical device at block 1306 .
- the system 100 determines an emplacement of a medical image (e.g., medical image) based at least in part on received second emplacement data.
- the medical image can be an intra-operative and/or real-time medical image, such as a live ultrasound or intra-operative CT scan, or can be a pre-operative image, such as a pre-operative CT or MRI scan image.
- a real-time medical image (or real-time medical imaging stream) can refer to a medical image (or real-time medical imaging stream) received in real-time.
- the medical image received in real-time can correspond to a live image, such as a live medical image generated by an ultrasound or other image, such as a pre-operative or intra-operative CT image or MRI image that is communicated in real-time.
- the system 100 can use the second emplacement data and one or more characteristics of the second emplacement sensor or associated second medical device (or a corresponding second virtual medical device) to determine the emplacement of the medical image.
- the characteristics may indicate an emplacement of the medical image relative to the second emplacement sensor or associated second medical device (or second virtual medical imaging device).
- the system 100 can determine the emplacement of the medical image relative to the second emplacement sensor and/or associated second medical device (or second virtual medical imaging device). For example, the system 100 can use a known relationship between the second emplacement data and the emplacement of the medical image (non-limiting example: the medical image begins 2 cm. away from the of the second emplacement data location in a particular direction and ends 5 cm. away) and/or use a known relationship between the emplacement of the second emplacement sensor and/or associated second medical device (or second virtual medical imaging device) and the emplacement of the medical image (non-limiting examples: the medical image begins 4 cm. from the tip of the second medical device (or second virtual medical imaging device) and ends at the tip of the second medical device (or second virtual medical imaging device), or the medical image extends 2 cm. in either direction from the ends of the second emplacement sensor).
- the medical image begins 4 cm. from the tip of the second medical device (or second virtual medical imaging device) and ends at the tip of the second medical device (or second virtual medical imaging device), or the medical image
- the system 100 can determine the emplacement of the medical image in one or more coordinate systems by mapping the first emplacement data, from one coordinate system to a second coordinate system.
- the second emplacement data may be received with respect to a first coordinate system, such as a position sensing coordinate system, and then mapped to a second coordinate system, such as a 3D scene coordinate system and/or a screen coordinate system.
- the emplacement of the medical image can be determined with respect to one or more of the coordinate systems.
- the emplacement of the medical image can be determined after the second emplacement data has been mapped to the second coordinate system, such as the 3D scene coordinate system and/or the screen coordinate system, or the emplacement of the medical image can be determined for the first coordinate system, such as the position sensing coordinate system, and then mapped to the 3D scene coordinate system and/or the screen coordinate system.
- the second coordinate system such as the 3D scene coordinate system and/or the screen coordinate system
- the emplacement of the medical image can be determined for the first coordinate system, such as the position sensing coordinate system, and then mapped to the 3D scene coordinate system and/or the screen coordinate system.
- the system 100 can also use point-of-view location (e.g., the location of the point-of-view for viewing the 3D space) to determine the emplacement of the medical image for viewing.
- point-of-view location e.g., the location of the point-of-view for viewing the 3D space
- the point-of-view location can be a fixed location, such as a predetermined distance/angle from the screen 220 or stand 170 and or a location configured by the user; or the point-of-view location can by dynamic.
- the system can track a user in real-time and determine the point-of-view location based at least in part on the tracked location of the user.
- the system 100 can also use an offset to determine the emplacement of the medical image for viewing. For example, the system 100 can determine an initial emplacement of the medical image in the 3D scene coordinate system and/or the screen coordinate system, and then apply an offset to the initial emplacement and/or the system 100 can determine an initial emplacement of the medical image in the position sensing coordinate system and apply an offset to the initial emplacement prior to mapping the emplacement of the medical image in the position sensing coordinate system to the 3D scene coordinate system and/or the screen coordinate system.
- the offset can be made in one, or a combination of, coordinate systems, and/or with respect to one, or a combination of, axes.
- the offset can be made along a y-axis (up/down) of the position sensing coordinate system, the 3D scene coordinate system and/or the screen coordinate system.
- the system 100 can adjust the y-coordinate (up/down coordinate) of the determined emplacement of the medical image (or the first emplacement sensor) in the position sensing coordinate system by the offset amount.
- the system 100 can use the adjusted emplacement.
- the system 100 can adjust the y-coordinate (up/down coordinate) of the determined emplacement of the medical image (or the first emplacement sensor) in the 3D scene coordinate system and/or the screen coordinate system by the offset amount. Any combination of the above-referenced examples can be used as desired.
- the offset can be made in any one or any combinations of the coordinate systems and with reference to any one or any combination of the axes. For example, the adjustment can be made along any one or any combination of the x-axis, y-axis, or z-axis.
- the offset can be a predetermined offset and/or a dynamic offset.
- a predetermined offset can be used.
- the system 100 can use a static offset based on an average height of males and/or females or average distance between elbows and hands, the height of the user, a distance between the user's elbow and eyes, expected location of a user with respect to the imaged volume, etc.
- the system 100 can use a dynamic offset, such as a determined emplacement of an HMD relative to one or more emplacement sensors, position sensing region, and/or position sensing unit coordinate system.
- the system 100 can determine the emplacement of an HMD relative to a medical device or imaged area and adjust the offset such that the medical image is always in view on the displays of an HMD.
- the system 100 can determine the change in relative emplacement between the HMD and the medical device or imaged area and adjust the offset such that the medical image remains in view in substantially the same emplacement.
- the offset described herein with reference to the medical image can be applied to any one or any combination of the objects to be displayed and/or to all contents of the virtual 3D scene. In some embodiments, the offset can be applied to some objects to be displayed but not others.
- the system 100 can determine a first intersection between an image plane and a trajectory of the first limb member along a drive path.
- a trajectory can be calculated, determined, and displayed for the first limb member 213 , the second limb member 210 and/or the joint member 211 .
- each of the trajectories can be calculated based on a drive path for the medical device.
- the trajectory can correspond to the path that the first limb member will follow if the medical device is moved along the drive path.
- trajectory indicators can indicate a trajectory along an axis that is different from and parallel to the primary axis (non-limiting example: an axis corresponding to a distal portion of the first limb member along the drive path).
- the trajectory indicators can extend through a distal portion (e.g., the tip) of the medical device (e.g., a distal portion of the first limb member.
- the first intersection indicator can indicate where the first limb member of the first virtual medical device will intersect the image plane (e.g., the plane corresponding to the medical image) if the virtual medical device is moved along an associated drive path and/or where one or more portions or axes of the first virtual medical device intersect the image plane.
- the drive path can be predetermined or dynamic.
- the system 100 can determine a second intersection between an image plane and a trajectory of the second limb member along the drive path.
- the second intersection indicator can be determined using the same techniques as described above and described with respect to block 1312 .
- the trajectory can correspond to the path that the second limb member will follow if the medical device is moved along the drive path.
- the trajectory e.g., the trajectory indicators
- the trajectory can extend through a distal portion (e.g., the tip) of the medical device (e.g., a distal portion of the second limb member.)
- the second intersection indicator can indicate where the second limb member of the first virtual medical device will intersect the image plane (e.g., the plane corresponding to the medical image) if the first virtual medical device is moved along an associated drive path and/or where one or more portions or axes of the first virtual medical device intersect the image plane.
- the system 100 can determine the points of intersection between one or more trajectories and/or trajectory indicators, one or more axes (or portions) of one or more display objects (for example, a virtual medical device, a ghost display object, an affected region, etc.) and/or one or more planes (e.g., the virtual medical device plane, the image plane, etc.) based on the emplacement information of one or more medical devices.
- one or more trajectories and/or trajectory indicators for example, a virtual medical device, a ghost display object, an affected region, etc.
- planes e.g., the virtual medical device plane, the image plane, etc.
- one or more intersection indicators can be determined and displayed for an intersection between a trajectory of a medical device (for example, a trajectory of the first limb member of a stapler) and a plane-of-interest (for example, an image plane).
- a trajectory of a medical device for example, a trajectory of the first limb member of a stapler
- a plane-of-interest for example, an image plane
- One or more intersection indicators can also be displayed for an intersection between a trajectory of the second limb member along a drive path (for example, an axis parallel to the primary axis and intersecting the second limb member) and a plane-of-interest (for example, an image plane).
- one or more intersection indicators can be displayed for an intersection between a trajectory of a medical device (for example, a stapler, an ultrasound transducer, medical imaging device, etc.) and an affected region (or trajectory of an affected region)
- a medical device for example, a stapler, an ultrasound transducer, medical imaging device, etc.
- an affected region or trajectory of an affected region
- one or more intersection indicators can be displayed for an intersection between a trajectory of a medical device (for example, a stapler, an ultrasound transducer, medical imaging device, etc.) and an axis, trajectory, or plane of a transecting knife.
- a medical device for example, a stapler, an ultrasound transducer, medical imaging device, etc.
- one or more intersection indicators can be displayed for an intersection between a trajectory of a ghost display object (for example, a ghost medical device, a ghost affected region, a ghost medical image, etc.), a stapler, an ultrasound transducer, medical imaging device, etc., and an a trajectory of another display object (for example, a medical device, an affected region, a medical image, etc.).
- a trajectory of a ghost display object for example, a ghost medical device, a ghost affected region, a ghost medical image, etc.
- a stapler for example, an ultrasound transducer, medical imaging device, etc.
- an ultrasound transducer for example, a medical imaging device, etc.
- another display object for example, a medical device, an affected region, a medical image, etc.
- the system 100 can cause a display to display a view of a virtual 3D scene including a perspective rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device.
- the 3D rendering of the at least a portion of the first virtual medical device includes at least a portion of the first limb member and at least a portion of the second limb member.
- the 3D rendering of the at least a portion of the first virtual medical device can include any portions of the first virtual device.
- the perspective view or rendering the medical image can be determined and displayed based at least in part on a point-of-view location.
- the point-of-view location can be a fixed point-of-view location or a dynamic point-of-view location.
- the point-of-view location can be with respect to a set location in front of the display and/or can be based on a tracked location of the display, the HMD, or the user.
- the point-of-view location is determined based at least in part on the relative emplacement of an HMD or user with respect to the position sensing region or imaged volume, or the emplacement of an HMD within a position sensing coordinate system.
- the system 100 can enable the user to view different perspective views of the virtual 3D scene based on the changing emplacement of the tracked object (HMD, user) relative to the medical devices 145 , 155 or the position sensing region.
- HMD tracked object
- the system 100 can determine multiple emplacements for the first virtual medical device. For example, the system 100 can determine the emplacement for the medical image for a right-eye view and a left-eye view of a stereoscopic display, such for a HMD. In this way, each display for the HMD can display the medical image from a slightly different perspective corresponding to a right-eye view and a left-eye view, etc.
- the system 100 can include the medical image in the virtual 3D scene and cause the display to concurrently display a perspective rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device, similar to the display of the medical described above with reference to block 1314 .
- the display of the second virtual medical device can be based at least in part on dimensions of a corresponding real medical device (non-limiting examples: size, shape, or other appearance).
- the perspective view or rendering the medical image can be determined and displayed based at least in part on a point-of-view location.
- the system 100 can cause a display to display a view of a virtual 3D scene including a perspective rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image.
- the perspective view or rendering the medical image can be determined and displayed based at least in part on a point-of-view location.
- the system 100 can cause a display to display the first and second trajectory indicators.
- a healthcare provider can use the displayed trajectory indicators to view and manipulate the emplacement of the medical device and its trajectory before it enters the patient's body along a drive path. In some embodiments, this is accomplished by the doctor positioning a trajectory indicator such that it is coincident (or passes through) an ultrasound representation of a target such as tissue that the doctor has spotted in the ultrasound. Positioning the trajectory indicator(s) such that it is properly aimed at the target can provide guidance to the doctor as she directs a medical device into the body until it reaches its desired target or destination. For example, if the doctor identifies tissue in the medical image which needs attention, she can align the medical such that the trajectory indicators on display intersect or otherwise indicate that the medical device, if directed along the appropriate axis, will reach the tissue.
- the system 100 can cause a display to display the first and second intersection indicators.
- the healthcare provider can identify where a medical device (e.g., a multi-axis device) will intersect the plane (e.g., the image plane) if moved along a drive path (e.g., a primary, a secondary axis, axis parallel to the primary or secondary axis, etc.). Based on the desired placement of the medical device, the healthcare provider can change the position and orientation of the medical device as desired.
- the intersection indicators as well as the trajectory indicators can aid a healthcare provider during the placement of a medical device.
- the system can calculate distances between the planes and the tips of the medical devices or other portions of the medical devices. The calculated distances can be displayed and/or used to display the medical devices.
- the system 100 can determine and display a point of intersection between a plane of interest (for example, an image plane, a medical device plane, etc.) and one or more medical device trajectories. For example, the system can determine and display a point of intersection between an image plane and a joint member trajectory. In some embodiments, the system 100 can determine and display a point of intersection between a medical device plane and another medical device trajectory. In some embodiments, the system 100 can determine and display a point of intersection between a ghost display object (or trajectory of a ghost display object) and a plane (for example, an image plane, a medical device plane, etc.).
- a point of intersection can be determined and displayed between a ghost display object (or its trajectory) and a medical device plane.
- a ghost display object is a ghost medical device
- an intersection indicator can be displayed at the intersection of a trajectory of the ghost joint member and the medical device plane.
- an intersection indicator can be displayed at the intersection of the trajectory of the ghost second limb member along the drive path and the medical image.
- the point(s) of intersection are located on the image plane, within the medical image. In other examples, the point(s) of intersection are located on the image plane, outside of the medical image.
- system 100 can implement one or more of the blocks concurrently and/or change the order, as desired.
- the system 100 can concurrently receive the emplacement data from different sources, concurrently receive the medical image, or receive the data in any order.
- the system 100 can concurrently determine the emplacement of the medical image and/or one or more virtual medical devices, etc.
- routine 1300 can include blocks for receiving emplacement data associated with additional emplacement sensors or medical devices, determining emplacements of one or more medical devices, corresponding virtual medical devices, other display objects, displays, and/or users.
- routine 1300 can include determining an emplacement of a medical device associated with the medical image and determining the emplacement of the medical image based at least in part on the determined emplacement of the medical device (or corresponding virtual medical device), and display the virtual medical device concurrently with the medical image.
- the system 100 can determine and display a variety of image guidance cues, such as trajectory indicators, affected region indicators, intersection indicators, medical devices in different states or configurations, ghost display objects, as described in greater detail above, and below with reference to FIG. 14 .
- the routine 1300 can omit certain blocks, such as, but not limited to, blocks 1302 , 1304 , 1308 , 1316 , 1318 , 1322 , and/or 1324 .
- the system may not determine emplacement of or display a portion of the second display object and/or may not display a portion of the first medical device, first trajectory indicator, and/or second trajectory indicator, etc.
- FIG. 14 is a flow diagram illustrative of an embodiment of a routine implemented by the system to project a display object onto a plane-of-interest to display a ghost display object.
- routine 1400 can be implemented by one or more computing devices/components that are associated with the system 100 , such as the position sensing unit 140 , the image guidance unit 130 , surgical system 149 , an HMD, and/or the imager 150 . Accordingly, routine 1400 has been logically associated as being generally performed by the system 100 . However, the following illustrative embodiment should not be construed as limiting. Furthermore, it will be understood that the various blocks described herein with reference to FIG.
- routine 14 can be implemented in a variety of orders.
- the system 100 can implement some blocks concurrently or change the order, as desired.
- fewer, more, or different blocks can be used as part of the routine 1400 .
- the solid blocks are part of routine 1400 and the dotted blocks are not implemented.
- any of the blocks (and more or different blocks) can be implemented as part of routine 1400 .
- the system 100 receives first emplacement data associated with a first emplacement sensor and/or a first medical device.
- the first emplacement sensor can be associated with the first medical device.
- the first medical device is a multi-axis medical device.
- the first medical device can have a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis.
- one or more emplacement sensors can be associated with a first limb member, a second limb member, and/or a joint member of the associated first medical device.
- the system 100 receives second emplacement data associated with a second emplacement sensor and/or a second medical device.
- the system 100 receives a medical image corresponding to the second medical device.
- the system 100 determines an emplacement of a first display object based at least in part on received first emplacement data.
- the first display object can be a multi-axis medical device having a joint member, a first limb member, and a second limb member.
- the second display object can be any medical display objects (e.g., virtual medical device, medical image, ghost display object, etc.) and/or any image guidance cue (e.g., trajectory indicator, intersection indicator, plane indicator, ghost display object, etc.).
- the system 100 determines an emplacement of a second display object based at least in part on received second emplacement data.
- the second display object can be a medical image.
- display object can be any medical display objects (e.g., virtual medical device, medical image, ghost display object, etc.) and/or any image guidance cue (e.g., trajectory indicator, intersection indicator, plane indicator, ghost display object, etc.).
- the system 100 determines an emplacement of a medical image (e.g., medical image) based at least in part on received second emplacement data.
- a medical image e.g., medical image
- the system 100 determines an emplacement of a plane-of-interest.
- the plane-of-interest is the plane to which that the display object is to be projected.
- the plane-of-interest can be an image plane (e.g., the plane corresponding to a medical image), a medical device plane (e.g., the plane corresponding to a virtual medical device), or any other display object plane or other desired plane.
- the plane-of-interest can include only a portion of a plane (e.g., the medical image portion of the image plane).
- a plane-of-interest can include more than a portion of any plane including but not limited to the image plane, medical device plane, affected region plane, or other display object plane.
- the plane-of-interest can be determined dynamically based on the display object plane.
- the plane-of-interest can be selected to be a plane that is parallel to the display object plane.
- the plane-of-interest can be selected as the display object plane (as described below) that is parallel to the image plane of a medical image.
- the plane-of-interest can be predetermined, such as a display object plane that bisects the display object into halves.
- the display object plane corresponds to any plane that is parallel to a primary or secondary axis of the display object.
- a virtual medical device plane can correspond to a plane that bisects the virtual medical device lengthwise or along its longitudinal axis.
- the plane can bisect the virtual medical device into left and right portions (non-limiting examples: halves or other division), top and bottom portions (non-limiting examples: halves or other division), upper-right and lower left portions, upper-left and lower right portions, etc.
- a display object plane can be determined dynamically based on the plane-of-interest.
- the display object plane can be selected to be a plane that bisects the display object and that is parallel to the plane-of-interest.
- the display object plane can be selected as a plane (e.g., a plane corresponding to an axis of the display object) that is parallel to the plane-of-interest.
- the display object plane can change as the display object moves throughout the system.
- the display object plane can be predetermined. The emplacement of the plane-of-interest can be determined using a similar technique as described above.
- the system 100 determines an emplacement of a ghost display object.
- the emplacement of the ghost display object can be determined using a variety of techniques, as described above.
- a ghost object can be projected onto the plane-of-interest.
- a ghost display object e.g., projected display object
- the ghost display object can be an orthographic projection of the display object onto the plane-of-interest.
- the display object is a virtual medical device and the plane-of-interest is an image plane.
- the system calculates a plurality of lines which are perpendicular to the image plane.
- a point is displayed on the image plane at the location corresponding to the intersecting perpendicular line.
- a plurality of points are displayed on the image plane which correspond to the plurality of perpendicular lines (or subset thereof) intersecting the virtual medical device.
- the plurality of points can illustrate an indication of the virtual medical device on the image plane.
- the plurality of points can be referred to as a ghost, a shadow, an outline, a marking, a projection, or an indication of the virtual medical device. It should be understood that the number of perpendicular lines (and the number of corresponding points) can vary across embodiments.
- the perpendicular lines can be spaced out at predefined intervals such that the ghost medical device includes a dotted outline or dotted representation of the medical device.
- the subset of perpendicular lines used to generate the ghost display object can correspond to lines that intersect with a particular portion of the virtual medical device, such as the edge thereby creating an outline on the plane-of-interest.
- a ghost display object can be projected onto a plane-of-interest using a virtual light source.
- a virtual light source that is at least as large as the display object to be projected or that covers an entire plane (e.g., a light source plane) positioned behind the display object (e.g., distal the plane-of-interest relative to the display object) and parallel to the plane-of-interest, and that emits parallel light waves.
- the ghost display object can be displayed on the plane-on-interest as a shadow of the display object. Because the light source is infinitely wide (and can be infinitely far away), the size of the ghost display object can correspond directly with the size of the display object. Further, it will be understood that in some cases, the system can display an outline of the shadow generated by the virtual light source or other indicator.
- a display object can be projected by identifying points on a display object plane that correspond (e.g., intersect) to the display and mapping (e.g., projecting, reproducing, copying) the identified points onto a plane-of-interest.
- the identified points on the display object plane can include a plurality of points (e.g., coordinates) that intersect with the corresponding display object.
- the plane of a virtual medical device can be a plane that bisects the virtual medical device.
- the plane can include a plurality of points that intersect with the virtual medical device.
- to project the virtual medical device onto an image plane at least some of the plurality of intersecting points can be drawn on the image plane.
- the resulting ghost medical device can include an outline or cross-section of the virtual medical device.
- a display object can be projected by identifying or determining which points (e.g., coordinates) of the plane-of-interest are closest (e.g., nearest) to points (e.g., coordinates) of the display object or points of the display object intersecting the display object plane (as described above).
- Each of the points corresponding to the display object (or the display object plane) can be projected onto the plane-of-interest at the location of the points of the plane-of-interest that were determined to be closest to the identified points of the display object.
- projecting can include determining portions of the plane-of-interest that are closest to the display object.
- the display object can be projected onto those portions of the plane-of-interest that are closest to the display object.
- projecting can include determining coordinates of a cross-sectional outline of the display object that are closest to the plane-of-interest.
- a medical image can be projected onto a plane of another medical device (e.g., a multi-axis medical device).
- the system 100 projects at least a portion of the medical image onto a virtual medical device plane of a virtual medical device based at least in part on the determined emplacement of the virtual medical device and the determined emplacement of the medical image and/or medical device associated with the medical image (e.g., a medical imaging device such as an ultrasound transducer).
- the ghost medical image can be displayed in a 2D view or 3D view and/or as a perspective view.
- the system 100 can display a 3D rendering of at least a portion of the first display object, the second display object, or the medical image.
- the system can display a 3D rendering of at least a portion of the ghost display object.
- a ghost display object is an outline or other indication of all or a portion of the display object.
- the ghost display object is a cross-sectional outline of the display object or a portion thereof.
- a ghost display object is a projection of a full or partial representation of the display object.
- a ghost medical device can include a representation of medical device having a first limb member, the second limb member, and/or joint member on the image plane.
- the system can display a rendering of at least a portion of the medical image projected onto the plane-of-interest.
- system 100 can implement one or more of the blocks concurrently and/or change the order, as desired.
- the system 100 can concurrently receive the emplacement data from different sources, concurrently receive the medical image, or receive the data in any order.
- the system 100 can concurrently determine the emplacement of the medical image and/or one or more virtual medical devices, etc.
- routine 1400 can include blocks for receiving emplacement data associated with additional emplacement sensors or medical devices, determining emplacements of one or more medical devices, corresponding virtual medical devices, other display objects, displays, and/or users.
- routine 1400 can include determining an emplacement of a medical device associated with the medical image and determining the emplacement of the medical image based at least in part on the determined emplacement of the medical device (or corresponding virtual medical device), and display the virtual medical device concurrently with the medical image.
- the system 100 can determine and display a variety of image guidance cues, such as trajectory indicators, affected region indicators, as described above.
- routine 1400 can omit certain blocks, such as, but not limited to, blocks 1402 , 1404 , 1406 , 1410 , 1412 , 1420 , and/or 1422 .
- the system may not determine emplacement of or display a portion of the second display object and/or the medical image.
- routine 1400 can be implemented in routine 1400 , in a variety of orders.
- system 100 can implement some or all of various blocks of FIG. 13 in routine 1400 concurrently or change the order as desired.
- FIG. 15 is a flow diagram illustrative of an embodiment of a routine implemented by the system to display an affected region of a medical device.
- routine 1500 can be implemented by one or more computing devices/components that are associated with the system 100 , such as the position sensing unit 140 , the image guidance unit 130 , surgical system 149 , and/or imaging unit 150 . Accordingly, routine 1500 has been logically associated as being generally performed by the system 100 . However, the following illustrative embodiment should not be construed as limiting. Furthermore, it will be understood that the various blocks described herein with reference to FIG. 15 can be implemented in a variety of orders.
- the system may implement some blocks concurrently or change the order, as desired.
- fewer, more, or different blocks can be used as part of the routine 1500 .
- the solid blocks are part of routine 1500 and the dotted blocks are not implemented.
- any of the blocks (and more or different blocks) can be implemented as part of routine 1500 .
- the system 100 receives first emplacement data associated with a first emplacement sensor and/or a first medical device.
- the first emplacement sensor can be associated with the first medical device.
- the first medical device can be a multi-axis medical device.
- the first medical device can have a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis.
- one or more emplacement sensors can be associated with a first limb member, a second limb member, and/or a joint member of the associated first medical device.
- the system 100 receives operating parameters of a medical device.
- the operating parameters can include information regarding make and model, power, dimensions of the medical device (such as length or width of the first limb member and/or the second limb member, angle data, distance from the hinge), range of movement data, etc.
- the operating parameters can be stored in a non-transitory, computer-readable medium associated with the system 100 and/or can be stored in the medical device.
- the operating parameters can be manually entered and/or automatically computed by the guidance system.
- the system 100 can determine an emplacement of a first virtual medical device based at least in part on the received first emplacement data.
- the second virtual medical device can correspond to a multi-axis device such as a transecting stapler, grasper, or vessel sealer.
- the virtual medical device corresponds to a single-axis device such as an ablation probe.
- the system 100 can determine an emplacement of an affected region.
- the system determines the affected region based at least in part on the operating parameters and/or measured parameters.
- an affected region can correspond to a region of tissue that will be affected when the device is operated.
- the first affected region can correspond to a volume located between the first limb member and the second limb member.
- the first affected region can be one or more sub-regions of the region affected when the medical device (e.g., multi-axis medical device) is operated.
- the affected region can correspond to the plane at which the transecting knife cuts and/or the one or more planes corresponding to the one or more staples.
- the emplacement of the affected region can be determined in a similar manner as described above with respect to determining emplacement of a medical image, virtual medical device, ghost display object, etc.
- the system 100 can cause the one or more displays to display a view of a virtual 3D scene including a perspective rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device.
- the 3D rendering of the at least a portion of the first virtual medical device includes at least a portion of the first limb member and at least a portion of the second limb member.
- the 3D rendering of the at least a portion of the first virtual medical device can include any portions of the first virtual device.
- the system 100 causes the one or more displays to display at least a portion of the affected region.
- the displayed affected region can be displayed in a 2D view or 3D view and/or as a perspective view.
- the displayed affected region can be displayed as a volume, area, and/or line.
- the displayed affected region can be wire-framed, transparent, semi-transparent, have varied opacity, brightness, and/or focus, include alternating bands/tiles, be textured, include solid or dashed lines, include spikes, etc.
- the at least a portion of the affected region corresponds to portions of the affected region that are unique to it, with respect to other affected regions.
- the displayed affected region corresponds to at least a portion of the affected region that is co-located with at least a portion of a medical display object (non-limiting examples: a virtual medical device, medical image, etc.) or its trajectory, also referred to as the surface display region.
- a medical display object non-limiting examples: a virtual medical device, medical image, etc.
- its trajectory also referred to as the surface display region.
- the system 100 causes the one or more displays to display an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path.
- the drive path can correspond to one of a plurality of paths.
- the drive path is parallel to a joint member axis.
- the medical image or medical image can traverse the affected region of the medical device.
- the affected region can correspond to a region of tissue that will be affected by a transecting knife and/or staples when a multi-axis device is operated.
- the medical image is traverse (or extends across) at least a portion of the affected region.
- the system 100 can receive emplacement data from one or more sensors corresponding to one or more medical devices, determine a emplacement of the medical devices and/or corresponding virtual medical devices (as a non-limiting example, the emplacement can be determined based at least in part on a point-of-view location), cause one or more displays to display the virtual medical devices and/or perspective views thereof, determine emplacement of the affected regions with respect to the virtual medical devices, determine emplacement of and display an medical image, alter the display of the medical image, display the medical display objects in a 2D view, a 3D view, and/or a perspective view, etc.
- the routine 1500 can omit certain blocks, such as, but not limited to, blocks 1502 , 1504 , and/or 1510 .
- the system may not display a portion of the virtual medical device, etc.
- routine 1500 can be implemented in routine 1500 , in a variety of orders.
- the system 100 can implement some or all of various blocks of FIGS. 13 and/or 14 in routine 1500 concurrently or change the order as desired.
- FIG. 16 is a flow diagram illustrative of an embodiment of a routine implemented by the system to determine emplacement of a virtual medical device and a medical image based at least in part on received emplacement data associated with a medical device.
- routine 1600 can be implemented by one or more computing devices/components that are associated with the system 100 , such as the position sensing unit 140 , the image guidance unit 130 , surgical system 149 , an HMD, and/or the imager 150 . Accordingly, routine 1600 has been logically associated as being generally performed by the system 100 . However, the following illustrative embodiment should not be construed as limiting. Furthermore, it will be understood that the various blocks described herein with reference to FIG.
- 16 can be implemented in a variety of orders.
- the system 100 can implement some blocks concurrently or change the order as desired.
- fewer, more, or different blocks can be used as part of the routine 1600 .
- the solid blocks are part of routine 1600 and the dotted blocks are not implemented.
- any of the blocks (and more or different blocks) can be implemented as part of routine 1600 .
- an ultrasound transducer for example, a miniature ultrasound transducer
- a multi-axis instrument such as a transecting stapler, grasper, or vessel sealer.
- an emplacement of a virtual medical device for example, a stapler
- the emplacement of a virtual medical image can be determined based at least in part on emplacement data associated with a single emplacement sensor.
- the system 100 receives first emplacement data associated with a first emplacement sensor and/or a first medical device.
- the first emplacement data can be generated by the first emplacement sensor and/or by a position sensing unit.
- the first emplacement sensor can be associated with the first medical device (e.g., a stapler).
- the system 100 can determine an emplacement of a first virtual medical device based at least in part on the received first emplacement data.
- the first virtual medical device can correspond to a multi-axis device such as a transecting stapler, grasper, or vessel sealer.
- the virtual medical device corresponds to a single-axis device such as an ablation probe.
- the system 100 can determine an emplacement of a second virtual medical device based at least in part on the received first emplacement data.
- a single sensor and/or single emplacement data can be used to determine the emplacement of the first medical device and the second medical device.
- the system 100 can determine an emplacement of a medical image based at least in part on the received first emplacement data.
- a single sensor and/or single emplacement data can be used to determine the emplacement of the first medical device, the second medical device, and the medical image.
- both the emplacement of the virtual medical device and the emplacement of the virtual ultrasound transducer (and medical image) can be determined based at least at least in part on the received first emplacement data.
- the system 100 can display a 3D rendering of at least a portion of the first display object, the second display object, or the medical image.
- each of these display objects are displayed based at least in part on the first emplacement data.
- system 100 can implement one or more of the blocks concurrently and/or change the order, as desired.
- the system 100 can concurrently receive the emplacement data from different sources, concurrently receive the medical image, or receive the data in any order.
- the system 100 can concurrently determine the emplacement of the medical image and/or one or more virtual medical devices, etc.
- routine 1600 can include blocks for receiving emplacement data associated with additional emplacement sensors or medical devices, determining emplacements of one or more medical devices, corresponding virtual medical devices, other display objects, displays, and/or users.
- routine 1600 can include determining an emplacement of a medical device associated with the medical image and determining the emplacement of the medical image based at least in part on the determined emplacement of the medical device (or corresponding virtual medical device), and display the virtual medical device concurrently with the medical image.
- the system 100 can determine and display a variety of image guidance cues, such as trajectory indicators, affected region indicators, as described above.
- routine 1600 can omit certain blocks, such as, but not limited to, blocks 1602 , 1606 , 1612 , and/or 1614 .
- the system may not determine emplacement of or display a portion of the second display object.
- routine 1600 can be implemented in routine 1600 , in a variety of orders.
- the system 100 can implement some or all of various blocks of FIGS. 14 and 15 in routine 1600 concurrently or change the order as desired.
- a method for medical device navigation comprising: receiving first emplacement data associated with a first medical device, the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
- a method for medical device navigation comprising:
- the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
- Clause 3 The method of Clause 2, further comprising:
- said determining the emplacement of the first virtual medical device is based at least in part on the received first emplacement data and said determining the emplacement of the medical image is based at least in part on the received second emplacement data.
- a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member, or
- Clause 6 The method of any of the Clauses 2-5, wherein the drive path is parallel to at least one of the joint member axis, the first axis, or the second axis.
- Clause 7 The method of any of the Clauses 2-6, wherein the drive path is perpendicular to at least one of the joint member axis, the first axis, or the second axis.
- Clause 8 The method of any of the Clauses 2-7, wherein the drive path is predefined.
- Clause 9 The method of any of the Clauses 2-8, wherein the drive path is dynamic and is based at least in part on detected movement of first medical device or the second medical device.
- Clause 10 The method of any of the Clauses 2-9, wherein a primary axis of the first medical device is the joint member axis.
- Clause 11 The method of any of the Clauses 2-10, wherein a secondary axis of the first medical device is the first axis and/or the second axis.
- Clause 12 The method of any of the Clauses 2-11, wherein the first axis is parallel to the joint member axis.
- Clause 13 The method of any of the Clauses 2-12, wherein the first axis is different from the joint member axis.
- Clause 14 The method of any of the Clauses 2-13, wherein the first intersection indicator includes an outline of a cross-section of the first limb member.
- Clause 15 The method of any of the Clauses 2-14, wherein the second intersection indicator includes an outline of a cross-section of the second limb member.
- Clause 16 The method of any of the Clauses 2-15, wherein the first intersection indicator and/or the second intersection indicator includes a cross-sectional outline of a knife path and/or the first virtual medical device, wherein the knife path corresponds to a transecting knife of the first medical device.
- Clause 17 The method of any of the Clauses 2-16, wherein the first intersection and/or the second intersection indicator includes an indication of at least a portion of the image plane.
- Clause 18 The method of any of the Clauses 2-17, wherein the indication of at least a portion of the image plane comprises a box representative of a portion of the image plane.
- Clause 19 The method of any of the Clauses 2-18, wherein the first trajectory indicator extends to at least a distal portion of the first limb member and corresponds to an axis along the drive path that is parallel to the joint member axis.
- Clause 20 The method of any of the Clauses 2-19, wherein the first trajectory indicator corresponds to an axis that intersects with the first axis at a distal portion of the first limb member and is parallel to the joint member axis and the drive path.
- Clause 22 The method of any of the Clauses 2-213, wherein the second trajectory indicator corresponds to an axis that intersects with the second axis at a distal portion of the second limb member and is parallel to the joint member axis and the drive path.
- Clause 23 The method of any of the Clauses 2-22, further comprising determining a cutting path associated with the first medical device, wherein causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of the cutting path.
- Clause 24 The method of Clause 23, wherein the cutting path corresponds to a trajectory of a knife of the first medical device.
- Clause 25 The method of any of the Clauses 2-24, further comprising:
- causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of:
- Clause 26 The method of Clause 25, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
- Clause 27 The method of Clause 25, wherein the at least a portion of the affected region comprises a stapling path of at least one staple of the first medical device.
- Clause 28 The method of Clause 25, wherein the at least a portion of the affected region comprises a cutting path of a knife of the first medical device.
- Clause 29 The method of Clause 25, further comprising determining an intersection between the affected region and the medical image.
- Clause 30 The method of Clause 29, wherein the at least a portion of the affected region comprises the intersection between the affected region and the medical image.
- Clause 31 The method of Clause 25, further comprising receiving operating parameters corresponding to the first medical device, wherein determining the emplacement of an affected region is based at least in part on the received operating parameters.
- Clause 32 The method of any of the Clauses 2-31, further comprising:
- causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of:
- Clause 33 The method of any of the Clauses 2-32, further comprising:
- causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of:
- Clause 34 The method of any of Clauses 32 or 33, further comprising projecting the display object on the plane-of-interest to display the ghost display object.
- Clause 35 The method of Clause 34, wherein projecting comprises determining at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
- Clause 36 The method of Clause 34, wherein projecting comprises determining an arc path corresponding to the display object and the plane-of-interest.
- Clause 37 The method of Clause 34, wherein projecting comprises determining which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
- Clause 38 The method of Clause 34, wherein projecting comprises using a virtual light source that is at least as large as the display object to be projected, wherein the light source is distal the plane-of-interest relative to the display object.
- a system comprising:
- a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
- a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
- Clause 43 The system of the Clause 42, wherein the computer system is further configured to:
- determine the emplacement of the medical image is based at least in part on the received second emplacement data.
- Clause 44 The system of any of clauses 42 or 43, wherein the computer system is further configured to cause the display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member, or
- Clause 45 The system of any of clauses 42-44, wherein the computer system is further configured to cause the display to concurrently display a perspective view of:
- Clause 50 The system of any of clauses 42-49, wherein a primary axis of the first medical device is the joint member axis.
- Clause 51 The system of any of clauses 42-50, wherein a secondary axis of the first medical device is the first axis and/or the second axis.
- Clause 52 The system of any of clauses 42-51, wherein the first axis is parallel to the joint member axis.
- Clause 53 The system of any of clauses 42-52, wherein the first axis is different from the joint member axis.
- Clause 54 The system of any of clauses 42-53, wherein the first intersection indicator includes an outline of a cross-section of the first limb member.
- Clause 55 The system of any of clauses 42-54, wherein the second intersection indicator includes an outline of a cross-section of the second limb member.
- Clause 56 The system of any of clauses 42-55, wherein the first intersection indicator and/or the second intersection indicator includes a cross-sectional outline of a knife path and/or the first virtual medical device, wherein the knife path corresponds to a transecting knife of the first medical device.
- Clause 57 The system of any of clauses 42-56, wherein the first intersection and/or the second intersection indicator includes an indication of at least a portion of the image plane.
- Clause 58 The system of clause 57, wherein the indication of at least a portion of the image plane comprises a box representative of a portion of the image plane.
- Clause 60 The system of clause 45, wherein the first trajectory indicator corresponds to an axis that intersects with the first axis at a distal portion of the first limb member and is parallel to the joint member axis and the drive path.
- Clause 63 The system of any of clauses 42-62, wherein the computer system is further configured to determine a cutting path associated with the first medical device and cause the display to concurrently display a perspective view of the cutting path.
- Clause 64 The system of clause 63, wherein the cutting path corresponds to a trajectory of a knife of the first medical device.
- Clause 65 The system of any of clauses 42-64, wherein the computer system is further configured to:
- Clause 66 The system of clause 65, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
- Clause 67 The system of clause 65, wherein the at least a portion of the affected region comprises a stapling path of at least one staple of the first medical device.
- Clause 68 The system of clause 65, wherein the at least a portion of the affected region comprises a cutting path of a knife of the first medical device.
- Clause 69 The system of clause 65, wherein the at least a portion of the affected region comprises the intersection between the affected region and the medical image.
- Clause 70 The system of any of clauses 42-69, wherein the computer system is further configured to determine an intersection between the affected region and the medical image.
- Clause 71 The system of any of clauses 42-70, wherein the computer system is further configured to receive operating parameters corresponding to the first medical device and determine the emplacement of an affected region based at least in part on the received operating parameters.
- Clause 72 The system of any of clauses 42-70, wherein the computer system is further configured to:
- the display to concurrently display a perspective view of a 3D rendering of at least a portion of the ghost medical device on the plane-of-interest based at least in part on the determined emplacement of the ghost medical device.
- the display to currently display a perspective view of a 3D rendering of at least a portion of the ghost medical image on the plane-of-interest based at least in part on the determined emplacement of the ghost medical image.
- Clause 74 The system of any one of clause 72 or 73, wherein the computer system is further configured to project the display object on the plane-of-interest to display the ghost display object.
- Clause 75 The system of the Clause 74, wherein the computer system is further configured to determine at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
- Clause 76 The system of any of clauses 74 or 75, wherein the computer system is further configured to determine an arc path corresponding to the display object and the plane-of-interest.
- Clause 77 The system of any of clauses 74-76, wherein the computer system is further configured to determine which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
- Clause 78 The system of any of clauses 74-77, wherein the computer system further comprises a virtual light source that is at least as large as the display object to be projected, wherein the light source is distal the plane-of-interest relative to the display object.
- a non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
- a non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
- Clause 83 The non-transitory computer-readable medium of clause 82, wherein the computer executable instructions further cause the one or more processors to:
- determine the emplacement of the medical image is based at least in part on the received second emplacement data.
- Clause 84 The non-transitory computer-readable medium of clauses 82 or 83, wherein the computer executable instructions further cause the one or more processors to cause the display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member, or
- Clause 85 The non-transitory computer-readable medium of clauses 82-84, wherein the computer executable instructions further cause the one or more processors to cause the display to concurrently display a perspective view of:
- Clause 86 The non-transitory computer-readable medium of any of clauses 82-85, wherein the drive path is parallel to at least one of the joint member axis, the first axis, or the second axis.
- Clause 87 The non-transitory computer-readable medium of any of clauses 82-86, wherein the drive path is perpendicular to at least one of the joint member axis, the first axis, or the second axis.
- Clause 88 The non-transitory computer-readable medium of any of clauses 82-87, wherein the drive path is predefined.
- Clause 90 The non-transitory computer-readable medium of any of clauses 82-89, wherein a primary axis of the first medical device is the joint member axis.
- Clause 91 The non-transitory computer-readable medium of any of clauses 82-90, wherein a secondary axis of the first medical device is the first axis and/or the second axis.
- Clause 92 The non-transitory computer-readable medium of any of clauses 82-91, wherein the first axis is parallel to the joint member axis.
- Clause 93 The non-transitory computer-readable medium of any of clauses 82-92, wherein the first axis is different from the joint member axis.
- Clause 94 The non-transitory computer-readable medium of any of clauses 82-93, wherein the first intersection indicator includes an outline of a cross-section of the first limb member.
- Clause 95 The non-transitory computer-readable medium of any of clauses 82-94, wherein the second intersection indicator includes an outline of a cross-section of the second limb member.
- Clause 96 The non-transitory computer-readable medium of any of clauses 82-95, wherein the first intersection indicator and/or the second intersection indicator includes a cross-sectional outline of a knife path and/or the first virtual medical device, wherein the knife path corresponds to a transecting knife of the first medical device.
- Clause 97 The non-transitory computer-readable medium of any of clauses 82-96, wherein the first intersection and/or the second intersection indicator includes an indication of at least a portion of the image plane.
- Clause 98 The non-transitory computer-readable medium of clause 97, wherein the indication of at least a portion of the image plane comprises a box representative of a portion of the image plane.
- Clause 100 The non-transitory computer-readable medium of clause 85, wherein the first trajectory indicator corresponds to an axis that intersects with the first axis at a distal portion of the first limb member and is parallel to the joint member axis and the drive path.
- Clause 103 The non-transitory computer-readable medium of any of clauses 82-102, wherein the computer system is further configured to determine a cutting path associated with the first medical device and cause the display to concurrently display a perspective view of the cutting path.
- Clause 104 The non-transitory computer-readable medium of clause 103, wherein the cutting path corresponds to a trajectory of a knife of the first medical device.
- Clause 105 The non-transitory computer-readable medium of any of clauses 82-104, wherein the computer executable instructions further cause the one or more processors to:
- Clause 106 The non-transitory computer-readable medium of clause 105, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
- Clause 107 The non-transitory computer-readable medium of clause 105, wherein the at least a portion of the affected region comprises a stapling path of at least one staple of the first medical device.
- Clause 108 The non-transitory computer-readable medium of clause 105, wherein the at least a portion of the affected region comprises a cutting path of a knife of the first medical device.
- Clause 109 The non-transitory computer-readable medium of clause 105, wherein the at least a portion of the affected region comprises the intersection between the affected region and the medical image.
- Clause 110 The non-transitory computer-readable medium of any of clauses 82-109, wherein the computer executable instructions further cause the one or more processors to determine an intersection between the affected region and the medical image.
- Clause 111 The non-transitory computer-readable medium of any of clauses 82-110, wherein the computer executable instructions further cause the one or more processors to receive operating parameters corresponding to the first medical device and determine the emplacement of an affected region based at least in part on the received operating parameters.
- Clause 112. The non-transitory computer-readable medium of any of clauses 82-110, wherein the computer executable instructions further cause the one or more processors to:
- the display to concurrently display a perspective view of a 3D rendering of at least a portion of the ghost medical device on the plane-of-interest based at least in part on the determined emplacement of the ghost medical device.
- Clause 113 The non-transitory computer-readable medium of any of clauses 82-110, wherein the computer executable instructions further cause the one or more processors to:
- the display to currently display a perspective view of a 3D rendering of at least a portion of the ghost medical image on the plane-of-interest based at least in part on the determined emplacement of the ghost medical image.
- Clause 114 The non-transitory computer-readable medium of any one of clause 112 or 73, wherein the computer executable instructions further cause the one or more processors to project the display object on the plane-of-interest to display the ghost display object.
- Clause 115 The non-transitory computer-readable medium of Clause 114, wherein the computer executable instructions further cause the one or more processors to determine at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
- Clause 116 The non-transitory computer-readable medium of any of clauses 114 or 115, wherein the computer executable instructions further cause the one or more processors to determine an arc path corresponding to the display object and the plane-of-interest.
- Clause 117 The non-transitory computer-readable medium of any of clauses 114-116, wherein the computer executable instructions further cause the one or more processors to determine which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
- Clause 118 The non-transitory computer-readable medium of any of clauses 114-117, wherein the computer executable instructions further cause the one or more processors to use a virtual light source that is at least as large as the display object to be projected, wherein the light source is distal the plane-of-interest relative to the display object.
- Clause 121 The method of any of the Clauses 3-40, wherein the received first emplacement data and the received second emplacement data correspond to a first position sensor.
- Clause 122 The system of any of clauses 43-80, wherein the received first emplacement data and the received second emplacement data correspond to a first position sensor.
- Clause 123 The non-transitory computer-readable medium of any of clauses 83-120, wherein the received first emplacement data and the received second emplacement data correspond to a first position sensor.
- a method for medical instrument navigation comprising:
- a method for medical instrument navigation comprising:
- Clause 3 The method of Clause 2, further comprising:
- said determining the emplacement of the display object is based at least in part on the received first emplacement data and said determining the emplacement of the plane-of-interest is based at least in part on the received second emplacement data.
- Clause 4 The method of Clause 3, further comprising
- determining the emplacement of the plane-of-interest is based at least in part on the determined emplacement of the medical image.
- determining the emplacement of the display object is based at least in part on the determined emplacement of the medical image.
- Clause 6 The method of any of clauses 2-5, further comprising projecting the display object on the plane-of-interest to display the ghost display object.
- Clause 7 The method of any of clauses 6-6, wherein said projecting comprises determining at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
- Clause 8 The method of any of clauses 6-7, wherein said projecting comprises determining an arc path corresponding to the display object and the plane-of-interest.
- Clause 9 The method of any of clauses 6-8, wherein said projecting comprises determining which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
- Clause 10 The method of any of clauses 6-9, wherein said projecting comprises using a virtual light source that is at least as large as the display object to be projected, wherein the virtual light source is distal to the plane-of-interest relative to the display object.
- Clause 14 The method of any of clauses 2-13, wherein the display object is an affected region and wherein the affected region corresponds to a volume located between a first limb member and a second limb member of a medical device.
- Clause 15 The method of any of clauses 2-14, wherein the display object is an affected region and wherein the affected region comprises a stapling path of at least one staple of a medical device.
- Clause 16 The method of any of clauses 2-15, wherein the display object is an affected region and wherein the affected region comprises a cutting path of a knife of a medical device.
- a system comprising:
- a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
- a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
- Clause 19 The system of Clause 18, wherein the computer system is further configured to:
- said determining the emplacement of the display object is based at least in part on the received first emplacement data and said determining the emplacement of the plane-of-interest is based at least in part on the received second emplacement data.
- Clause 20 The system of Clause 19, wherein the computer system is further configured to:
- Clause 21 The system of any of clauses 18-21, wherein the computer system is further configured to project the display object on the plane-of-interest to display the ghost display object.
- Clause 22 The system of any of clauses 21-22, wherein the computer system is further configured to determine at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
- Clause 23 The system of any of clauses 21-23, wherein the computer system is further configured to determine an arc path corresponding to the display object and the plane-of-interest.
- Clause 24 The system of any of clauses 21-24, wherein the computer system is further configured to determine which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
- Clause 25 The system of any of clauses 21-25, wherein the computer system is further configured to use a virtual light source that is at least as large as the display object to be projected, wherein the virtual light source is distal to the plane-of-interest relative to the display object.
- Clause 29 The system of any of clauses 18-29, wherein the display object is an affected region and wherein the affected region corresponds to a volume located between a first limb member and a second limb member of a medical device.
- Clause 30 The system of any of clauses 18-30, wherein the display object is an affected region and wherein the affected region comprises a stapling path of at least one staple of a medical device.
- Clause 31 The system of any of clauses 18-31, wherein the display object is an affected region and wherein the affected region comprises a cutting path of a knife of a medical device.
- a non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
- a non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
- Clause 34 The non-transitory computer-readable medium of Clause 18, wherein the computer executable instructions further cause the one or more processors to:
- said determining the emplacement of the display object is based at least in part on the received first emplacement data and said determining the emplacement of the plane-of-interest is based at least in part on the received second emplacement data.
- Clause 35 The non-transitory computer-readable medium of Clause 19, wherein the computer executable instructions further cause the one or more processors to:
- Clause 36 The non-transitory computer-readable medium of Clause 19, wherein the computer executable instructions further cause the one or more processors to:
- Clause 37 The non-transitory computer-readable medium of any of clauses 18-21, wherein the computer executable instructions further cause the one or more processors to project the display object on the plane-of-interest to display the ghost display object.
- Clause 38 The non-transitory computer-readable medium of any of clauses 21-22, wherein the computer executable instructions further cause the one or more processors to determine at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
- Clause 39 The non-transitory computer-readable medium of any of clauses 21-23, wherein the computer executable instructions further cause the one or more processors to determine an arc path corresponding to the display object and the plane-of-interest.
- Clause 40 The non-transitory computer-readable medium of any of clauses 21-24, wherein the computer executable instructions further cause the one or more processors to determine which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
- Clause 41 The non-transitory computer-readable medium of any of clauses 21-25, wherein the computer executable instructions further cause the one or more processors to use a virtual light source that is at least as large as the display object to be projected, wherein the virtual light source is distal to the plane-of-interest relative to the display object.
- Clause 45 The non-transitory computer-readable medium of any of clauses 18-29, wherein the display object is an affected region and wherein the affected region corresponds to a volume located between a first limb member and a second limb member of a medical device.
- Clause 46 The non-transitory computer-readable medium of any of clauses 18-30, wherein the display object is an affected region and wherein the affected region comprises a stapling path of at least one staple of a medical device.
- Clause 47 The non-transitory computer-readable medium of any of clauses 18-31, wherein the display object is an affected region and wherein the affected region comprises a cutting path of a knife of a medical device.
- a method for medical instrument navigation comprising:
- the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
- a method for medical instrument navigation comprising:
- the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
- Clause 3 The method of Clause 2, further comprising:
- said determining the emplacement of the virtual medical device is based at least in part on the received emplacement data, wherein said determining the emplacement of the affected region is based at least in part on the received emplacement data and the operating parameters.
- Clause 4 The method of Clause 2, wherein the 3D rendering of the virtual medical includes at least the first limb member and the second limb member.
- Clause 5 The method of Clause 2, further comprising causing the one or more displays to display a 3D rendering of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device, wherein the 3D rendering of the virtual medical device includes at least a first virtual limb member corresponding to the first limb member and a second virtual limb member corresponding to the second limb member.
- Clause 6 The method of Clause 2, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
- Clause 7 The method of Clause 2, wherein the affected region comprises a stapling path of at least one staple of the medical device.
- Clause 8 The method of Clause 2, wherein the affected region comprises a cutting path of a knife of the medical device.
- a system comprising:
- a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
- a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
- Clause 11 The system of Clause 10, wherein the computer system is further configured to:
- Clause 12 The system of any of clauses 10-11, wherein the 3D rendering of the virtual medical includes at least the first limb member and the second limb member.
- Clause 13 The system of any of clauses 10-12, wherein the computer system is further configured to cause the display to display a 3D rendering of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device, wherein the 3D rendering of the virtual medical device includes at least a first virtual limb member corresponding to the first limb member and a second virtual limb member corresponding to the second limb member.
- Clause 14 The system of any of clauses 10-13, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
- Clause 15 The system of any of clauses 10-14, wherein the affected region comprises a stapling path of at least one staple of the medical device.
- Clause 16 The system of any of clauses 10-15, wherein the affected region comprises a cutting path of a knife of the medical device.
- a non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
- a non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
- the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis,
- Clause 19 The non-transitory computer-readable medium of Clause 18, wherein the computer executable instructions further cause the one or more processors to:
- Clause 20 The non-transitory computer-readable medium of any of clauses 18-19, wherein the 3D rendering of the virtual medical includes at least the first limb member and the second limb member.
- Clause 21 The non-transitory computer-readable medium of any of clauses 18-20, wherein the computer executable instructions further cause the one or more processors to cause the display to display a 3D rendering of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device, wherein the 3D rendering of the virtual medical device includes at least a first virtual limb member corresponding to the first limb member and a second virtual limb member corresponding to the second limb member.
- Clause 22 The non-transitory computer-readable medium of any of clauses 18-21, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
- Clause 23 The non-transitory computer-readable medium of any of clauses 18-22, wherein the affected region comprises a stapling path of at least one staple of the medical device.
- Clause 24 The non-transitory computer-readable medium of any of clauses 18-23, wherein the affected region comprises a cutting path of a knife of the medical device.
- a method for medical instrument navigation comprising: receiving emplacement data associated with a first medical device, the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
- a method for medical instrument navigation comprising:
- Clause 3 The method of Clause 2, wherein the display object is a first virtual medical device corresponding to a first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis.
- Clause 4 The method of Clause 3, further comprising:
- Clause 6 The method of Clause 5, the 3D rendering of at least a portion of the first virtual medical device includes at least a first limb member and a second limb member of the virtual medical device.
- a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
- a system comprising:
- a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
- Clause 10 The system of Clause 9, wherein the display object is a first virtual medical device corresponding to a first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis.
- Clause 11 The system of Clause 10, wherein the computer system is further configured to:
- Clause 12 The system of any of clauses 9-11, wherein the display object is a first virtual medical device and the computer system is further configured to cause the display to concurrently display a 3D rendering of at least a portion of the first virtual medical device.
- Clause 13 The system of Clause 12, the 3D rendering of at least a portion of the first virtual medical device includes at least a first limb member and a second limb member of the virtual medical device.
- Clause 14 The system of any of clauses 9-14, where the computer system is further configured to cause the display to concurrently display:
- a non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
- a non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
- Clause 17 The system of Clause 16, wherein the display object is a first virtual medical device corresponding to a first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis.
- Clause 18 The non-transitory computer-readable medium of Clause 17, wherein the computer executable instructions further cause the one or more processors to:
- Clause 19 The non-transitory computer-readable medium of any of clauses 16-18, wherein the display object is a first virtual medical device and wherein the computer executable instructions further cause the one or more processors to cause the display to concurrently display a 3D rendering of at least a portion of the first virtual medical device.
- Clause 20 The non-transitory computer-readable medium of Clause 19, the 3D rendering of at least a portion of the first virtual medical device includes at least a first limb member and a second limb member of the virtual medical device.
- Clause 21 The non-transitory computer-readable medium of any of clauses 16-20, wherein the computer executable instructions further cause the one or more processors to cause the display to concurrently display:
- Conditional language such as, among others, “can,” “could,” “might,” or “may,” unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements, and/or steps. Thus, such conditional language is not generally intended to imply that features, elements and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether these features, elements and/or steps are included or are to be performed in any particular embodiment.
- the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to.”
- the terms “connected,” “coupled,” or any variant thereof means any connection or coupling, either direct or indirect, between two or more elements; the coupling or connection between the elements can be physical, logical, or a combination thereof.
- the words “herein,” “above,” “below,” and words of similar import when used in this application, refer to this application as a whole and not to any particular portions of this application.
- words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively.
- the word “or” in reference to a list of two or more items covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list.
- the term “and/or” in reference to a list of two or more items covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list.
- operations, acts, events, or functions of any of the algorithms described herein can be performed in a different sequence, can be added, merged, or left out altogether (non-limiting example: not all are necessary for the practice of the algorithms).
- operations, acts, functions, or events can be performed concurrently, non-limiting examples: through multi-threaded processing, interrupt processing, or multiple processors or processor cores or on other parallel architectures, rather than sequentially.
- processor device can be a microprocessor, but in the alternative, the processor device can be a controller, microcontroller, or combinations of the same, or the like.
- a processor device can include electrical circuitry configured to process computer-executable instructions.
- a processor device includes an FPGA or other programmable device that performs logic operations without processing computer-executable instructions.
- a processor device can also be implemented as a combination of computing devices, non-limiting examples: a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
- a processor device may also include primarily analog components. For example, some or all of the signal processing algorithms described herein may be implemented in analog circuitry or mixed analog and digital circuitry.
- a computing environment can include any type of computer system, including, but not limited to, a computer system based on a microprocessor, a mainframe computer, a digital signal processor, a portable computing device, a device controller, or a computational engine within an appliance, to name a few.
- a software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of a non-transitory computer-readable storage medium.
- An exemplary storage medium can be coupled to the processor device such that the processor device can read information from, and write information to, the storage medium.
- the storage medium can be integral to the processor device.
- the processor device and the storage medium can reside in an ASIC.
- the ASIC can reside in a user terminal.
- the processor device and the storage medium can reside as discrete components in a user terminal.
- processing of the various components of the illustrated systems can be distributed across multiple machines, networks, and other computing resources.
- two or more components of a system can be combined into fewer components.
- Various components of the illustrated systems can be implemented in one or more virtual machines, rather than in dedicated computer hardware systems and/or computing devices.
- Virtualization technologies allow a single physical computing device to host one or more instances of a virtual machine, which virtual machine instance appears to a user as an independent computing device.
- the host computing device can create, maintain, delete, or otherwise manage virtual machines instances in a dynamic manner.
- users can request computing resources, including single computing devices or a configuration of networked computing devices, and be provided with virtual machine instances that provide the requested computing resources.
- An instance of a virtual machine may be configured to provide specific functionality.
- a virtual machine instance may be associated with different combinations of software applications and operating systems or operating system configurations to enable a virtual machine to provide different desired functionalities, or to provide similar functionalities more efficiently.
- Disjunctive language such as the phrase “at least one of X, Y, or Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., may be either X, Y, or Z, or any combination thereof (non-limiting examples: X, Y, and/or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, or at least one of Z to each be present.
- a device configured to are intended to include one or more recited devices. Such one or more recited devices can also be collectively configured to carry out the stated recitations.
- a processor configured to carry out recitations A, B and C can include a first processor configured to carry out recitation A working in conjunction with a second processor configured to carry out recitations B and C.
Landscapes
- Health & Medical Sciences (AREA)
- Surgery (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Animal Behavior & Ethology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Pathology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Robotics (AREA)
- Radiology & Medical Imaging (AREA)
- Gynecology & Obstetrics (AREA)
- Orthopedic Medicine & Surgery (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
A system and method for providing image guidance for placement of one or more medical devices at a target location. The system can determine one or more intersections between a medical device and an image region based at least in part on first emplacement data and second emplacement data. Using the determined intersections, the system can cause one or more displays to display perspective views of image guidance cues, including an intersection ghost in a virtual 3D space.
Description
The present application claims priority benefit to U.S. Provisional Application No. 62/413,925, which is hereby incorporated herein by reference in its entirety. Any and all applications for which a foreign or domestic priority claim is identified in the Application Data Sheet as filed with the present application are incorporated by reference under 37 CFR 1.57 and made a part of this specification.
Various systems are available to aid a healthcare provider to guide a medical device in a patient or to provide a user viewing an object with additional information. The systems can provide image guidance cues to aid the healthcare provider or user and can also provide additional information for the user's benefit.
Implementations disclosed herein provide systems, methods, and apparatus for displaying medical images, such as, but not limited to ultrasound, CT, and/or MRI images, facilitating medical device insertion into tissue by an operator. Certain embodiments pertain to a free-hand medical device guidance system. The system can provide the healthcare provider manual control over the medical device, while making the spatial relationships between the target, medical device and medical image (also referred to as an image slice or rendered medical image), or image area corresponding to the medical image (also referred to as an image slice area or scan area), more intuitive via a visual display. Using this visual feedback, the operator can adjust the medical device's position, orientation, or trajectory. Certain of the contemplated embodiments can be used in conjunction with systems described in greater detail in U.S. patent application Ser. No. 13/014,587, filed Jan. 26, 2011, entitled SYSTEMS, METHODS, APPARATUSES, AND COMPUTER-READABLE MEDIA FOR IMAGE MANAGEMENT IN IMAGE-GUIDED MEDICAL PROCEDURES and U.S. patent application Ser. No. 13/753,274, filed Jan. 29, 2013, entitled MULTIPLE MEDICAL DEVICE GUIDANCE (the '274 Application), U.S. patent application Ser. No. 14/212,933, filed Mar. 14, 2014, entitled MEDICAL DEVICE GUIDANCE, U.S. patent application Ser. No. 15/199,630, filed Jun. 30, 2016, entitled LOUPE DISPLAY, and U.S. patent application Ser. No. 14/872,930, filed Oct. 1, 2015, entitled AFFECTED REGION DISPLAY (the '930 Application), each of which is hereby incorporated by reference in its entirety.
Medical interventions typically involve using an instrument to resect, cauterize, staple, seal, or otherwise manipulate soft tissue and organs. A physician must take great care to minimize blood loss and minimize damage to ancillary tissue while performing these tissue-damaging interventions. This is even more difficult with minimally-invasive surgeries, such as laparoscopic, endoscopic, and robotic surgeries. A physician may use ultrasound to image the internal structures of an organ before stapling, transecting, resecting, sealing, or grasping tissue, helping her avoid critical structures such as blood vessels. However, even with ultrasound imaging, there is a significant possibility of inadvertent damage to surrounding tissue and blood vessels during these procedures. This is because it is not obvious in the externally displayed medical image where a given internal structure is located relative to the medical device.
The system can aid the healthcare provider in guiding one or more medical devices through or around tissue of the patient and/or placing the medical devices. The system can be used to aid in stapling, transecting, resecting, sealing, and/or grasping tissue. Additionally, the system can be used for treatment of tumors, fibroids, cysts, damaged blood vessels, or other damages internal structures of a patient. The system can be used during open surgery, laparoscopic surgery, endoscopic procedures, robotic surgeries, biopsies, and/or interventional radiology procedures.
The system can be used in conjunction with live intraoperative ultrasound (U/S), pre-operative CT, or any cross-sectional medical imaging modality (e.g. MRI, OCT, etc.). In addition, the system can use a variety of techniques to determine the position and/or orientation of one or more medical devices. For example, the system can use the NDI Aurora magnetic system, NDI Polaris optical system, etc. In some embodiments, a position sensor can be embedded inside, or affixed to each medical device, at the tip, along the shaft, and/or on the handle. Sensors can be built into the medical devices or attached after manufacturing, as described in greater detail in U.S. application Ser. No. 14/212,184, filed Mar. 14, 2014, entitled SENSOR MOUNT, incorporated herein in its entirety.
Each medical device can be associated with one or more sensors, which can continually, or repeatedly, report position and/or orientation, or a single sensor can be used for all the medical devices. In some embodiments where one sensor is used, the healthcare provider can attach the sensor to the particular medical device that she is intentionally repositioning, and then, once she has placed that medical device, she can remove the sensor and attach it to the next medical device she is repositioning. In some embodiments, the medical devices can be manipulated by the healthcare provider. In certain embodiments, the system can be used with a robotic manipulator, where the robot controls the medical devices. In some embodiments, visually-detectable fiducials can be used to determine position and/or orientation for one or more of the medical devices.
In some embodiments, the handles of medical devices can have push-button switches, to allow the user to select a medical device, indicate a tissue target, etc. The handle can also have an indicator light to indicate to the users which medical device is selected. Finally, the handle can have an encoder to detect how much length of electrode has been exposed by the user, and report this information to the guidance system and therapeutic generator.
Image Guidance Systems
Although only two medical devices 145 and 155 are shown in FIG. 1 , it will be understood that additional medical devices can be tracked and associated data can be provided to the image guidance unit 130. The image guidance unit 130 can process or combine the data and show image guidance data on display 120. This image guidance data can be used by a healthcare provider to guide a procedure and improve care. There are numerous other possible embodiments of system 100. For example, many of the depicted components can be joined together to form a single component and can be implemented in a single computer or machine. Further, additional position sensing units can be used in conjunction with position sensing unit 140 to track all relevant medical devices 145 and 155, as discussed in more detail below. Additional imaging units 150 can be included, and combined imaging data from the multiple imaging units 150 can be processed by image guidance unit 130 and shown on display unit 120. Additionally, two or more surgical systems 149 can also be included.
Information about and from multiple surgical systems 149 and attached medical devices 145 (and additional medical devices not shown) can be processed by image guidance unit 130 and shown on display 120. These and other possible embodiments are discussed in more detail below. It will be understood that any combination of the display objects (such as, but not limited to medical display objects (e.g., virtual medical device, medical image, ghost display objects, etc.) and/or image guidance cues (e.g., trajectory indicators, intersection indicators, plane indicators, ghost display objects, etc.)) described herein can be displayed concurrently, or simultaneously. Further, reference to displaying objects “concurrently” and/or “simultaneously” is to be interpreted broadly and may refer to displaying objects in such a way that to a human observer the objects are visible at the same time.
In some embodiments, system 100 comprises a display unit 120 and a position sensing unit 140 communicatively coupled to image guidance unit 130. In some embodiments, position sensing unit 140, display unit 120, and image guidance unit 130 are coupled to the stand 170. Image guidance unit 130 can be used to produce images 125 that are displayed on display unit 120. The images 125 produced on display unit 120 by the image guidance unit 130 can be determined based on ultrasound or other visual images from the first medical device 145 and second medical device 155. In the illustrated embodiment, the images 125 include a 2D viewing area and a 3D viewing area. In the 2D viewing area, some or all of the display objects can be displayed as 2D objects. For instance, the 2D viewing area can include a 2D view of each of an ultrasound slice, a virtual medical device corresponding to the first medical device, a virtual imaging device corresponding to the second medical device, intersection indicator, trajectory indicators and/or other image guidance cues. It will be understood that some or all of the display objects in the 2D viewing area can be displayed as 3D objects. In the illustrated embodiment, the 3D viewing area includes perspective views of each of the medical image 121, the virtual medical device 122, a displayed affected region 129, the virtual imaging device 123, intersection indicator 126, trajectory and other image guidance cues 127A, 127B (generically referred as image guidance cues 127 and/or trajectory indicators 127), and a patient orientation indicator 128. It will be understood that any combination of the aforementioned display objects can be displayed in the 2D view and/or 3D view as desired.
As a non-limiting example, if the first medical device 145 is a stapler 145 having a joint member and two limb members coupled to the joint member at different angles and the second medical device 155 is an ultrasound probe 155, then images 125 produced on display 120 can include the images, or video, from the ultrasound probe 155 (e.g., medical image 121) combined with other medical display objects and image guidance cues, such as display object trajectories (e.g., trajectory indicators 127) or affected region 129 (e.g., trajectory of area corresponding to a volume located between the first limb member and the second limb member), determined based on the emplacement of the stapler 145. If the first medical device 145 is an ultrasound probe 145 and the second medical device 155 is a laparoscopic camera 155, then images 125 produced on display 120 can include the video from the laparoscopic camera 155 combined with ultrasound data superimposed on the laparoscopic image. More medical devices can be added to the system. For example, the system can include a stapler, ultrasound probe, ablation needle, laparoscopic camera, scalpel, grasper, vessel sealer, electrocauterizer, resecting device, transecting device and/or any other surgical instrument or medical device. The system can also process and/or display collected data, such as preoperative CT scans, X-Rays, MRIs, laser scanned 3D surfaces etc.
The term “emplacement” as used herein is a broad term and may refer to, without limitation, position and/or orientation or any other appropriate location information. The term “pose” as used herein is a broad term encompassing its plain and ordinary meaning and may refer to, without limitation, position and orientation or any other appropriate location information. In some embodiments, the imaging data obtained from one or both of medical devices 145 and 155 can include other modalities such as a CT scan, MRI, open-magnet MRI, optical coherence tomography (“OCT”), positron emission tomography (“PET”) scans, fluoroscopy, ultrasound, or other preoperative, or intraoperative 2D or 3D anatomical imaging data. In some embodiments, medical devices 145 and 155 can also be scalpels, implantable hardware, or any other device used in surgery. Any appropriate surgical system 149 or imaging unit 150 can be attached to the corresponding medical instruments 145 and 155.
As noted above, images 125 produced can also be generated based on live, intraoperative, or real-time data obtained using the second medical device 155, which is coupled to second imaging unit 150. The term “real time” as used herein is a broad term and has its ordinary and customary meaning, including without limitation instantaneously or nearly instantaneously. The use of the term real time can also mean that actions are performed or data is obtained with the intention to be used immediately, upon the next cycle of a system or control loop, or any other appropriate meaning. Additionally, as used herein, real-time data can be data that is obtained at a frequency that would allow a healthcare provider to meaningfully interact with the data during surgery. For example, in some embodiments, real-time data can be a medical image of a patient that is updated one time per second. In some embodiments, real-time data can be ultrasound data that is updated multiple times per second.
The medical devices 145, 155 can be communicatively coupled to the position sensing unit 140 (e.g., sensors embedded or coupled to the medical devices 145, 155 can be communicatively coupled with the position sensing unit 140). The position sensing unit 140 can be part of imaging unit 150 or it can be separate. The position sensing unit 140 can be used to determine the emplacement of first medical device 145 and/or the second medical device 155. In some embodiments, the position sensing unit 140 can include a magnetic tracker and/or one or more magnetic coils can be coupled to medical devices 145 and/or 155. In some embodiments, the position sensing unit 140 can include an optical tracker and/or one or more visually-detectable fiducials can be coupled to medical devices 145 and/or 155. In some embodiments, the position sensing unit 140 can be located below the patient. In such embodiments, the position sensing unit 140 can be located on or below the table 180. For example, in embodiments where the position sensing unit 140 is a magnetic tracker, it can be mounted below the surgical table 180. Such an arrangement can be useful when the tracking volume of the position sensing unit 140 is dependent on the location of the position sensing unit 140, as with many magnetic trackers. In some embodiments, magnetic tracking coils can be mounted in or on the medical devices 145 and 155.
In some embodiments, the position sensing unit 140 can be an electromagnetic measurement system (e.g., NDI Aurora system) using sensor coils for tracking units attached to the first and/or second surgical devices 145 and 155. In some embodiments, the second position sensing unit 140 can be an optical 3D tracking system using fiducials. Such optical 3D tracking systems can include the NDI Polaris Spectra, Vicra, Certus, PhaseSpace IMPULSE, Vicon MX, InterSense IS-900, NaturalPoint OptiTrack, Polhemus FastTrak, IsoTrak, or Claron MicronTracker2. In some embodiments, the position sensing unit 140 can each be an inertial 3D tracking system comprising a compass, accelerometer, tilt sensor, and/or gyro, such as the InterSense InertiaCube or the Nintendo Wii controller. In some embodiments, the position sensing unit 140 can be attached to or affixed on the corresponding surgical device 145 and 155.
In some embodiments, the position sensing units 140, can include sensing devices such as the HiBall tracking system, a GPS device, or signal emitting device that would allow for tracking of the position and/or orientation (e.g., emplacement) of the tracking unit (also referred to as an emplacement sensor). In some embodiments, a position sensing unit 140 can be affixed to either or both of the surgical devices 145 and 155. The surgical devices 145 or 155 can be tracked by the position sensing unit 140. A room coordinate system reference, such as the display 120 can also be tracked by the position sensing unit 140 in order to determine the emplacements of the surgical devices 145 and 155 with respect to the room coordinate system. Devices 145 and 155 can also include or have coupled thereto one or more accelerometers, which can be used to estimate movement, position, and location of the devices.
In some embodiments, the position sensing unit 140 can be an Ascension Flock of Birds, Nest of Birds, driveBAY, medSAFE, trakSTAR, miniBIRD, MotionSTAR, pciBIRD, or Calypso 2D Localization System and tracking units attached to the first and/or second medical devices 145 and 155 can be magnetic tracking coils.
The term “tracking unit” (also referred to as an emplacement sensor), as used herein, is a broad term encompassing its plain and ordinary meaning and includes without limitation all types of magnetic coils or other magnetic field sensing devices for use with magnetic trackers, fiducials or other optically detectable markers for use with optical trackers, such as those discussed above and below. In some embodiments, the tracking units can be implemented using optical position sensing devices, such as the HiBall tracking system and the position sensing unit 140 can form part of the HiBall tracking system. Tracking units can also include a GPS device or signal emitting device that allows for tracking of the position and/or orientation of the tracking unit. In some embodiments, a signal emitting device might include a radio-frequency identifier (RFID). In such embodiments, the position sensing unit 140 can use the GPS coordinates of the tracking units or can, for example, triangulate the radio frequency signal being emitted by the RFID associated with tracking units. The tracking systems can also include one or more 3D mice.
In some embodiments, the display unit 120 displays 3D images to a user, such as a healthcare provider. Stereoscopic 3D displays separate the imagery shown to each of the user's eyes. This can be accomplished by a stereoscopic display, a lenticular auto-stereoscopic display, or any other appropriate type of display. The display 120 can be an alternating row or alternating column display. Example alternating row displays include the Miracube G240S, as well as Zalman Trimon Monitors. Alternating column displays include devices manufactured by Sharp, as well as many “auto-stereoscopic” displays (e.g., Philips). In some embodiments, Sony Panasonic 3D passive displays and LG, Samsung, and/or Vizio 3D TVs can be used as well. Display 120 can also be a cathode ray tube. Cathode Ray Tube (CRT) based devices, can use temporal sequencing, showing imagery for the left and right eye in temporal sequential alternation. This method can also be used for projection-based devices, as well as by liquid crystal display (LCD) devices, light emitting diode (LED) devices, and/or organic LED (OLED) devices.
In certain embodiments, the display unit 120 can be a head mounted display worn by the user in order to receive 3D images from the image guidance unit 130. In such embodiments, a separate display, such as the pictured display unit 120, can be omitted. The 3D graphics can be produced using underlying data models, stored in the image guidance unit 130 and projected onto one or more 2D planes in order to create left and right eye images for a head mount, lenticular, or other 3D display. The underlying 3D model can be updated based on the relative emplacements of the various devices 145 and 155, as determined by the position sensing unit(s) 140, and/or based on new data associated with the devices 145 and 155. For example, if the second medical device 155 is an ultrasound probe, then the underlying data model can be updated to reflect the most recent medical image. If the first medical device 145 is a stapler, then the underlying model can be updated to reflect any changes related to the jaws, such as information regarding the affected region or angles of the jaws and/or transecting knife. Any appropriate 3D graphics processing can be used for rendering including processing based on OpenGL, Direct3D, Java 3D, etc. Whole, partial, or modified 3D graphics packages can also be used, such packages including 3DS Max, SolidWorks, Maya, Form Z, Cybermotion 3D, VTK, Slicer, or any others. In some embodiments, various parts of the needed rendering can occur on traditional or specialized graphics hardware. The rendering can also occur on the general CPU, on programmable hardware, on a separate processor, be distributed over multiple processors, over multiple dedicated graphics cards, or using any other appropriate combination of hardware or technique.
One or more components, units, devices, or elements of various embodiments can be packaged and/or distributed as part of a kit. For example, in one embodiment, a stapler, one or more tracking units, 3D viewing glasses, and/or a portion of an ultrasound wand can form a kit. Other embodiments can have different elements or combinations of elements grouped and/or packaged together. Kits can be sold or distributed separately from or with the other portions of the system.
One will readily recognize that there are numerous other examples of image guidance systems which can use, incorporate, support, or provide for the techniques, methods, processes, and systems described herein.
Coordinate Systems
A position sensing coordinate system 180 can be used to determine the emplacement of various objects within a position sensing region. For example, in some embodiments, the position sensing coordinate system 180 can refer to the coordinate system 180A used by a magnetic tracker (not shown) tracking objects within a magnetic field volume, or to the coordinate system 180B used by an optical tracker 196 tracking objects within a volume. In some cases, multiple position sensing coordinate systems 180 can be used together. For example, a magnetic position sensing coordinate system 180A can be used in conjunction with a magnetic tracker tracking sensor coils within a position sensing region that are coupled to medical devices and an optical position sensing coordinate system 180B can be used in conjunction with an optical tracker tracking a fiducial coupled to a head mounted display (HMD) or a user, or to an optical tracker analyzing an image captured by an image sensor. It will be understood that any combination of the tracker systems and position sensing coordinate systems 180 can be used as desired. For example, in some cases, a single position sensing coordinate system 180 can be used to track tracking sensors associated with one or more medical devices and/or tracking sensors associated with a user or HMD. Similarly, a different position sensing coordinate system 180 can be used for each tracking sensor, or any combination thereof.
Room coordinate systems 182 can be used to determine the emplacement of objects within a room, such as an operating room. For example, the room coordinate system 182 can be used to determine or identify the relative emplacement of the position sensing unit, medical devices, tracking sensors, user, display, etc. relative to each other within a room.
A 3D scene coordinate system 184, which may also be referred to as a 3D volume or scene graph coordinate system, can be used to determine the emplacement of display objects within a virtual 3D scene. In some cases, the 3D scene coordinate system 184 can identify the relative emplacement of virtual objects within the 3D scene. In certain embodiments, the virtual objects can correspond to real objects, such as to medical devices 145, 155 and/or to computer-generated objects, such as such as trajectory cues 127, affected region 129, etc. Additionally, in certain embodiments, display objects can correspond to real objects, virtual objects, and/or computer generated objects.
A display coordinate system 186 can be used to determine the emplacement of display objects for display on the display 102. For example, the display coordinate system 186 can be used to determine the emplacement of virtual medical devices, medical images, image guidance cues, and the like, within a display 102. In some embodiments, the display coordinate system 186 can be used to determine how the objects within the 3D scene are to be displayed on the display. For example, the display coordinate system 186 can be used to determine the point-of-view location, or eye point, relative to the 3D scene (or 3D volume coordinate system 184) or scene graph for viewing the contents of the 3D scene. As mentioned above, multiple display coordinate systems 186 can be used. For example, left-eye, right-eye, and/or center-eye display coordinate systems 186 can be used to display different perspective of the display objects within a 3D scene, such as when a 3D display and/or an HMD is being used.
A medical image coordinate system 194 can be used in conjunction with medical images used and/or processed by the system. As described previously, the medical images can be ultrasound images, CT image, MRI, images, etc. The images can be different sizes or shapes. For example, one ultrasound can output an image having one size and shape while a different ultrasound can output an image having a different size and/or shape. Similarly, CT, MRI, and ultrasound images may have different sizes and shapes. Accordingly, the medical image coordinate system can be used to identify the particular size and shape of the medical image being used and/or processed by the system 100.
It will be understood that fewer, more, or different coordinate systems can be used as desired. For example, in some embodiments, the 3D scene coordinate system 184 can be omitted or combined with display coordinate system 186 and/or the position sensing coordinate system 180. Furthermore, in some cases, one or more tracking sensor coordinate systems 188A, 188B, medical device coordinate systems 190A, 190B, 190C, 190D, virtual medical device coordinate systems 192A, 192B, 192C, 192D, or other objects etc., can have their own coordinate system. The coordinate systems for the tracking sensors, medical devices, and/or virtual medical devices can be used to identify the dimensions of the sensor/device/display object and relationship of the sensor/device/display object to another sensor/device/display object or other coordinate systems. For example, a medical device coordinate system (or virtual medical device coordinate system) can identify the dimensions of a corresponding medical device or virtual medical device, as well as the emplacement of a tracking sensor relative to the medical device (or vice versa). Similarly, a medical imaging device coordinate system can identify the dimensions of the corresponding medical imaging device (or virtual medical imaging device) and/or an emplacement of a medical image relative to the medical imaging device (non-limiting example: the emplacement of an ultrasound image relative to the corresponding ultrasound transducer), or vice versa. The system 100 can use various coordinate systems to determine the emplacement of a portion or the entire object with respect to each other and with respect to the other coordinate systems.
The system 100 can use the various coordinate systems to determine emplacement of objects relative to each other and determine how to display the display objects on a display, such as the display 102. As a non-limiting example, to display a virtual rendering of an ultrasound transducer 123 and ultrasound image 121 on the display 102, the system 100 can determine the emplacement of a magnetic tracking sensor coupled to the ultrasound transducer 155 within a magnetic position sensing coordinate system 180. Using a magnetic tracking sensor coordinate system 188B, the system 100 can determine the location of each portion of the magnetic tracking sensor within the magnetic position sensing coordinate system 180. The system 100 can also determine the emplacement of each portion the ultrasound transducer 155 within the magnetic position sensing coordinate system by mapping the ultrasound transducer coordinate system 190B to the magnetic tracking sensor coordinate system 188B (or vice versa) and/or to the magnetic position sensing coordinate system 180.
In addition, the system 100 can map each portion of the ultrasound image 121 corresponding to the ultrasound transducer 155 to the magnetic position sensing coordinate system 180 by mapping an ultrasound image coordinate system 194 to the ultrasound transducer coordinate system 190B, the magnetic tracking sensor coordinate system 188B, and/or to the magnetic position sensing coordinate system 180.
To display the virtual ultrasound transducer 123 and ultrasound image slice 121, the system 100 can map the various objects from the magnetic position sensing coordinate system 180 to a room coordinate system 182, which can identify the relative emplacement of the position sensing coordinate system 180 to a display. The system can then map data to the 3D scene coordinate system 194 and/or the display coordinate system 186. For 3D viewing, the system 100 can map the objects to multiple display coordinate systems 186, such as left-eye and/or right-eye coordinate systems.
With continued reference to the non-limiting example, the system 100 can determine an emplacement of an optical tracking sensor corresponding to a user within an optical position sensing coordinate system 180B. The emplacement of the optical tracking sensor within the optical position sensing coordinate system 180B can be mapped to the room coordinate system 182, the 3D scene coordinate system 184, and/or the display coordinate systems 186 for display. In this way the system 100 can determine the emplacement of the ultrasound transducer 155 and ultrasound image slice 121 relative to the user and display a virtual rendering of the ultrasound transducer 123 and ultrasound image slice 121 within the 3D scene relative to the determined emplacement of the user.
To display the virtual ultrasound transducer 123 and ultrasound image slice 121, the system 100 can map the various objects from the magnetic position sensing coordinate system 180 to a room coordinate system 182, which can identify the relative emplacement of the position sensing coordinate system 180 to a display. The system can then map data to the 3D scene coordinate system 194 and/or the display coordinate system 186. For 3D viewing, the system 100 can map the objects to multiple display coordinate systems 186, such as left-eye and/or right-eye coordinate systems.
Although the non-limiting examples have been described as mapping the various objects and coordinate systems, to a position sensing coordinate system 180, the room coordinate system 182, the 3D scene coordinate system 184, and to display coordinate systems 186, it will be understood that the one or more of the objects or coordinate systems can be mapped directly or indirectly to any other coordinate system. For example, the medical device image can be mapped directly to a left-eye display coordinate system 186, etc. Thus, any of the real or virtual objects described herein may be represented, detected, or imaged in any coordinate system, and conversion between the various coordinate systems can be performed in components of the system such as image guidance unit 130, position sensing unit 140, imager 150, the an HMD, or other components.
Furthermore, it will be understood that once the system 100 determines an emplacement of a medical device in one coordinate system, such as a position sensing coordinate system 180, the system 100 can determine the emplacement of a corresponding virtual medical device in a different coordinate system, such as the 3D scene coordinate system 184 or the screen coordinate system 186, by mapping the coordinates of the first coordinate system to the coordinates of the second coordinate system, or vice versa. Accordingly, references made herein to determining an emplacement of the medical device can also refer to determining an emplacement of a virtual medical device corresponding to the medical device, or vice versa. Similarly, references made herein to determining an emplacement of a display object (non-limiting example: medical image) relative to the medical device can also refer to determining the emplacement of the display object relative to a corresponding virtual medical device.
Depicting Medical Devices
It can often be difficult to discern the content of a 3D scene from a 2D depiction of it, or even from a 3D depiction of it. Therefore, various embodiments herein provide image guidance that can help the healthcare provider better understand the scene, relative emplacements or poses of object in the scene and thereby provide improved image guidance.
Although only one virtual medical device 202 is displayed, it will be understood that multiple medical devices can be tracked and displayed concurrently, or simultaneously, on screen 220, as described in greater detail in the '274 Application, previously incorporated by reference. For example, a virtual rendering of the medical imaging device 222 can be displayed.
The virtual medical device 202 (which can also be referred to as an avatar) can be displayed in a virtual 3D space with the screen 220 acting as a window into the virtual 3D space. Thus, as the medical device 242 is moved to the right with respect to a point-of-view location (e.g., the location of the point-of-view for viewing the 3D space), the virtual medical device 202 also moves to the right. Similarly, if the medical device 242 is rotated 90 degrees so that the tip of the medical device is pointing away from the point-of-view location (e.g., at the screen 220), the virtual medical device 202 will likewise show the change in orientation, and show the tip of the virtual medical device 202 in the background and the other end of the virtual medical device 202 in the foreground. In some embodiments, as described in greater detail in U.S. patent application Ser. No. 14/212,933, filed Mar. 14, 2014, entitled MEDICAL DEVICE GUIDANCE (the '933 Application), incorporated herein by reference in its entirety, the point-of-view location can be a fixed location, such as a predetermined distance/angle from the screen 220 or stand 170 and/or a location configured by the user; or the point-of-view location can by dynamic. For example, the system can track a user in real-time and determine the point-of-view location based at least in part on the tracked location of the user.
Some models of medical devices have markings such as bands around the shaft or jaws to indicate distance along the shaft or jaws. Healthcare providers performing medical device procedures are often familiar with these markings and can use them to help understand the spatial relationship between the medical device and anatomy. In some embodiments, the make and model of the medical device 242 is known to the image guidance system and the virtual medical device 202 displayed in display 220 can resemble medical device 242. The features of medical devices that can be rendered in the scene include the overall shape (diameter, angles, cross sectional shape, curvature, etc.), color, distance markers, angle of the jaws, visuals or echogenic fiduciary markers, the state of deployable elements such as tines, paddles, anchors, resection loops, stiffening or steerable sleeves, temperature, radiation, light or magnetic field sensors, lens, waveguides, fluid transfer channels, and the like.
The type of medical device being used can be input into the image guidance system 100, can be a system default, can be detected by a camera or other device, can be received as data from an attached medical device, such as surgical system 149 in FIG. 1 , or the information can be received in any other appropriate manner. Displaying on display 220, a virtual medical device 202 that resembles the medical device 242 can help healthcare providers associate the image guidance data with the real world and can provide more familiar guidance information to a healthcare provider, thereby further aiding the healthcare provider in the guidance task. For example, the healthcare provider can see the familiar markings on the medical device 202 being displayed on the display 220 and therefore be familiar with the distance and relative placement of the displayed medical device 202 with respect to other data, such as tissue 212 seen in a medical image 204, 205. This knowledge of relative placement of items being displayed can help the healthcare provider move the medical device 242 into place. It should be understood that the terms medical device and virtual medical device can sometimes be used interchangeably as they can generally relate to the same object, the medical device being the object in the real world and virtual medical device being a representation of the object, such as an avatar, in virtual space. Additionally, in some instances virtual medical device can sometimes be referred to as virtual surgical instrument, surgical instrument, rendered surgical instrument, rendered medical device, avatar, rendered avatar, and virtual avatar.
Consider an embodiment in which the virtual medical device 202 in the display 220 is a virtual stapler depicting the portion of the stapler that will perform the stapling. The displayed virtual medical device can include a joint member 211, a first limb member 213, and a second limb member 210. If the display 220 also includes ultrasound data, then the doctor can find the tissue she wishes to staple by moving the ultrasound probe 222 until she identifies the target tissue. In various embodiments, she will be able to see the displayed ultrasound data and its location relative to the displayed medical device 202. She can then direct the medical device 242 until she sees, on display 220, that the affected region of the virtual medical device 202 encompasses a desired stapled region of the tissue in the medical image. When she activates the stapler 242, she can have a higher degree of confidence that she has stapled the correct portion of the tissue. Various embodiments of this are discussed below.
As another example, consider the physical markings that can be on the instruments themselves. These markings can help orient a healthcare provider during use of the instrument. In some embodiments, the image guidance unit can represent these markings in the images displayed in the display. For example, certain ultrasound transducers are built with an orientation mark (e.g., a small bump) on one side of the transducing array. That mark can also be shown in the medical image on the scanner's display, to help the healthcare provider understand where the scanned anatomical structures shown on screen are located under the transducer, inside the patient.
In some embodiments, the image guidance system can display a symbolic 3D representation of an orientation mark 214 both next to the motion-tracked ultrasound slice (e.g., moving with the displayed ultrasound slice) and next to the 2D view of the ultrasound slice also displayed by the system. An example of this orientation mark is displayed in FIG. 2 , where a small rectilinear volume 214 is shown both in proximity to the ultrasound slice displayed in the 3D view and the ultrasound slice displayed in a 2D view. In some embodiments, the orientation mark 214 corresponds to a feature, such as a physical marking, of the ultrasound probe. In some embodiments, the orientation mark 214 is displayed to provide assistance in associating the 3D view and the 2D view.
It will be understood that a medical image can correspond to image data received from an imaging device, such as an ultrasound transducer. In some embodiments, the image data can correspond to a cross-section of tissue having a certain thickness. In some instances, the imaging device can compact the image data, and/or treat the image data as 2D data, such that there is no perceived thickness. In certain embodiments, when the medical image is displayed in a 3D view, the system can treat the medical image as a 2D or quasi 2D object. In such embodiments, the system can cause the medical image to have little to no perceptible thickness. Accordingly, in certain embodiments, when the medical image is oriented orthogonally or perpendicularly with respect to the point-of-view location, the system can cause the display to display nothing or a line having a relatively small thickness, such as a few pixels, etc. In some cases, the number of pixels used to display the relatively small thickness of the medical image can correspond to the size of the display. For example, more pixels can be used for a larger display and fewer pixels can be used for a smaller display, etc.
Other embodiments can track and display other types of instruments and their features. For example, a healthcare provider may want to track one or more of a stapler, vessel sealer, grasper, scalpel, a biopsy, a cauterizer (including an electrocauterizer and Bovies), forceps, cutting loops on hysteroscopes, harmonic sheers, lasers (including CO2 lasers), etc. For example, in various embodiments, the following devices can be tracked and various aspects of their design displayed on display 220: Olympus™ OES Pro Hystero-Resectoscope, SonoSurg Ultrasonic Surgical System Olympus™ GF-UC 160 Endoscope Wallus™ Embryo Transfer Catheter AngioDynamics® NanoKnife™, VenaCure™ laser, StarBurst, Uniblade, Habib® Resector Bovie™ Electrodes, Covidien Evident™, Cool-Tip™ Ablation Antennas, Opti4™ Electrodes Microsulis MEA (microwave endometrial ablation), Acculis Halt™ Medical System Optimed BigLumen Aspiration Catheter Optimed Optipure Stent Central venous catheterization introducer medical device (such as those made by Bard and Arrow).
Once tracked, a healthcare provider is able to see image guidance data on display 220 that will allow her to know the relative pose, location, or emplacement of the tracked instrument(s) with respect to one another or with respect to imaging data and will be able to see, on display 220, the features of the instrument rendered in the scene.
With continued reference to FIG. 2 , in some embodiments, the image guidance system can constantly display an additional 2D view 205 of the medical image, simultaneous to the 3D depiction 204, so that the medical image is always visible, regardless of the emplacement in which the healthcare provider holds the transducer 222. The 2D view 205 of the medical data can be similar to what a healthcare provider is accustomed to seeing with traditional medical displays, such as ultrasound displays. This can be useful to provide the healthcare provider with imaging to which she is accustomed and allows a healthcare provider to see the medical data regardless of the then-current emplacement of the imaging device with respect to the user.
In some embodiments, the 2D view 205 of the medical image is depicted in the upper right corner of the monitor (though it can be placed in any location). In some embodiments, the guidance system can automatically (and continually) choose a corner in which to render the 2D view 205 of the medical image, based on the 3D position of the medical devices in the rendered scene. For example, in FIG. 2 , stapler 242 can be held in the healthcare provider's left hand and the medical device shaft is to the left of the 3D view of the medical image, so that the 2D view 202 of the medical image in the upper right corner of display 220 does not cover any of the 3D features of the medical device (or vice-versa). If the medical device were held in the healthcare provider's right hand, the virtual medical device shaft would appear on the right side. To prevent the 2D view 205 in the corner of display 220 from covering the medical device shaft, the system can automatically move it to a corner that would not otherwise be occupied by graphics or data.
In some embodiments, the system attempts to avoid having the 2D view 205 of the medical image quickly moving among corners of the display in order to avoid overlapping with graphics and data in the display. For example, a function f can be used to determine which corner is most suitable for the 2D medical image to be drawn in. The inputs to f can include the locations, in the screen coordinate system, of the displayed medical device tip, the corners of the 3D view of the medical image, etc. In some embodiments, f s output for any given point in time is independent of f s output in the previous frames, which can cause the medical image to move among corners of the display rapidly. In some embodiments, the image guidance system will filter f's output over time. For example, the output of a filter g, for any given frame, could be the corner, which has been output by f the most number of times over the last n frames, possibly weighting the most recent values for f most heavily. The output of the filter g can be used to determine in which corner of display 220 to display the 2D medical image and the temporal filtering provided by g can allow the 2D view 205 of the medical image display to move more smoothly among the corners of the display 220.
In some embodiments, other appropriate virtual information and/or image guidance cues can be overlaid on the 2D view 205 of the medical image as well as the 3D view 204. Examples include: orientation indicator 214, an indication of the distance between the medical device's tip (for example, the tip of the first limb member or the tip of the second limb member) and the point in the plane of the medical image that is closest to the medical device tip; ghost affected area (e.g., a cross section or outline of the affected region projected on the ultrasound slice); and/or intersection indicators (e.g., a point, box, outline, etc.) indicating an intersection between one or more axes or trajectories of a display object and a plane-of-interest (e.g., a medical image plane, a medical device plane, etc.).
Furthermore, it will be understood that other image guidance cues can be generated and displayed on the display as described in greater detail in the '274 Application, previously incorporated herein by reference. For example, the system 100 can generate and/or display graphical indicators that help indicate the spatial relationship between a medical device and an medical image plane (e.g., graphical image plane indicators) or other plane (e.g., graphical plane indicators), indicators to indicate the relative positions of the medical device(s) and medical image(s), features of interest, annotations, plane indicators, plane intersection indicators, other graphical indicators, approximate medical device location indicators, etc. As described in greater detail above and in the '274 Application, the various image guidance cues can be generated based at least in part on the emplacement information of the medical devices used with the system 100.
Multi-Axis Device
With reference to FIGS. 1B and 2 , one or more of the medical devices or the system 100 can be a multi-axis medical device. For instance, the multi-axis medical device 203 can be a number of devices including two or more axes or jaws, such as, but not limited to graspers, staplers, vessel sealers, electrocauterizers, resecting devices, transecting devices and scalpels. As depicted, a distal end of the multi-axis medical device 202 includes a joint member 211, and two limb members 213, 210 coupled to the joint member 211 at different angles. In some examples, the multi-axis medical device 202 can have a number of associated axes. For instance, the joint member 211 can have a joint member axis 198A, the first limb member 213 can have a first axis 198B, and the second limb member 210 can have a second axis 198C that is different from the joint member axis 198A and the first axis 198B. In some embodiments, such as the illustrated embodiment of FIG. 2 , the joint member axis 198A is the same as the first axis 198B. However, it will be understood that in some cases, the joint member axis 198A, first axis 198B, and second axis 198C can all be different. For example, the first member 213 may be angled relative to the joint member 211 and the second member 210.
In some examples, the multi-axis medical device 202 can have one or more primary axis and/or one or more secondary axis. For instance, in some embodiments, the primary axis can correspond to the longitudinal axis of a body of a medical device or portion of a medical device where two or more limbs are joined, such as joint member 211 (or joint member axis 198A) and the secondary axis or axes can correspond to the longitudinal axis or axes of the limb members of the medical device, such as limb members 213, 210 (or first axis 198B and/or second axis 198C). In some cases, one or more of the secondary axes can be coincident with the primary axis or parallel to the primary axis. In certain cases, one or more of the secondary axes can move relative to the primary axis.
The one or more limb members 213, 210 can be movable and/or fixed. For instance, in the illustrated embodiment of FIG. 2 , the first limb member 213 (or lower jaw) is fixed relative to the joint member 211, and the secondary axis corresponding to the limb member 213 is parallel to the primary axis. In some cases, one or more of the limb members can be an extension of the joint member 211. For example, if the medical device is a stapler with two jaws, the primary axis can correspond to the joint member axis 198A of the stapler or where the two jaws meet and the secondary axes can correspond to the axes of the limb members 198B, 198C.
With continued reference to the embodiment illustrated in FIG. 2 , a second secondary axis can correspond to the axis 198C of the second limb member 210 (or upper jaw) and is not parallel to the joint member axis 198A or the first axis 198B in a first configuration. The second limb member 210 can be moveable relative to the joint member 211 and the first limb member 213. In some cases, the second limb member 210 is hingeably connected to the joint member 211. In certain cases, the range of motion of the second limb member 210 can be an operating parameter input into the system 100. In some embodiments, the range of motion of the second limb member 210 is limited by the hinge. In certain embodiments, a limb member (e.g., upper or lower jaw) can be affixed to the joint member 211.
In a second configuration, such as when the multi-axis device is operated or closed, the second limb member 210 can be parallel or approximately parallel (e.g., within 5, 10, 15, or 20 degrees of parallel (+/−a few degrees)) to the joint member 211. Thus, the secondary axis 198C can change relative to the primary axis as the second jaw 210 moves. For example, if the multi-axis device 202 is a stapler, the stapler 202 can be in a first configuration when the stapler in not activated. The stapler may be in a second (or stapling) configuration when the stapler in activated (e.g., the staples are being applied). In the stapling configuration, the second jaw 210 can be parallel (or approximately parallel) to the joint member 205. In such cases, in the first configuration, the secondary axis 198C is not parallel to the primary axis 198A, and in the second configuration, the secondary axis 198C is parallel (or approximately parallel) to the primary axis 198A. Further, while moving from the first configuration to the second configuration, the secondary axis 198C is not parallel to the primary axis 198A.
In certain embodiments, both the first limb member 213 and the second limb member 210 are affixed to the joint member at different angles such that the primary axis and secondary axis or axes are all different (e.g., the joint member axis 198A, the first axis 198B, and the second axis 198C are all nonparallel). Thus, in some examples, there are at least three nonparallel axes between the joint member 311 and limb members—the primary axis, the first secondary axis and the second secondary axis.
In some embodiments, the multi-axis medical device 202 can have more than three axes and/or can have more than one primary axis. For instance, the joint member 211 can have more than one axis. As an example, the joint member can include one or more bends or hinges (as depicted in FIG. 10 ) wherefrom the joint member 211 can be articulated relative to a proximal shaft of the medical device.
Depicting Medical Device Placement, Trajectory, and Other Image Guidance Cues
In certain procedures, the system can provide image prediction information related to the medical devices as image guidance cues. With continued reference to FIG. 2 , the display 220 illustrates a 3D model of a multi-axis medical device 202, a second medical device 223, a medical image 204, and image guidance cues. While FIG. 2 depicts the multi-axis medical device as a virtual stapler 202, the virtual medical device 202 could include any two jaw device, including but not limited to a transector, resector, sealer, or grasper. Further, while the depicted multi-axis device includes three axes, in some embodiments, the multi-axis device can include two, three, four, five or more axes. For example, the multi-axis device could include three limb members extending out along nonparallel axes from a joint member.
In the illustrated embodiment, the medical device includes a joint member 211 having a joint member axis 198A, a first limb member 213 having a first axis 198B, and a second limb member 210 having a second axis 198C that is different from the joint member axis 198A and the first axis 198B. As mentioned above, the primary axis can correspond to the joint member axis 198A or longitudinal axis of the joint member 211, and the secondary axes can correspond to the first axis 198B and the second axis 198C. In the illustrated embodiment of FIG. 2 , one secondary axis (e.g., first axis 198B) is parallel to the primary axis and another secondary axis (e.g., second axis 198C) is not parallel to the primary axis. However, it should be understood that any of the secondary axis can be parallel or not parallel to the primary axis. It should also be noted that, in some embodiments, a display object includes more than one primary axis.
In some embodiments, a trajectory (or trajectory indicator) of a display object can be determined and displayed. For instance, a trajectory indicator can be displayed to indicate the trajectory of an associated display object, if that display object is moved along a drive path. In some embodiments, the drive path can be dynamic in that it can correlate to the path of the medical device when moved by a user. For instance, the system can determine that the display object (e.g., medical device) is being moved and, in response, can calculate a drive path which corresponding to the path the medical device would take if it continued to move in the same direction. However, it should be understood that the drive path can be predetermined or dynamic. For instance, the drive path can correspond to a path that is parallel to a primary axis of the display object (akin to moving the medical device forward/backward). As another example, the drive path can correspond to a path that is perpendicular to a primary axis of the display object (akin to moving the medical device up/down or side-to-side). Similarly, the drive path can be parallel or perpendicular to a secondary axis. In some embodiments, a display object can have a predetermined drive path when the system determines the display object is not moving and can have a dynamic drive path when the system detects movement.
In a non-limiting example, the display object can be a multi-axis medical device (e.g., a virtual stapler 202) having a joint member 211, a first limb member 213, and second limb member 210. A trajectory can be calculated, determined, and displayed for the first limb member 213, the second limb member 210 and/or the joint member 211. For example, each of the trajectories can be calculated based on a drive path for the medical device. In some cases, the trajectory for each member of the medical device can correspond to the path that that portion of the medical device (i.e., the joint member 211, the first limb member 213, and the second limb member) will follow if the medical device is moved along the drive path. In some embodiments, the system 100 can display a trajectory indicator for a distal portion of each limb member and/or joint member of the medical device. However, it should be understood that a trajectory indicator can be displayed for any and/or all portions of a display object. Similarly, in some embodiments (as illustrated in FIG. 2 ), a trajectory indicator may be displayed for some portions (e.g., the first limb member) and not displayed for other portions (e.g., the second limb member).
Referring to the non-limiting example above, trajectory indicators (e.g., 308A, 308B, 308C, and 308D of FIG. 3 ) can indicate a trajectory along an axis that is different from and parallel to the primary axis (non-limiting example: an axis corresponding to a distal portion of a limb member along the drive path). In addition, the trajectory indicators can extend through or to a distal portion (e.g., the tip) of the medical device (e.g., a distal portion of the second limb member 310). In some cases, the intersection indicators can correspond to, or track, an axis that intersects with a limb member axis (or secondary axis) at a distal portion of the limb member and is parallel to the joint member axis (or primary axis) and/or the drive path.
In some embodiments, such as when both limb members are nonparallel to the joint member, the trajectory indicator(s) can be determined based on a trajectory of the medical device along a primary axis (non-limiting example: drive path that is parallel to the primary axis). In such embodiments, the trajectory of the limb members can be determined based on axes that are different from, but parallel to the primary axis and extend through a distal (or other) portion of the limb members.
While FIG. 2 illustrates a trajectory indicator 208 of a knife path of the first limb member 213, the system can additionally or alternatively include a trajectory indicator of the second limb member 210 (as shown in FIG. 3 ), joint member 211, and/or a transecting knife (as shown in FIG. 3 ). In some embodiments, the trajectory indicator can indicate the trajectory of the medical device along a primary axis and/or the secondary axis. As mentioned above, the system can display one or more trajectory indicators, which can indicate the trajectory of a display object if the display object is moved along a drive path. The trajectory (e.g., trajectory indicator 208) of the medical device 202 can be depicted on the display 220 and can show the healthcare provider the projected path that the medical device will take when moved.
A trajectory indicator can be displayed for any display object (for example, virtual medical devices, affected regions, ghost display objects, image guidance cues, etc.) to indicate the trajectory of the display object if moved along a drive path (e.g., corresponding to a primary axis, a secondary axis, an axis parallel to the primary axis, etc.). In some embodiments, a trajectory indicator indicates the trajectory of a display object if the display object is moved along an axis parallel to the axis of another display object's axis or plane.
In some embodiments, as described below, a display object can be geometrically projected onto a plane-of-interest (i.e., the plane to which that the display object is to be projected) to display a ghost display object (e.g., an outline, cross section, or indication of the display object on the plane-of-interest). In a non-limiting example, the ghost display object can be a ghost medical device having a joint member, a first limb member, and second limb member. The system can determine and/or calculate a trajectory of the ghost medical device by determining the trajectory of the joint member, first limb member, and/or second limb member if the ghost medical device is moved along a primary or secondary axis of the ghost medical device. It will be understood that the trajectory of multiple display objects can be determined and displayed simultaneously on screen 220, as described in greater detail in the '274 Application. Further, it will be understood that the ghost display object can correspond to any display object. For example, the ghost display object can correspond a single axis medical device, such as a needle, as described in greater detail in the '274 Application.
In addition, in some examples, as described in greater detail below with respect to FIG. 3 , the guidance cues can include a ghost medical device 230 (also referred to as a projected or mapped medical device) on the image plane. In these examples, the system can additionally or alternatively display one or more trajectories (e.g., trajectory indicators) of the ghost medical device, such as trajectory indicator 231.
In some embodiments, to implement trajectory indicators, the image guidance system can draw a number of rings about the joint member axis 198A, the first axis 198B, the second axis 198C, or an axis parallel or perpendicular to the joint member axis 198A, the first axis 198B, or the second axis 198C and emanating from a distal (or other) end of a limb member. The trajectory indicators can be extrapolated beyond the tip of the first limb member, second limb member or joint member. In examples which include a ghost medical device, the image guidance system can draw similar trajectory indicators about the one or more axes corresponding to the ghost medical device 230. For example, ghost medical device 330 can include a first and second limb member from which a trajectory indicator can be drawn. Furthermore, the image guidance system can draw a number of rings about an axis of another medical device (for instance, the imaging device).
A healthcare provider can view and manipulate the emplacement of the medical device 242 and its trajectory (via an associated trajectory indicator) before it enters the patient's body along a drive path. In some embodiments, this is accomplished by the doctor positioning a trajectory indicator (e.g., virtual rings or cues) such that it is coincident (or passes through) an ultrasound representation of a target such as tissue that the doctor has spotted in the ultrasound. Positioning the trajectory indicator(s) such that it is properly aimed at the target can provide guidance to the doctor as she directs a medical device 242 into the body until it reaches its desired target or destination. For example, if the doctor identifies tissue 212 in the medical image which needs attention, she can align the stapler 242 such that the trajectory indicators on display 220 intersect or otherwise indicate that the medical device, if directed along the appropriate axis, will reach the tissue 212.
In some embodiments, it may be advantageous for the doctor to position the trajectory indicator (e.g., virtual rings, cues, etc.) such that it is not coincident (or does not pass through) the ultrasound representation of a target. For example, with respect to a stapler, it may be advantageous to align the stapler such that it is substantially parallel with and on or adjacent to the image plane, rather than aligning the stapler such that it intersects the image plane. By aligning the stapler in this manner, the healthcare provider can use the medical image 204 to see more of what tissue will be affected (e.g., the affected region) when the stapler is activated.
The rings can, in some embodiments, be spaced at regular (e.g., 0.5, 1, or 2 cm) intervals to provide the healthcare provider with visual or guidance cues regarding the distance from the medical device tip to the targeted anatomy. In some embodiments, the spacing of the rings can indicate other aspects of the data, such as the speed of the medical device, the density of the tissue, the distance to a landmark, such as the ultrasound data, or any other appropriate guidance data or property. In some embodiments, the rings or other trajectory indicators can extend beyond the medical device tip, by a distance equal to the length of the medical device shaft. This way, the user knows if the medical device is long enough to reach the target, even before the tip(s) enter the patient. That is, in some embodiments, if the rings do not reach the target with the tip still outside the body, then the tip will not reach the target even when the entire device is inserted into the body. In some embodiments, such as when it is preferable to aim the medical device 202 at a target beyond the medical image 204, the trajectory indicator 308 can extend to or even past the image plane.
Other display markers can be used to show trajectory, such as a dashed, dotted, or solid line, transparent medical device shaft, point cloud, wire frame, etc. In some embodiments, three-dimensional rings can be used and provide depth cues and obscure little of the medical image. Virtual rings or other virtual markers can be displayed semi-transparently, so that they obscure less of the medical image than an opaque marker would.
In addition, connecting lines 225 can also be drawn from a plane-of-interest (e.g., medical image plane) to a display object (e.g., virtual medical device 202) or a display object plane (e.g., virtual medical device plane). In some embodiments, the connecting lines extend orthogonally or perpendicularly from a plane-of-interest to a display object plane or a display object. In some embodiments, the connecting lines extend orthogonally or perpendicularly from a display object plane or a display object to a plane-of-interest. However, it will be understood that in some embodiments the connecting lines 225 are not orthogonal (or perpendicular) to any plane. In some embodiments, the connecting lines 225 can be used to determine or display the difference in angle between a plane-of-interest and a display object plane. In some cases, the connecting lines can have an arc. The arc can be determined by treating the intersection of the two display object trajectories as the center of a sphere or circle and treating the connecting line as a perimeter of the circle or sphere.
Other prediction information can also be displayed as image guidance cues. For example, if a stapler is being tracked by the image guidance system, then a cutting plane corresponding to the trajectory of a transecting knife of the stapler can be displayed. Such a cutting plan can be coplanar with the transecting knife of the stapler and can extend from the transecting knife of the stapler (for instance, along the axis of the first member) to, for instance, a plane-of-interest. For example, the trajectory indicator of the transecting knife (e.g., cutting plane) can show where the stapler would cut if the doctor were to advance the stapler along a drive path. Similar prediction information can be estimable or determinable for graspers, vessel sealers, resecting devices, transecting devices, scalpels cauterizers, lasers, and numerous other medical devices.
Furthermore, the data from two or more devices can be combined and displayed based on their relative emplacements or poses. For example, the system 100 can determine the emplacement of an image plane based on the emplacement information of the ultrasound probe 222. Further, the medical image 204 can be displayed on the image plane with respect to the virtual medical device 202 on the display 220 in a manner that estimates the relative emplacements or poses of an ultrasound probe 222 and the medical device 242. As illustrated in FIG. 2 , the image guidance cues associated with the virtual medical device 202, including trajectory indicators, are shown spatially located with the medical image 204 on display 220.
Projecting a Ghost Display Object onto a Plane-of-Interest
In some embodiments, the system 100 can provide additional guidance cues that facilitate understanding of the relationship between the image plane and the medical device(s). For example, display objects (e.g., medical display objects (medical devices, medical images, affected regions, etc.) or image guidance cues (trajectory indicators, intersection indicators, plane indicators, etc.)) can be geometrically projected onto a plane-of-interest such that a shadow, outline (e.g., a dashed, dotted, or solid line), marking, transparent display object, point cloud, wire frame, or other indication of the display object is displayed on the plane-of-interest. In some embodiments, the projected display objects (e.g., projected medical devices, projected medical images, projected image guidance cues, etc.) are known collectively as ghost display objects.
In some embodiments, the plane-of-interest is the plane to which that the display object is to be projected. For example, the plane-of-interest can be an image plane (e.g., the plane corresponding to a medical image), a medical device plane (e.g., the plane corresponding to a virtual medical device), or any other display object plane or other desired plane. In some embodiments, the plane-of-interest can include only a portion of a plane (e.g., the medical image portion of the image plane). However, it will be understood that a plane-of-interest can include more than a portion of any plane including but not limited to the image plane, medical device plane, affected region plane, or other display object plane. In some embodiments, the plane-of-interest can be determined dynamically based on the display object plane. Further, in some cases, the plane-of-interest can be selected to be a plane that is parallel to the display object plane. For example, the plane-of-interest can be selected as the display object plane (as described below) that is parallel to the image plane of a medical image. However, it will be understood that in some cases, the plane-of-interest can be predetermined, such as a display object plane that bisects the display object into halves.
In some embodiments, the display object plane corresponds to any plane that is parallel to a primary or secondary axis of the display object. In some embodiments, the display object plane corresponds to a plane that bisects the display object (e.g., a virtual medical device) into portions (e.g., left and right portions, top and bottom portions, etc.). In some cases, a display object plane can be determined dynamically based on the plane-of-interest. Further, in certain cases, the display object plane can be selected to be a plane that bisects the display object and that is parallel to the plane-of-interest. For example, the display object plane can be selected as a plane (e.g., a plane corresponding to an axis of the display object) that is parallel to the plane-of-interest. Thus, the display object plane can change as the display object moves throughout the system. However, it will be understood that in some cases, the display object plane can be predetermined.
A ghost display object (e.g., projected display object) can be projected onto a plane-of-interest using a variety of techniques. For example, the ghost display object can be an orthographic projection of the display object onto the plane-of-interest. As a non-limiting example, the display object is a virtual medical device and the plane-of-interest is an image plane. The system calculates a plurality of lines which are perpendicular to the image plane. For each perpendicular line that intersects the virtual medical device (or a subset thereof), a point is displayed on the image plane at the location corresponding to the intersecting perpendicular line. As a result, a plurality of points are displayed on the image plane which correspond to the plurality of perpendicular lines (or subset thereof) intersecting the virtual medical device. The plurality of points can illustrate an indication of the virtual medical device on the image plane. In some embodiments, the plurality of points can be referred to as a ghost, a shadow, an outline, a marking, a projection, or an indication of the virtual medical device. It should be understood that the number of perpendicular lines (and the number of corresponding points) can vary across embodiments. For example, in some embodiments, there can be almost infinite perpendicular lines such that a complete representation of the virtual medical device is displayed on the image plane. In some embodiments, the perpendicular lines can be spaced out at predefined intervals such that the ghost medical device includes a dotted outline or dotted representation of the medical device. Similarly, in some cases, the subset of perpendicular lines used to generate the ghost display object can correspond to lines that intersect with a particular portion of the virtual medical device, such as the edge thereby creating an outline on the plane-of-interest.
In some embodiments, a ghost display object can be projected onto a plane-of-interest using a virtual light source. Consider a virtual light source that is at least as large as the display object to be projected or that covers an entire plane (e.g., a light source plane) positioned behind the display object (e.g., distal the plane-of-interest relative to the display object) and parallel to the plane-of-interest, and that emits parallel light waves. In examples such as these, the ghost display object can be displayed on the plane-on-interest as a shadow of the display object. Because the light source is infinitely wide (and can be infinitely far away), the size of the ghost display object can correspond directly with the size of the display object. Further, it will be understood that in some cases, the system can display an outline of the shadow generated by the virtual light source or other indicator.
In some embodiments, a display object can be projected by identifying points on a display object plane that correspond (e.g., intersect) to the display and mapping (e.g., projecting, reproducing, copying) the identified points onto a plane-of-interest. In some embodiments, the identified points on the display object plane can include a plurality of points (e.g., coordinates) that intersect with the corresponding display object. For instance, the plane of a virtual medical device can be a plane that bisects the virtual medical device. The plane can include a plurality of points that intersect with the virtual medical device. In a non-limiting example, to project the virtual medical device onto an image plane, at least some of the plurality of intersecting points can be drawn on the image plane. In certain cases, as described above, the resulting ghost medical device can include an outline or cross-section of the virtual medical device.
In some embodiments, a display object can be projected by identifying or determining which points (e.g., coordinates) of the plane-of-interest are closest (e.g., nearest) to points (e.g., coordinates) of the display object or points of the display object intersecting the display object plane (as described above). Each of the points corresponding to the display object (or the display object plane) can be projected onto the plane-of-interest at the location of the points of the plane-of-interest that were determined to be closest to the identified points of the display object. In some embodiments, projecting can include determining portions of the plane-of-interest that are closest to the display object. The display object can be projected onto those portions of the plane-of-interest that are closest to the display object. In some embodiments, projecting can include determining coordinates of a cross-sectional outline of the display object that are closest to the plane-of-interest.
As a non-limiting example, the ghost display object can be a ghost medical device (for example, a ghost stapler) and the plane-of-interest can be an image plane. The virtual medical device (for example, the virtual stapler) can include a plurality of points on a stapler plane (e.g., a plane corresponding to an axis of the stapler, a plane that bisects the stapler into approximate halves, a plane that is parallel to the image plane, etc.). The system 100 determines which points on the image plane are closest or nearest to the points of the virtual stapler on the stapler plane. The ghost medical device is projected by drawing the points of the virtual medical device onto the determined nearest points of the image plane. In some embodiments, a ghost display object is an outline or other indication of all or a portion of the display object. In some embodiments, the ghost display object is a cross-sectional outline of the display object or a portion thereof. In some embodiments, a ghost display object is a projection of a full or partial representation of the display object. For example, with continued reference to the non-limiting example, the ghost medical device can include a representation of a first limb member, the second limb member, and/or joint member on the image plane.
As a non-limiting example, the ghost display object can be a ghost medical image and the plane-of-interest can be a medical device plane. As mentioned above, the medical device plane can be a plane that bisects the left and right portions (or halves) or top and bottom portions (or halves) of the virtual medical device or portion thereof, such as the joint member. The virtual medical image can include a plurality of points on the image plane. The system 100 determines which points on the medical device plane are closest or nearest to the points of the virtual medical image. The ghost medical image is projected by drawing the points of the virtual medical image onto the determined nearest points of the medical device plane.
As a non-limiting example, the ghost display object is a ghost affected region and the plane-of-interest is an image plane. The affected region can include a plurality of points on a display object plane (for example, a medical device plane, an affected region plane, etc.). The system 100 can determine which points on the image plane are closest or nearest to the points of the affected region on the display object plane. The ghost affected region can be projected by drawing the points from the affected region on the display object plane onto the determined nearest points of the image plane.
In some embodiments, a display object can be projected by determining an arc path and projecting the display object on the plane-of-interest along the arc path. In some embodiments, the system can determine the arc path by first determining an intersection of a display object trajectory and the plane of interest. The system can use the intersection as the center of a sphere and determine an angle between the display object trajectory and plane-of-interest. An arc path can be determined from the angle and each of the points corresponding to the display object (as described above) can then be radially mapped to the plane-of-interest using the arc path. In some embodiments, a distance between each of the points corresponding to the display object and the point of intersection can be determined. In examples such as these, each of the points can then be projected onto the plane-of-interest by placing the points on the plane-of-interest at the same distances they were on the display object plane.
As a non-limiting example, the ghost display object is a ghost medical device and the plane-of-interest is an image plane. The system 100 determines the intersection of the trajectory of the virtual medical device and the image plane. As described above, the virtual medical device can include a plurality of points on the medical device plane. The system 100 determines the distance between the intersection point (i.e., the center of the circle) and each of the plurality points defining the medical device on the medical device plane. The system then projects the points of the medical device onto the image plane at the same distance from the intersection point, but along the image plane.
Although FIG. 3 depicts the ghost medical device 330 projected across the entire image plane, in some examples, the ghost medical device 330 is projected on only a portion of the image plane, such as within the medical image. However, it will be understood that the ghost medical device 330 can be projected onto more than a portion of the image plane. Additionally, in some examples, only the affected region or a portion of the affected region is projected onto the plane-of-interest, such as an image plane or medical image, thereby allowing a healthcare provider to see the region of tissue that would be affected if the medical device 302 were positioned on the image plane. However, it will be understood that any portion of the medical device and/or affected region can be project to the plane-of-interest.
In the illustrated embodiment, the system includes trajectory indicators of multiple display objects (e.g., virtual medical device 302 and ghost medical device 330.) For example, FIG. 3 illustrates trajectory indicator 308A and trajectory indicator 308B, which correspond to the first and second limb member, respectively, if the virtual medical device is moved along a projected drive path (e.g., the primary axis.) Similarly, FIG. 3 illustrates trajectory 308C and trajectory 308D corresponding to the ghost first and second limb member, respectively, if the ghost medical device is moved along a projected drive path (e.g., the primary axis of the ghost medical device). It should be noted that a trajectory can be displayed for any display object, including but not limited to a virtual medical device, a ghost medical device, a medical image, a ghost medical image, an affected area, a ghost affected area, etc. In addition, the trajectory can be relative to any axis. In embodiments where a display object has more than one axis, the display can display a trajectory indicator along a primary and/or secondary axis.
Trajectory indicators for ghost display objects can be calculated, determined, and displayed using a variety of techniques. In some embodiments, the trajectory indicators 308C, 308D can be projected onto the plane-of-interest using techniques as described above with respect to projecting a display object onto a plane-of-interest. In some embodiments, the trajectory indicators 308C, 308D can be determined by calculating a trajectory along an axis of the plane-of-interest. In some embodiments, the trajectory indicators 308C, 308D of the ghost medical device 330 correspond to the trajectory indicators 308A, 308B of the medical device 302 such that the trajectory indicators 308C, 308D intersect with trajectory indicators 308A, 308B. In some embodiments, the trajectory indicators 308C, 308D can indicate a trajectory along an axis different than the trajectory axis of the trajectory indicators 308A, 308B such that the trajectory indicators do not intersect. For instance, trajectory indicators 308C, 308D can extend along a secondary axis of the ghost medical device 330 and trajectory indicators 308A, 308B can extend along a primary axis of the virtual medical device 302.
Intersection Indicators
Referring back to FIG. 2 and as mentioned above, the system can display one or more trajectory indicators 231, 208. For instance, as described above, the system can display trajectory indicator 208 to indicate a trajectory of the medical device 202 along its projected drive path and the system can display trajectory indicator 231 to indicate a trajectory of the ghost display object 230 (e.g., ghost medical device) if moved along its drive path. In some embodiments, because the ghost medical device is a projected version of the virtual medical device, moving the ghost medical device along its projected drive path means moving the virtual medical device along its projected drive path. In some examples, as illustrated in FIG. 4 , the system can also display trajectory indicator 429 to indicate a trajectory another medical device 423 (e.g., the ultrasound transducer) if directed along an associated drive path. In some examples, a trajectory of the transecting knife path can also be displayed. As mentioned above, the system can display a trajectory indicator for any medical device or image and along any axis.
In addition to the trajectory indicators, the display 220 can include another image guidance cue in the form of one or more intersection indicators, such as intersection indicator 209 (or intersection indicators 309A, 309B of FIG. 3 ). The intersection indicators can indicate where one or more portions of the virtual medical device 202 will intersect a plane (for example, the plane corresponding to the medical image 204) if the virtual medical device is moved along an associated drive path and/or where one or more portions or axes of the virtual medical device 202 intersect the plane. In some cases, at the point of intersection 209, the system can display an outline of a portion of the medical device and/or a box or other shape. For example, the system can display a cross-sectional outline of the medical device 202 or a portion thereof. In the illustrated embodiment, the system displays a cross-sectional outline 209 of the lower portion of the medical device 202 and a cross-sectional outline of a knife path 208. However, it will be understood that the system can display more, less, or different parts of the virtual medical device 202. For example, the system can display the upper portion at the point of intersection and display a box around it as well, as desired. Further, additional intersection indicators can be used to show the intersection of one or more trajectories of the virtual medical device and the image plane if the virtual medical device is moved along a different drive path, such as the secondary axis 198C. In addition, trajectory lines can be displayed extending away from the virtual medical device to show where the medical will go if moved forward along its principal or primary axis and/or one or more of the secondary axes. Furthermore, connecting lines 232 can be drawn from the medical image 204 to the intersection location.
In some examples, the intersection indicator 209 can indicate where the trajectory of the virtual medical device, or trajectory indicator 208, intersects the trajectory of the ghost medical device 230 or trajectory indicator 231. It should be noted that an intersection indicator can be displayed for the intersection of any two or more trajectories or trajectory indicators, or for the intersection of any one or more portions of the virtual medical device with the image plane. For instance, in some examples, as depicted in FIG. 4 , the intersection indicator 416 can indicate the intersection between the trajectory of the knife path, or trajectory indicator 408, and the trajectory 429 of the second virtual medical device 423, or trajectory indicator 423.
When data related to two devices or medical devices are displayed with relative emplacement, it can be difficult to orient their relative locations if they do not intersect. In some embodiments, an image guidance system will render relative location information. The relative location information may be shown with color (e.g., objects may be rendered in brighter colors if they are closer), with rendering techniques (e.g., objects may be rendered with transparency so that one object behind another may be visible, but visually appear behind the closer object), with geometry (e.g., a geometric connector may be shown that will allow the viewer to discern the relative relationships), or with any other appropriate technique. For example, in some embodiments, if the intersection indicator is outside of the area of the ultrasound slice, the image guidance system can draw geometry, such as a line (or rectangle) in the plane of the slice to indicate the needle's and medical image's relative positions. In some embodiments, the relative locations could also be represented using vertical and horizontal elements coplanar with the ultrasound or other medical image.
Depicting Affected Region and Other Information
Embodiments of the system can include image guidance cues as part of the image guidance data to depict information related to the region or regions that will be affected by the use of medical devices. For example, in some embodiments, an image guidance cue displayed by the image guidance system can include affected region information. The illustrated embodiment of FIG. 5 shows a virtual multi-axis medical device 502 and an additional 3D cue showing the affected region 526 for the multi-axis medical device 502, indicating, for instance, a region of tissue that will be affected when the multi-axis device 502 is activated. In some examples, the affected region can correspond to the inter-jaw area which can be a roughly triangular area with a curved edge, located between the multiple axes.
In some embodiments, the affected region can correspond to the volume located between the two jaws (i.e., the first limb member 513 and the second limb member 510). In some cases, the affected region can be calculated based at least in part on a distance from the hinge. Additional affected sub-regions can be calculated and displayed as well. For example, the plane at which the transecting knife cuts 514 (the line shown in the middle portion of the bottom jaw 513 running generally left to right) can be calculated and shown as well as the one or more planes corresponding to the one or more staples (in the illustrated embodiment, the five lines 527 or bands running transverse to the knife plane.
In some cases, the affected sub-regions can be transverse or perpendicular to each other. For example, an affected knife sub-region can be transverse to an affected staple sub-region. Further, any of the cues described herein can be displayed in conjunction with the aforementioned cues.
In some embodiments, such as when a multi-axis medical device is aimed at the medical image (e.g., a trajectory of the medical device intersects the medical image), the system can display the intersection of the affected region or trajectory of the affected region along a drive path with the medical image (or image plane). Thus, in some embodiments, such as when the medical device is roughly parallel with the image plane (or the medical device trajectory does not intersect with the medical image) the system can project the affected region, or portions thereof, onto the image plane. In examples such as these, the ghost affected region can be a roughly triangular area, which can correspond to the area or volume between the limb members that is projected onto the image plane or medical image. In certain embodiments, such as when the medical device is roughly perpendicular (e.g., with 45 degrees of being perpendicular) to the image plane (or the medical device trajectory intersects with the medical image), the system can display an intersection indicator on the image plane that corresponds to the intersection of the trajectory of the first limb member and the medical image or the intersection of the first limb member with the medical image. In addition, an intersection indicator can be displayed at the intersection of the trajectory of the second limb member and the medical image or the intersection of the second limb member with the medical image.
In some embodiments, the system can use the operating parameters of the medical device 502 and/or measured parameters to determine the affected region (and display the displayed affected region 526). For example, the affected region's size (e.g., girth and length) can be dependent on the length of the first limb member 513, the length of the second limb member 510, the range of movement of the first limb member, the range of movement of the second limb member, etc. These dimensions and operating ranges can be manually entered and/or automatically computed by the guidance system based on device or operating parameters, such as, but not limited to, the medical device make and model, power settings, and the like. Similarly, the system can use measured parameters to determine the affected region, such as, but not limited to, range of motion of the limb member(s), distance between the limb members, available space within the body of the patient, dimensions of tissue to be stapled, etc. For example, in the context of a stapler, the stapler may have a normal range of motion (known as default settings) that can be adjusted based at least in part on the size of the area to be stapled. In some embodiments, the measured parameters can be received in real-time as real-time data. For instance, as distance between the stapler jaws is increased or decreased, the system can display an adjusted affected region. In some examples, the system can use one or more formulas, look-up-tables, fixed or default values, or any other appropriate available information, etc. to determine the affected region.
In addition, the system can determine affected regions prior to operating the medical device and/or during operation of the medical device. For example, prior to operating the medical device, the system can determine one or more affected regions and/or during operation of the medical device, the system can determine one or more dynamic affected regions. In some embodiments, the affected region is static until operation of the medical device and then change as the device is operated. In certain embodiments, the system may rely more on operating parameters of the medical device to determine an initial affected region (i.e., the starting distance between the jaws of the medical device) and measured parameters to determine a dynamic affected region (i.e., as the jaws are opening or closing). However, it will be understood that operating parameters and/or measured parameters can be used to determine the initial affected region and/or the dynamic affected region. In some instance, the medical device 300 is inserted into and/or is moved around inside the patient in a closed or partially closed state in order to reduce the likelihood of causing damage with the medical device. In such examples, the display affected region can correspond to the smaller region located between the first limb member and the second limb member or the displayed affected area can be displayed as if the medical device was fully open.
In some circumstances, the operating parameters, measured parameters, formulas, a look-up-tables, fixed or default values, or other information used to determine the affected regions may include some amount of error or variance. The variance may be due, for instance, to uncertainty regarding the distance between the first limb member 313 and the second limb member 310.
Returning to FIG. 2 , the system can also determine the emplacement of the affected region. In some cases, the emplacement of the affected region can be based at least in part on the emplacement of some or all of the corresponding medical device (or virtual medical device), such as medical device 242. For example, the system can receive emplacement data from one or more emplacement sensors associated with the medical device 242 (non-limiting examples: coupled to or integrated with the medical device 242, within an optical path of the medical device, etc.). The system can use the emplacement data to determine the emplacement of the tracked medical device and/or the emplacement of the virtual medical device 202 corresponding to the medical device 242. In some instances, the system can determine the emplacement of the medical device 242 and/or virtual medical device 202 with respect to a point-of-view location.
As yet another example, if the medical device is a stapler and the affected region is a volume located between the first limb member and the second limb member, the emplacement of the affected region can be based at least in part on the emplacement of the stapler (or its virtual version) or at least a portion of it, such as the location(s) on the stapler where the staples are located or portions of the first or second limb members. Specifically, in some embodiments, the affected region can be centered at a location on the medical device, such as the portions on the medical device that move when the medical device is operated. In some examples, if multiple medical devices are used, the posed can be based at least in part on the emplacement of the medical devices.
The system can display the affected region in a variety of ways. Furthermore, although the illustrated embodiment of FIG. 2 includes only one displayed affected region 206, it will be understood that one or more affected regions can be displayed corresponding to each medical device 242 that is displayed on the screen 220 and/or multiple affected regions can be displayed corresponding to a single medical device 242. In some embodiments, the system can display a perspective view of the affected region and/or non-perspective view, such as by displaying the affected region on or with the medical image displayed in the 2D view. Some or all of the affected regions can be displayed as desired. In some embodiments, the portion of the affected region that is displayed can be referred to as the displayed affected.
Although the illustrated embodiment of FIG. 2 refers to the affected region as a volume located between the first limb member and the second limb member, it will be understood that the affected region can correspond to a variety of medical procedures. For example, if a cauterizer is tracked as part of an image guidance system, then the affected region can correspond to a cauterization volume. If a laser is tracked as part of the image guidance system, then the affected region can correspond to a projected laser path. If an ablation needed is tracked as part of the image guidance system, then the affected region can correspond to an ablation volume. Similarly, the affected region can correspond to a biopsy volume, an electroporated volume, cryoablation volume, laser ablation volume, high-frequency focused ultrasound ablation (HIFU) volume, external beam radiation therapy volume, and drilling volume (where the display volume corresponds to the region of bone and other tissue that the manually operated, or computer-controlled drill would remove), depending on the type of medical instrument being used.
As discussed above, in some embodiments, the system 100 projects the affected region (or a portion of the affected region) of the virtual medical device to the image plane or medical image. In some embodiments, the affected region (or a portion of the affected region) is projected onto the plane-of-interest (for example, the image plane) in addition to the ghost medical device. In some embodiments, the affected region (or a portion of the affected region) is projected onto the plane-of-interest without projecting a ghost medical device. In some examples, projecting the ghost medical device and the affected region causes the image plane to be crowded. Thus, projecting only the affected region can make it easier for the physician to see exactly what tissue will be affected. However, it will be understood that more than the affected region can be projected onto the plane-of-interest.
Although FIG. 5 illustrates a multi-axis medical device having two different axes, a multi-axis device can have any number of axes. For example, the multi-axis device can have three, four, five or more axes. As a non-limiting example, the multi-axis medical device can be a three-prong grasper (not shown). The three-prong grasper can have an axis corresponding to each prong and an axis corresponding to its body. In this non-limiting example, the affected region corresponding to the three-prong grasper can include the region between the three prongs, the region corresponding to the volume that will be affected if the prongs are activated, etc.
Tracking a Medical Instrument
It will be understood that the medical device 602 can be tracked in a variety of ways. As shown above, visual tracking fiducials 632 can be placed on different portions of the medical device 602. In some cases, sensors, such as 5 DOF sensors or other sensors can be used. In some embodiments, multiple sensors can be coupled to the medical device 602. For example, one or more sensors can be located on the medical device proximal to the jaw hinge and/or proximal to a second hinge (e.g., second hinge 1028 shown in FIG. 10 ), and/or one or more sensors can be coupled to a lower portion and/or the upper portion of the first and/or second limb member.
As described above, in some examples, a medical device 602 can also have markings such as bands around the shaft or jaws, for example, to indicate distance along the shaft or jaws. Healthcare providers performing medical device procedures are often familiar with these markings and can use them to help understand the spatial relationship between the medical device and anatomy. In some embodiments, the make and model of the medical device is known to the image guidance system and the virtual medical device 602 displayed in display 220 can resemble medical device 242. The features of medical devices that can be rendered in the scene include the overall shape (diameter, angles, cross sectional shape, curvature, etc.), color, distance markers, angle of the jaws, visuals or echogenic fiduciary markers, the state of deployable elements such as tines, paddles, anchors, resection loops, stiffening or steerable sleeves, temperature, radiation, light or magnetic field sensors, lens, waveguides, fluid transfer channels, and the like.
In addition, the system can use a switch, sensor, or button or other mechanism to determine when the jaw has been closed. For example, a switch or sensor can be near the handle or other actuator used to close the jaw. Once activated, the system can update the virtual display to show that the jaw has been closed. In some examples, as described above, jaw open or closure and distance between the jaws can be operating parameters of the medical device that are used to display the virtual medical device.
In some instances, a first medical device can be used to mark a path for a second medical instrument. Accordingly, the system 100 can track the first medical device such that the system can display the marked path for the second medical device. FIG. 7 illustrates an example of tracking a first medical device (e.g., a cauterizing device 702) which is used to cauterize (or mark) a path (e.g., a stapling, transecting, resecting, grasping path). In this example, the navigation software can assist the physician to cauterize (or mark) tissue while inspecting the medical image relative to the tip of the cauterizing device. While using the medical image for guidance, the cauterizing device 702 and/or tip of the cauterizing device can be tracked and subsequently used as guidance for a second medical device (e.g., a stapler).
Any of the navigation features and/or guidance cues can also be used for with the cauterizing device 730. For instance, the cauterizing device 730 can have associated trajectory indicators, shadow bars, ghost cauterizing device, projections, tip-to-plane distance markers, and any other guidance cues as described herein.
Medical Image can Traverse the Affected Region
In some embodiments, the medical image 804 can traverse the affected region 826 of the medical device 802. In such cases, the system 100 can determine and display a point or area of intersection of the affected region 826 with the medical image 804.
In some examples, the system 100 can determine and display a point of area of intersection between a plane (for example, an image plane) or image area and an affected region 826. As shown in FIG. 8 , an intersection indicator 830 can be projected onto the image plane and/or medical image 804 to illustrate the location on the medical image 804 that the stapler will intersect if the stapler is activated (e.g., closed). In some examples, the depicted intersection indicator can appear like a schematic of the multi-axis device superimposed onto the image plane. In other examples, the intersection indicator 830 can be markings such as the guidance cues described above. Using the intersection indicator 830, the healthcare provider can identify where a medical device will intersect a plane (e.g., the image plane) if the stapler is operated.
Combining an Ultrasound Probe with a Two-Jaw Instrument
In some embodiments, an ultrasound transducer (for example, a miniature ultrasound transducer) can be integrated with a multi-axis instrument such as a transecting stapler, grasper, or vessel sealer. A physician may use the integrated ultrasound transducer and jawed instrument to ensure that the physician can view, in ultrasound, the internal tissue structures before they are stapled, transected, resected, sealed, or grasped, thereby helping the physician avoid critical structures (such as blood vessels). As described above, a 3D viewing area can include perspective views of each of a virtual integrated device, a medical image, navigation cues (e.g. affected region as shown in FIG. 5 ) or other cues described herein. In some examples, the integrated device can include one or more sensors located at different locations on the device. For example, a sensor can be embedded inside, or affixed to one or more of the joint member, the first limb member, the second limb member, and the ultrasound transducer (such as at the tip, along the shaft, or on a handle). Furthermore, the system can use the known configuration of the device to determine the proper location between the affected region and the medical image for display purposes.
Although FIG. 9 illustrates the transducer mounted to the tip of the first limb member, the transducer can be mounted anywhere on the medical device. For example, the transducer can be mounted to the second limb member 910 (such as at the tip, along the body, or near the hinge), the first limb member 913 (such as at the tip, along the body, or near the hinge), or the joint member (such as at the tip, along the body, or near/on the hinge).
It will be understood that the ultrasound transducer can be integrated with the medical device in a variety of ways and that FIGS. 9-12 provide non-limiting examples. Further, the transducers can cover a variety of angles. They can be attached to any portion of the multi-axis instrument (such as either the first limb member or the second limb member) and angled differently than shown. In addition, 2D phased arrays can be flat and thus integrate more seamlessly with tool designs than shown, avoiding local tissue displacement at the transducing element.
Flow Diagrams
At block 1302, the system 100 receives first emplacement data associated with a first emplacement sensor and/or a first medical device. The first emplacement data can be generated by the first emplacement sensor and/or by a position sensing unit. In some embodiments, the first emplacement sensor can be associated with the first medical device. For example, the first emplacement sensor can be associated with and/or attached to an ultrasound transducer, as described above. In some embodiments, the first medical device is a multi-axis medical device. Furthermore, the first medical device has a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis. As described above, one or more emplacement sensors can be associated with a first limb member, a second limb member, and/or a joint member of the associated first medical device. The system can receive emplacement data from each of these emplacement sensors.
At block 1304, the system 100 receives second emplacement data associated with a second emplacement sensor and/or a second medical device. The second emplacement data can be generated by the second emplacement sensor and/or by a position sensing unit. In some embodiments, the second emplacement sensor can be associated with the second medical device. For example, the second emplacement sensor can be associated with and/or attached to a stapler, as described above with respect to FIGS. 9-12 . In some embodiments, the second medical device is an imaging device such as an ultrasound transducer.
At block 1306, the system 100 can determine an emplacement of a first virtual medical device based at least in part on the received first emplacement data. As described above, the first virtual medical device can correspond to a multi-axis device such as a transecting stapler, grasper, or vessel sealer. In some embodiments, the virtual medical device corresponds to a single-axis device such as an ablation probe.
In some embodiments, the system 100 can use the first emplacement data and one or more characteristics of the associated first medical device (or a corresponding virtual medical device) to determine the emplacement of at least a portion of the first medical device. For example, the at least a portion of the first medical device can include a portion the first limb member, a portion of the second limb member, or a portion of the joint member. As a non-limiting example, the characteristics of the associated first medical device may indicate an emplacement of the second limb member relative to the associated first medical device.
Using the received emplacement data, the system 100 can determine the emplacement of the first medical device, for example, relative to known location of emplacement sensor(s) on the first medical device. For example, the system 100 can use a known relationship between the first emplacement data and the emplacement of the medical device (non-limiting example: the second limb member extends 8 cm. away from the hinge at a 45 degree angle relative to the first limb member) and/or use a known relationship between the emplacement of the emplacement sensor(s) and/or associated medical device (or virtual medical device) and the emplacement of the medical device (non-limiting examples: the first limb member begins 4 cm. from the tip of the first medical device (or first virtual medical device) and ends at the hinge of the first medical device (or first virtual medical device), or the first limb member extends 4 cm. in either direction from the ends of one of the emplacement sensors).
In addition, in certain embodiments, the system 100 can determine the emplacement of the first medical device in one or more coordinate systems by mapping the first emplacement data, from one coordinate system to a second coordinate system. For example, the first emplacement data may be received with respect to a first coordinate system, such as a position sensing coordinate system, and then mapped to a second coordinate system, such as a 3D scene coordinate system and/or a screen coordinate system. The emplacement of the first medical device can be determined with respect to one or more of the coordinate systems. For example, the emplacement of the first medical device can be determined after the first emplacement data has been mapped to the second coordinate system, such as the 3D scene coordinate system and/or the screen coordinate system, or the emplacement of the first medical device can be determined for the first coordinate system, such as the position sensing coordinate system, and then mapped to the 3D scene coordinate system and/or the screen coordinate system.
In certain embodiments, the system 100 can also use point-of-view location (e.g., the location of the point-of-view for viewing the 3D space) to determine the emplacement of the first medical device for viewing. In some embodiments, as described in greater detail in '933 Application, the point-of-view location can be a fixed location, such as a predetermined distance/angle from the screen 220 or stand 170 and or a location configured by the user; or the point-of-view location can by dynamic. For example, the system can track a user in real-time and determine the point-of-view location based at least in part on the tracked location of the user.
In certain embodiments, the system 100 can also use an offset to determine the emplacement of the first virtual medical device for viewing. For example, the system 100 can determine an initial emplacement of the first medical device in the 3D scene coordinate system and/or the screen coordinate system, and then apply an offset to the initial emplacement and/or the system 100 can determine an initial emplacement of the first medical device in the position sensing coordinate system and apply an offset to the initial emplacement prior to mapping the emplacement of the first medical device in the position sensing coordinate system to the 3D scene coordinate system and/or the screen coordinate system.
The offset can be made in one, or a combination of, coordinate systems, and/or with respect to one, or a combination of, axes. In certain embodiments, the offset can be made along a y-axis (up/down) of the position sensing coordinate system, the 3D scene coordinate system and/or the screen coordinate system. For example, the system 100 can adjust the y-coordinate (up/down coordinate) of the determined emplacement of the first medical device (or the first emplacement sensor(s)) in the position sensing coordinate system by the offset amount. When mapped to the 3D scene coordinate system and/or the screen coordinate system, the system 100 can use the adjusted emplacement. As yet another example, the system 100 can adjust the y-coordinate (up/down coordinate) of the determined emplacement of the first medical device (or the first emplacement sensor(s)) in the 3D scene coordinate system and/or the screen coordinate system by the offset amount. Any combination of the above-referenced examples can be used as desired. Furthermore, it will be understood that the offset can be made in any one or any combinations of the coordinate systems and with reference to any one or any combination of the axes. For example, the adjustment can be made along any one or any combination of the x-axis, y-axis, or z-axis.
Furthermore, the offset can be a predetermined offset and/or a dynamic offset. In some embodiments, a predetermined offset can be used. For example, the system 100 can use a static offset based on an average height of males and/or females or average distance between elbows and hands, the height of the user, a distance between the user's elbow and eyes, expected location of a user with respect to the imaged volume, etc. In certain embodiments, the system 100 can use a dynamic offset, such as a determined emplacement of an HMD relative to one or more emplacement sensors, position sensing region, and/or position sensing unit coordinate system. For example, the system 100 can determine the emplacement of an HMD relative to a medical device or imaged area and adjust the offset such that the medical image is always in view on the displays of an HMD. With continued reference to the example, if the wearer crouches down or turns to the side, the system 100 can determine the change in relative emplacement between the HMD and the medical device or imaged area and adjust the offset such that the medical image remains in view in substantially the same emplacement.
In addition, it will be understood that the offset described herein with reference to the first medical device can be applied to any one or any combination of the objects to be displayed and/or to all contents of the virtual 3D scene. In some embodiments, the offset can be applied to some objects to be displayed but not others.
At block 1308, the system 100 can determine an emplacement of a second virtual medical device based at least in part on the received second emplacement data. As described above, the second virtual medical device can correspond to an imaging device such as an ultrasound transducer. The emplacement of the second virtual medical device can be determined in a similar manner as described with respect to the first medical device at block 1306.
At block 1310, the system 100 determines an emplacement of a medical image (e.g., medical image) based at least in part on received second emplacement data. The medical image can be an intra-operative and/or real-time medical image, such as a live ultrasound or intra-operative CT scan, or can be a pre-operative image, such as a pre-operative CT or MRI scan image. A real-time medical image (or real-time medical imaging stream) can refer to a medical image (or real-time medical imaging stream) received in real-time. The medical image received in real-time can correspond to a live image, such as a live medical image generated by an ultrasound or other image, such as a pre-operative or intra-operative CT image or MRI image that is communicated in real-time.
In some embodiments, the system 100 can use the second emplacement data and one or more characteristics of the second emplacement sensor or associated second medical device (or a corresponding second virtual medical device) to determine the emplacement of the medical image. For example, the characteristics may indicate an emplacement of the medical image relative to the second emplacement sensor or associated second medical device (or second virtual medical imaging device).
Using this information, the system 100 can determine the emplacement of the medical image relative to the second emplacement sensor and/or associated second medical device (or second virtual medical imaging device). For example, the system 100 can use a known relationship between the second emplacement data and the emplacement of the medical image (non-limiting example: the medical image begins 2 cm. away from the of the second emplacement data location in a particular direction and ends 5 cm. away) and/or use a known relationship between the emplacement of the second emplacement sensor and/or associated second medical device (or second virtual medical imaging device) and the emplacement of the medical image (non-limiting examples: the medical image begins 4 cm. from the tip of the second medical device (or second virtual medical imaging device) and ends at the tip of the second medical device (or second virtual medical imaging device), or the medical image extends 2 cm. in either direction from the ends of the second emplacement sensor).
In addition, in certain embodiments, the system 100 can determine the emplacement of the medical image in one or more coordinate systems by mapping the first emplacement data, from one coordinate system to a second coordinate system. For example, the second emplacement data may be received with respect to a first coordinate system, such as a position sensing coordinate system, and then mapped to a second coordinate system, such as a 3D scene coordinate system and/or a screen coordinate system. The emplacement of the medical image can be determined with respect to one or more of the coordinate systems. For example, the emplacement of the medical image can be determined after the second emplacement data has been mapped to the second coordinate system, such as the 3D scene coordinate system and/or the screen coordinate system, or the emplacement of the medical image can be determined for the first coordinate system, such as the position sensing coordinate system, and then mapped to the 3D scene coordinate system and/or the screen coordinate system.
In certain embodiments, the system 100 can also use point-of-view location (e.g., the location of the point-of-view for viewing the 3D space) to determine the emplacement of the medical image for viewing. In some embodiments, as described in greater detail in '933 Application, the point-of-view location can be a fixed location, such as a predetermined distance/angle from the screen 220 or stand 170 and or a location configured by the user; or the point-of-view location can by dynamic. For example, the system can track a user in real-time and determine the point-of-view location based at least in part on the tracked location of the user.
As described above with respect to block 1306, in certain embodiments, the system 100 can also use an offset to determine the emplacement of the medical image for viewing. For example, the system 100 can determine an initial emplacement of the medical image in the 3D scene coordinate system and/or the screen coordinate system, and then apply an offset to the initial emplacement and/or the system 100 can determine an initial emplacement of the medical image in the position sensing coordinate system and apply an offset to the initial emplacement prior to mapping the emplacement of the medical image in the position sensing coordinate system to the 3D scene coordinate system and/or the screen coordinate system.
The offset can be made in one, or a combination of, coordinate systems, and/or with respect to one, or a combination of, axes. In certain embodiments, the offset can be made along a y-axis (up/down) of the position sensing coordinate system, the 3D scene coordinate system and/or the screen coordinate system. For example, the system 100 can adjust the y-coordinate (up/down coordinate) of the determined emplacement of the medical image (or the first emplacement sensor) in the position sensing coordinate system by the offset amount. When mapped to the 3D scene coordinate system and/or the screen coordinate system, the system 100 can use the adjusted emplacement. As yet another example, the system 100 can adjust the y-coordinate (up/down coordinate) of the determined emplacement of the medical image (or the first emplacement sensor) in the 3D scene coordinate system and/or the screen coordinate system by the offset amount. Any combination of the above-referenced examples can be used as desired. Furthermore, it will be understood that the offset can be made in any one or any combinations of the coordinate systems and with reference to any one or any combination of the axes. For example, the adjustment can be made along any one or any combination of the x-axis, y-axis, or z-axis.
Furthermore, the offset can be a predetermined offset and/or a dynamic offset. In some embodiments, a predetermined offset can be used. For example, the system 100 can use a static offset based on an average height of males and/or females or average distance between elbows and hands, the height of the user, a distance between the user's elbow and eyes, expected location of a user with respect to the imaged volume, etc. In certain embodiments, the system 100 can use a dynamic offset, such as a determined emplacement of an HMD relative to one or more emplacement sensors, position sensing region, and/or position sensing unit coordinate system. For example, the system 100 can determine the emplacement of an HMD relative to a medical device or imaged area and adjust the offset such that the medical image is always in view on the displays of an HMD. With continued reference to the example, if the wearer crouches down or turns to the side, the system 100 can determine the change in relative emplacement between the HMD and the medical device or imaged area and adjust the offset such that the medical image remains in view in substantially the same emplacement.
In addition, it will be understood that the offset described herein with reference to the medical image can be applied to any one or any combination of the objects to be displayed and/or to all contents of the virtual 3D scene. In some embodiments, the offset can be applied to some objects to be displayed but not others.
At block 1312, the system 100 can determine a first intersection between an image plane and a trajectory of the first limb member along a drive path. As described above with respect to FIGS. 2 and 3 , a trajectory can be calculated, determined, and displayed for the first limb member 213, the second limb member 210 and/or the joint member 211. For example, each of the trajectories can be calculated based on a drive path for the medical device. With respect to the first limb member, the trajectory can correspond to the path that the first limb member will follow if the medical device is moved along the drive path.
In some examples, trajectory indicators can indicate a trajectory along an axis that is different from and parallel to the primary axis (non-limiting example: an axis corresponding to a distal portion of the first limb member along the drive path). In addition, the trajectory indicators can extend through a distal portion (e.g., the tip) of the medical device (e.g., a distal portion of the first limb member.
The first intersection indicator can indicate where the first limb member of the first virtual medical device will intersect the image plane (e.g., the plane corresponding to the medical image) if the virtual medical device is moved along an associated drive path and/or where one or more portions or axes of the first virtual medical device intersect the image plane. As described above, the drive path can be predetermined or dynamic.
At block 1314, the system 100 can determine a second intersection between an image plane and a trajectory of the second limb member along the drive path. The second intersection indicator can be determined using the same techniques as described above and described with respect to block 1312. With respect to the second limb member, the trajectory can correspond to the path that the second limb member will follow if the medical device is moved along the drive path. In some embodiments, the trajectory (e.g., the trajectory indicators) can extend through a distal portion (e.g., the tip) of the medical device (e.g., a distal portion of the second limb member.)
The second intersection indicator can indicate where the second limb member of the first virtual medical device will intersect the image plane (e.g., the plane corresponding to the medical image) if the first virtual medical device is moved along an associated drive path and/or where one or more portions or axes of the first virtual medical device intersect the image plane.
As described in greater detail above with reference to FIGS. 2 and 3 , the system 100 can determine the points of intersection between one or more trajectories and/or trajectory indicators, one or more axes (or portions) of one or more display objects (for example, a virtual medical device, a ghost display object, an affected region, etc.) and/or one or more planes (e.g., the virtual medical device plane, the image plane, etc.) based on the emplacement information of one or more medical devices.
In a non-limiting example, one or more intersection indicators can be determined and displayed for an intersection between a trajectory of a medical device (for example, a trajectory of the first limb member of a stapler) and a plane-of-interest (for example, an image plane). One or more intersection indicators can also be displayed for an intersection between a trajectory of the second limb member along a drive path (for example, an axis parallel to the primary axis and intersecting the second limb member) and a plane-of-interest (for example, an image plane).
In a non-limiting example, one or more intersection indicators can be displayed for an intersection between a trajectory of a medical device (for example, a stapler, an ultrasound transducer, medical imaging device, etc.) and an affected region (or trajectory of an affected region)
In a non-limiting example, one or more intersection indicators can be displayed for an intersection between a trajectory of a medical device (for example, a stapler, an ultrasound transducer, medical imaging device, etc.) and an axis, trajectory, or plane of a transecting knife.
In a non-limiting example, one or more intersection indicators can be displayed for an intersection between a trajectory of a ghost display object (for example, a ghost medical device, a ghost affected region, a ghost medical image, etc.), a stapler, an ultrasound transducer, medical imaging device, etc., and an a trajectory of another display object (for example, a medical device, an affected region, a medical image, etc.).
At block 1316, the system 100 can cause a display to display a view of a virtual 3D scene including a perspective rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device. In some embodiments, the 3D rendering of the at least a portion of the first virtual medical device includes at least a portion of the first limb member and at least a portion of the second limb member. However, it will be understood that the 3D rendering of the at least a portion of the first virtual medical device can include any portions of the first virtual device.
As described above, the perspective view or rendering the medical image can be determined and displayed based at least in part on a point-of-view location. The point-of-view location can be a fixed point-of-view location or a dynamic point-of-view location. For example, the point-of-view location can be with respect to a set location in front of the display and/or can be based on a tracked location of the display, the HMD, or the user. In some embodiments, the point-of-view location is determined based at least in part on the relative emplacement of an HMD or user with respect to the position sensing region or imaged volume, or the emplacement of an HMD within a position sensing coordinate system. In embodiments, in which the point-of-view location is based on a tracked location, the system 100 can enable the user to view different perspective views of the virtual 3D scene based on the changing emplacement of the tracked object (HMD, user) relative to the medical devices 145, 155 or the position sensing region.
In certain embodiments, the system 100 can determine multiple emplacements for the first virtual medical device. For example, the system 100 can determine the emplacement for the medical image for a right-eye view and a left-eye view of a stereoscopic display, such for a HMD. In this way, each display for the HMD can display the medical image from a slightly different perspective corresponding to a right-eye view and a left-eye view, etc.
At block 1316, the system 100 can include the medical image in the virtual 3D scene and cause the display to concurrently display a perspective rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device, similar to the display of the medical described above with reference to block 1314. In some embodiments, the display of the second virtual medical device can be based at least in part on dimensions of a corresponding real medical device (non-limiting examples: size, shape, or other appearance). As described above with respect to block 1314, the perspective view or rendering the medical image can be determined and displayed based at least in part on a point-of-view location.
At block 1318, the system 100 can cause a display to display a view of a virtual 3D scene including a perspective rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image. As described above with respect to block 1314, the perspective view or rendering the medical image can be determined and displayed based at least in part on a point-of-view location.
At blocks 1322 and 1324, the system 100 can cause a display to display the first and second trajectory indicators. As described above, a healthcare provider can use the displayed trajectory indicators to view and manipulate the emplacement of the medical device and its trajectory before it enters the patient's body along a drive path. In some embodiments, this is accomplished by the doctor positioning a trajectory indicator such that it is coincident (or passes through) an ultrasound representation of a target such as tissue that the doctor has spotted in the ultrasound. Positioning the trajectory indicator(s) such that it is properly aimed at the target can provide guidance to the doctor as she directs a medical device into the body until it reaches its desired target or destination. For example, if the doctor identifies tissue in the medical image which needs attention, she can align the medical such that the trajectory indicators on display intersect or otherwise indicate that the medical device, if directed along the appropriate axis, will reach the tissue.
At blocks 1326 and 1328, the system 100 can cause a display to display the first and second intersection indicators. As described above, using the intersection indicators, the healthcare provider can identify where a medical device (e.g., a multi-axis device) will intersect the plane (e.g., the image plane) if moved along a drive path (e.g., a primary, a secondary axis, axis parallel to the primary or secondary axis, etc.). Based on the desired placement of the medical device, the healthcare provider can change the position and orientation of the medical device as desired. Thus, the intersection indicators as well as the trajectory indicators can aid a healthcare provider during the placement of a medical device. Furthermore, the system can calculate distances between the planes and the tips of the medical devices or other portions of the medical devices. The calculated distances can be displayed and/or used to display the medical devices.
In some embodiments, the system 100 can determine and display a point of intersection between a plane of interest (for example, an image plane, a medical device plane, etc.) and one or more medical device trajectories. For example, the system can determine and display a point of intersection between an image plane and a joint member trajectory. In some embodiments, the system 100 can determine and display a point of intersection between a medical device plane and another medical device trajectory. In some embodiments, the system 100 can determine and display a point of intersection between a ghost display object (or trajectory of a ghost display object) and a plane (for example, an image plane, a medical device plane, etc.). In some embodiments, a point of intersection can be determined and displayed between a ghost display object (or its trajectory) and a medical device plane. For example, if the ghost display object is a ghost medical device, an intersection indicator can be displayed at the intersection of a trajectory of the ghost joint member and the medical device plane. Alternatively or in addition, an intersection indicator can be displayed at the intersection of the trajectory of the ghost second limb member along the drive path and the medical image. In some examples, the point(s) of intersection are located on the image plane, within the medical image. In other examples, the point(s) of intersection are located on the image plane, outside of the medical image.
It will be understood that the various blocks described herein can be implemented in a variety of orders, and that the system 100 can implement one or more of the blocks concurrently and/or change the order, as desired. For example, the system 100 can concurrently receive the emplacement data from different sources, concurrently receive the medical image, or receive the data in any order. Similarly, the system 100 can concurrently determine the emplacement of the medical image and/or one or more virtual medical devices, etc.
Furthermore, it will be understood that fewer, more, or different blocks can be used as part of the routine 1300. For example, the routine 1300 can include blocks for receiving emplacement data associated with additional emplacement sensors or medical devices, determining emplacements of one or more medical devices, corresponding virtual medical devices, other display objects, displays, and/or users. In some embodiments, the routine 1300 can include determining an emplacement of a medical device associated with the medical image and determining the emplacement of the medical image based at least in part on the determined emplacement of the medical device (or corresponding virtual medical device), and display the virtual medical device concurrently with the medical image. Furthermore, the system 100 can determine and display a variety of image guidance cues, such as trajectory indicators, affected region indicators, intersection indicators, medical devices in different states or configurations, ghost display objects, as described in greater detail above, and below with reference to FIG. 14 . In addition, in some cases, the routine 1300 can omit certain blocks, such as, but not limited to, blocks 1302, 1304, 1308, 1316, 1318, 1322, and/or 1324. For example, in some embodiments, the system may not determine emplacement of or display a portion of the second display object and/or may not display a portion of the first medical device, first trajectory indicator, and/or second trajectory indicator, etc.
At block 1402, as also described above with reference to block 1302 of FIG. 13 , the system 100 receives first emplacement data associated with a first emplacement sensor and/or a first medical device. In some embodiments, the first emplacement sensor can be associated with the first medical device. In some embodiments, the first medical device is a multi-axis medical device. Furthermore, the first medical device can have a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis. As described above, one or more emplacement sensors can be associated with a first limb member, a second limb member, and/or a joint member of the associated first medical device.
At block 1404, as also described above with reference to block 1304 of FIG. 13 , the system 100 receives second emplacement data associated with a second emplacement sensor and/or a second medical device.
At block 1406, the system 100 receives a medical image corresponding to the second medical device.
At block 1408, as described similarly above with reference to block 1306 of FIG. 13 , the system 100 determines an emplacement of a first display object based at least in part on received first emplacement data. In some embodiments, the first display object can be a multi-axis medical device having a joint member, a first limb member, and a second limb member. In some embodiments, the second display object can be any medical display objects (e.g., virtual medical device, medical image, ghost display object, etc.) and/or any image guidance cue (e.g., trajectory indicator, intersection indicator, plane indicator, ghost display object, etc.).
At block 1410, the system 100 determines an emplacement of a second display object based at least in part on received second emplacement data. In some embodiments, the second display object can be a medical image. In some embodiments, display object can be any medical display objects (e.g., virtual medical device, medical image, ghost display object, etc.) and/or any image guidance cue (e.g., trajectory indicator, intersection indicator, plane indicator, ghost display object, etc.).
At block 1412, as also described above with reference to block 1310 of FIG. 13 , the system 100 determines an emplacement of a medical image (e.g., medical image) based at least in part on received second emplacement data.
At block 1414, the system 100 determines an emplacement of a plane-of-interest. In some embodiments, the plane-of-interest is the plane to which that the display object is to be projected. For example, the plane-of-interest can be an image plane (e.g., the plane corresponding to a medical image), a medical device plane (e.g., the plane corresponding to a virtual medical device), or any other display object plane or other desired plane. In some embodiments, the plane-of-interest can include only a portion of a plane (e.g., the medical image portion of the image plane). However, it will be understood that a plane-of-interest can include more than a portion of any plane including but not limited to the image plane, medical device plane, affected region plane, or other display object plane. In some embodiments, the plane-of-interest can be determined dynamically based on the display object plane. Further, in some cases, the plane-of-interest can be selected to be a plane that is parallel to the display object plane. For example, the plane-of-interest can be selected as the display object plane (as described below) that is parallel to the image plane of a medical image. However, it will be understood that in some cases, the plane-of-interest can be predetermined, such as a display object plane that bisects the display object into halves.
In some embodiments, the display object plane corresponds to any plane that is parallel to a primary or secondary axis of the display object. In some embodiments, a virtual medical device plane can correspond to a plane that bisects the virtual medical device lengthwise or along its longitudinal axis. For example, the plane can bisect the virtual medical device into left and right portions (non-limiting examples: halves or other division), top and bottom portions (non-limiting examples: halves or other division), upper-right and lower left portions, upper-left and lower right portions, etc. In some cases, a display object plane can be determined dynamically based on the plane-of-interest. Further, in certain cases, the display object plane can be selected to be a plane that bisects the display object and that is parallel to the plane-of-interest. For example, the display object plane can be selected as a plane (e.g., a plane corresponding to an axis of the display object) that is parallel to the plane-of-interest. Thus, the display object plane can change as the display object moves throughout the system. However, it will be understood that in some cases, the display object plane can be predetermined. The emplacement of the plane-of-interest can be determined using a similar technique as described above.
At block 1416, the system 100 determines an emplacement of a ghost display object. The emplacement of the ghost display object can be determined using a variety of techniques, as described above. For examples, a ghost object can be projected onto the plane-of-interest. A ghost display object (e.g., projected display object) can be projected onto a plane-of-interest using a variety of techniques. For example, the ghost display object can be an orthographic projection of the display object onto the plane-of-interest. As a non-limiting example, the display object is a virtual medical device and the plane-of-interest is an image plane. The system calculates a plurality of lines which are perpendicular to the image plane. For each perpendicular line that intersects the virtual medical device (or a subset thereof), a point is displayed on the image plane at the location corresponding to the intersecting perpendicular line. As a result, a plurality of points are displayed on the image plane which correspond to the plurality of perpendicular lines (or subset thereof) intersecting the virtual medical device. The plurality of points can illustrate an indication of the virtual medical device on the image plane. In some embodiments, the plurality of points can be referred to as a ghost, a shadow, an outline, a marking, a projection, or an indication of the virtual medical device. It should be understood that the number of perpendicular lines (and the number of corresponding points) can vary across embodiments. For example, in some embodiments, there can be almost infinite perpendicular lines such that a complete representation of the virtual medical device is displayed on the image plane. In some embodiments, the perpendicular lines can be spaced out at predefined intervals such that the ghost medical device includes a dotted outline or dotted representation of the medical device. Similarly, in some cases, the subset of perpendicular lines used to generate the ghost display object can correspond to lines that intersect with a particular portion of the virtual medical device, such as the edge thereby creating an outline on the plane-of-interest.
In some embodiments, a ghost display object can be projected onto a plane-of-interest using a virtual light source. Consider a virtual light source that is at least as large as the display object to be projected or that covers an entire plane (e.g., a light source plane) positioned behind the display object (e.g., distal the plane-of-interest relative to the display object) and parallel to the plane-of-interest, and that emits parallel light waves. In examples such as these, the ghost display object can be displayed on the plane-on-interest as a shadow of the display object. Because the light source is infinitely wide (and can be infinitely far away), the size of the ghost display object can correspond directly with the size of the display object. Further, it will be understood that in some cases, the system can display an outline of the shadow generated by the virtual light source or other indicator.
In some embodiments, a display object can be projected by identifying points on a display object plane that correspond (e.g., intersect) to the display and mapping (e.g., projecting, reproducing, copying) the identified points onto a plane-of-interest. In some embodiments, the identified points on the display object plane can include a plurality of points (e.g., coordinates) that intersect with the corresponding display object. For instance, the plane of a virtual medical device can be a plane that bisects the virtual medical device. The plane can include a plurality of points that intersect with the virtual medical device. In a non-limiting example, to project the virtual medical device onto an image plane, at least some of the plurality of intersecting points can be drawn on the image plane. In certain cases, as described above, the resulting ghost medical device can include an outline or cross-section of the virtual medical device.
In some embodiments, a display object can be projected by identifying or determining which points (e.g., coordinates) of the plane-of-interest are closest (e.g., nearest) to points (e.g., coordinates) of the display object or points of the display object intersecting the display object plane (as described above). Each of the points corresponding to the display object (or the display object plane) can be projected onto the plane-of-interest at the location of the points of the plane-of-interest that were determined to be closest to the identified points of the display object. In some embodiments, projecting can include determining portions of the plane-of-interest that are closest to the display object. The display object can be projected onto those portions of the plane-of-interest that are closest to the display object. In some embodiments, projecting can include determining coordinates of a cross-sectional outline of the display object that are closest to the plane-of-interest.
Alternatively, or in addition, a medical image can be projected onto a plane of another medical device (e.g., a multi-axis medical device). In certain embodiments, the system 100 projects at least a portion of the medical image onto a virtual medical device plane of a virtual medical device based at least in part on the determined emplacement of the virtual medical device and the determined emplacement of the medical image and/or medical device associated with the medical image (e.g., a medical imaging device such as an ultrasound transducer). The ghost medical image can be displayed in a 2D view or 3D view and/or as a perspective view.
At steps 1418-1422, as similarly described above with respect to blocks 1316-1320 of FIG. 13 , the system 100 can display a 3D rendering of at least a portion of the first display object, the second display object, or the medical image.
At step 1424, the system can display a 3D rendering of at least a portion of the ghost display object. In some embodiments, a ghost display object is an outline or other indication of all or a portion of the display object. In some embodiments, the ghost display object is a cross-sectional outline of the display object or a portion thereof. In some embodiments, a ghost display object is a projection of a full or partial representation of the display object. For example, a ghost medical device can include a representation of medical device having a first limb member, the second limb member, and/or joint member on the image plane. In some embodiments, the system can display a rendering of at least a portion of the medical image projected onto the plane-of-interest.
It will be understood that the various blocks described herein can be implemented in a variety of orders, and that the system 100 can implement one or more of the blocks concurrently and/or change the order, as desired. For example, the system 100 can concurrently receive the emplacement data from different sources, concurrently receive the medical image, or receive the data in any order. Similarly, the system 100 can concurrently determine the emplacement of the medical image and/or one or more virtual medical devices, etc.
Furthermore, it will be understood that fewer, more, or different blocks can be used as part of the routine 1400. For example, the routine 1400 can include blocks for receiving emplacement data associated with additional emplacement sensors or medical devices, determining emplacements of one or more medical devices, corresponding virtual medical devices, other display objects, displays, and/or users. In some embodiments, the routine 1400 can include determining an emplacement of a medical device associated with the medical image and determining the emplacement of the medical image based at least in part on the determined emplacement of the medical device (or corresponding virtual medical device), and display the virtual medical device concurrently with the medical image. Furthermore, the system 100 can determine and display a variety of image guidance cues, such as trajectory indicators, affected region indicators, as described above. In addition, in some cases, the routine 1400 can omit certain blocks, such as, but not limited to, blocks 1402, 1404, 1406, 1410, 1412, 1420, and/or 1422. For example, in some embodiments, the system may not determine emplacement of or display a portion of the second display object and/or the medical image.
In addition, it will be understood that the various blocks described herein with reference to FIG. 13 can be implemented in routine 1400, in a variety of orders. For example, the system 100 can implement some or all of various blocks of FIG. 13 in routine 1400 concurrently or change the order as desired.
At block 1502, as also described above with reference to block 1302 of FIG. 13 , the system 100 receives first emplacement data associated with a first emplacement sensor and/or a first medical device. In some embodiments, the first emplacement sensor can be associated with the first medical device. In some embodiments, the first medical device can be a multi-axis medical device. Furthermore, the first medical device can have a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis. As described above, one or more emplacement sensors can be associated with a first limb member, a second limb member, and/or a joint member of the associated first medical device.
At block 1504, the system 100 receives operating parameters of a medical device. As described in greater detail above, the operating parameters can include information regarding make and model, power, dimensions of the medical device (such as length or width of the first limb member and/or the second limb member, angle data, distance from the hinge), range of movement data, etc. The operating parameters can be stored in a non-transitory, computer-readable medium associated with the system 100 and/or can be stored in the medical device. The operating parameters can be manually entered and/or automatically computed by the guidance system.
At block 1506, as also described above with reference to block 1306 of FIG. 13 , the system 100 can determine an emplacement of a first virtual medical device based at least in part on the received first emplacement data. As described above, the second virtual medical device can correspond to a multi-axis device such as a transecting stapler, grasper, or vessel sealer. In some embodiments, the virtual medical device corresponds to a single-axis device such as an ablation probe.
At block 1508, the system 100 can determine an emplacement of an affected region. In some embodiments, the system determines the affected region based at least in part on the operating parameters and/or measured parameters. As described previously, an affected region can correspond to a region of tissue that will be affected when the device is operated. As a non-limiting example, the first affected region can correspond to a volume located between the first limb member and the second limb member. In some examples, the first affected region can be one or more sub-regions of the region affected when the medical device (e.g., multi-axis medical device) is operated. For example, the affected region can correspond to the plane at which the transecting knife cuts and/or the one or more planes corresponding to the one or more staples. The emplacement of the affected region can be determined in a similar manner as described above with respect to determining emplacement of a medical image, virtual medical device, ghost display object, etc.
At block 1510, similar to block 1316 of FIG. 13 , the system 100 can cause the one or more displays to display a view of a virtual 3D scene including a perspective rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device. In some embodiments, the 3D rendering of the at least a portion of the first virtual medical device includes at least a portion of the first limb member and at least a portion of the second limb member. However, it will be understood that the 3D rendering of the at least a portion of the first virtual medical device can include any portions of the first virtual device.
At block 1512, the system 100 causes the one or more displays to display at least a portion of the affected region. As described previously, the displayed affected region can be displayed in a 2D view or 3D view and/or as a perspective view. In addition, as described in greater detail above, the displayed affected region can be displayed as a volume, area, and/or line. The displayed affected region can be wire-framed, transparent, semi-transparent, have varied opacity, brightness, and/or focus, include alternating bands/tiles, be textured, include solid or dashed lines, include spikes, etc. In certain embodiments, the at least a portion of the affected region corresponds to portions of the affected region that are unique to it, with respect to other affected regions. In some embodiments, the displayed affected region corresponds to at least a portion of the affected region that is co-located with at least a portion of a medical display object (non-limiting examples: a virtual medical device, medical image, etc.) or its trajectory, also referred to as the surface display region.
At block 1514, the system 100 causes the one or more displays to display an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path. As described above, for example with respect to block 1312 of FIG. 3 , the drive path can correspond to one of a plurality of paths. For example, in some embodiments, the drive path is parallel to a joint member axis.
In some embodiments (non-limiting examples illustrated in FIGS. 9-12 ), the medical image or medical image can traverse the affected region of the medical device. For example, the affected region can correspond to a region of tissue that will be affected by a transecting knife and/or staples when a multi-axis device is operated. In some embodiments, the medical image is traverse (or extends across) at least a portion of the affected region.
With continued reference to FIG. 15 , it will be understood that fewer or more blocks can be included. For example, as described in greater detail above, the system 100 can receive emplacement data from one or more sensors corresponding to one or more medical devices, determine a emplacement of the medical devices and/or corresponding virtual medical devices (as a non-limiting example, the emplacement can be determined based at least in part on a point-of-view location), cause one or more displays to display the virtual medical devices and/or perspective views thereof, determine emplacement of the affected regions with respect to the virtual medical devices, determine emplacement of and display an medical image, alter the display of the medical image, display the medical display objects in a 2D view, a 3D view, and/or a perspective view, etc. In some cases, the routine 1500 can omit certain blocks, such as, but not limited to, blocks 1502, 1504, and/or 1510. For example, in some embodiments, the system may not display a portion of the virtual medical device, etc.
In addition, it will be understood that the various blocks described herein with reference to FIGS. 13 and/or 14 can be implemented in routine 1500, in a variety of orders. For example, the system 100 can implement some or all of various blocks of FIGS. 13 and/or 14 in routine 1500 concurrently or change the order as desired.
In some embodiments, such as those illustrated in FIGS. 9-12 , an ultrasound transducer (for example, a miniature ultrasound transducer) can be integrated with a multi-axis instrument such as a transecting stapler, grasper, or vessel sealer. In examples such as these, an emplacement of a virtual medical device (for example, a stapler) and the emplacement of a virtual medical image can be determined based at least in part on emplacement data associated with a single emplacement sensor.
At block 1602, as described above with reference to block 1303 of FIG. 13 , the system 100 receives first emplacement data associated with a first emplacement sensor and/or a first medical device. The first emplacement data can be generated by the first emplacement sensor and/or by a position sensing unit. In some embodiments, the first emplacement sensor can be associated with the first medical device (e.g., a stapler).
At block 1604, as described above with reference to block 1306 of FIG. 13 , the system 100 can determine an emplacement of a first virtual medical device based at least in part on the received first emplacement data. As described above, the first virtual medical device can correspond to a multi-axis device such as a transecting stapler, grasper, or vessel sealer. In some embodiments, the virtual medical device corresponds to a single-axis device such as an ablation probe.
At block 1606, similar to block 1308 of FIG. 13 , the system 100 can determine an emplacement of a second virtual medical device based at least in part on the received first emplacement data. As described above, a single sensor and/or single emplacement data can be used to determine the emplacement of the first medical device and the second medical device.
At block 1608, similar to block 1310 of FIG. 13 , the system 100 can determine an emplacement of a medical image based at least in part on the received first emplacement data. As described above, a single sensor and/or single emplacement data can be used to determine the emplacement of the first medical device, the second medical device, and the medical image. As mentioned above, in some embodiments, because the medical device (for example, the stapler) and the ultrasound transducer are integrated together, both the emplacement of the virtual medical device and the emplacement of the virtual ultrasound transducer (and medical image) can be determined based at least at least in part on the received first emplacement data.
At steps 1610-1614, as similarly described above with respect to blocks 1316-1320 of FIG. 13 , the system 100 can display a 3D rendering of at least a portion of the first display object, the second display object, or the medical image. However, it should be noted that each of these display objects are displayed based at least in part on the first emplacement data.
It will be understood that the various blocks described herein can be implemented in a variety of orders, and that the system 100 can implement one or more of the blocks concurrently and/or change the order, as desired. For example, the system 100 can concurrently receive the emplacement data from different sources, concurrently receive the medical image, or receive the data in any order. Similarly, the system 100 can concurrently determine the emplacement of the medical image and/or one or more virtual medical devices, etc.
Furthermore, it will be understood that fewer, more, or different blocks can be used as part of the routine 1600. For example, the routine 1600 can include blocks for receiving emplacement data associated with additional emplacement sensors or medical devices, determining emplacements of one or more medical devices, corresponding virtual medical devices, other display objects, displays, and/or users. In some embodiments, the routine 1600 can include determining an emplacement of a medical device associated with the medical image and determining the emplacement of the medical image based at least in part on the determined emplacement of the medical device (or corresponding virtual medical device), and display the virtual medical device concurrently with the medical image. Furthermore, the system 100 can determine and display a variety of image guidance cues, such as trajectory indicators, affected region indicators, as described above. In some cases, the routine 1600 can omit certain blocks, such as, but not limited to, blocks 1602, 1606, 1612, and/or 1614. For example, in some embodiments, the system may not determine emplacement of or display a portion of the second display object.
In addition, it will be understood that the various blocks described herein with reference to FIGS. 14 and 15 can be implemented in routine 1600, in a variety of orders. For example, the system 100 can implement some or all of various blocks of FIGS. 14 and 15 in routine 1600 concurrently or change the order as desired.
Various example embodiments of methods, systems and non-transitory computer-readable medium relating to one or more intersection indicators corresponding to an intersection between a plane-of-interest and one or more trajectories of a medical device can be found in the following clauses:
receiving second emplacement data associated with a second medical device;
determining an emplacement of a first virtual medical device corresponding to the first medical device based at least in part on the received first emplacement data;
determining an emplacement of a second virtual medical device corresponding to the second medical device based at least in part on the received second emplacement data;
determining an emplacement of a medical image based at least in part on the received second emplacement data;
determining a first intersection between an image plane and a trajectory of the first limb member along a drive path that is parallel to the joint member axis, wherein the image plane corresponds to the medical image;
determining a second intersection between the image plane and a trajectory of the second limb member along the drive path; and
causing one or more displays to concurrently display a perspective view of:
-
- a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member,
- a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device,
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image,
- a first trajectory indicator corresponding to the trajectory of the first limb member along the drive path,
- a second trajectory indicator corresponding to the trajectory of the second limb member along the drive path,
- a first intersection indicator corresponding to the first intersection, and
- a second intersection indicator corresponding to the second intersection.
Clause 2. A method for medical device navigation, the method comprising:
determining an emplacement of a first virtual medical device corresponding to a first medical device, the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
determining an emplacement of a medical image;
determining a first intersection between an image plane corresponding to the medical image and a trajectory of the first limb member along a drive path;
determining a second intersection between the image plane and a trajectory of the second limb member along the drive path; and
causing one or more displays to concurrently display a perspective view of:
-
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image,
- a first intersection indicator corresponding to the first intersection, and
- a second intersection indicator corresponding to the second intersection.
Clause 3. The method of Clause 2, further comprising:
receiving first emplacement data associated with the first medical device;
receiving second emplacement data associated with a second medical device; and
determining an emplacement of a second virtual medical device corresponding to the second medical device based at least in part on the received second emplacement data,
wherein said determining the emplacement of the first virtual medical device is based at least in part on the received first emplacement data and said determining the emplacement of the medical image is based at least in part on the received second emplacement data.
Clause 4. The method of any of the Clauses 2-3, wherein causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of:
a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member, or
a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device.
Clause 5. The method of any of the Clauses 2-4, wherein causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of:
a first trajectory indicator corresponding to the trajectory of the first limb member along the drive path, or
a second trajectory indicator corresponding to the trajectory of the second limb member along the drive path.
Clause 6. The method of any of the Clauses 2-5, wherein the drive path is parallel to at least one of the joint member axis, the first axis, or the second axis.
Clause 7. The method of any of the Clauses 2-6, wherein the drive path is perpendicular to at least one of the joint member axis, the first axis, or the second axis.
Clause 8. The method of any of the Clauses 2-7, wherein the drive path is predefined.
Clause 9. The method of any of the Clauses 2-8, wherein the drive path is dynamic and is based at least in part on detected movement of first medical device or the second medical device.
Clause 10. The method of any of the Clauses 2-9, wherein a primary axis of the first medical device is the joint member axis.
Clause 11. The method of any of the Clauses 2-10, wherein a secondary axis of the first medical device is the first axis and/or the second axis.
Clause 12. The method of any of the Clauses 2-11, wherein the first axis is parallel to the joint member axis.
Clause 13. The method of any of the Clauses 2-12, wherein the first axis is different from the joint member axis.
Clause 14. The method of any of the Clauses 2-13, wherein the first intersection indicator includes an outline of a cross-section of the first limb member.
Clause 15. The method of any of the Clauses 2-14, wherein the second intersection indicator includes an outline of a cross-section of the second limb member.
Clause 16. The method of any of the Clauses 2-15, wherein the first intersection indicator and/or the second intersection indicator includes a cross-sectional outline of a knife path and/or the first virtual medical device, wherein the knife path corresponds to a transecting knife of the first medical device.
Clause 17. The method of any of the Clauses 2-16, wherein the first intersection and/or the second intersection indicator includes an indication of at least a portion of the image plane.
Clause 18. The method of any of the Clauses 2-17, wherein the indication of at least a portion of the image plane comprises a box representative of a portion of the image plane.
Clause 19. The method of any of the Clauses 2-18, wherein the first trajectory indicator extends to at least a distal portion of the first limb member and corresponds to an axis along the drive path that is parallel to the joint member axis.
Clause 20. The method of any of the Clauses 2-19, wherein the first trajectory indicator corresponds to an axis that intersects with the first axis at a distal portion of the first limb member and is parallel to the joint member axis and the drive path.
Clause 21. The method of 20, wherein the second trajectory indicator extends to at least a distal portion of the second limb member and corresponds to an axis along the drive path that is different from and parallel to the joint member axis.
Clause 22. The method of any of the Clauses 2-213, wherein the second trajectory indicator corresponds to an axis that intersects with the second axis at a distal portion of the second limb member and is parallel to the joint member axis and the drive path.
Clause 23. The method of any of the Clauses 2-22, further comprising determining a cutting path associated with the first medical device, wherein causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of the cutting path.
Clause 24. The method of Clause 23, wherein the cutting path corresponds to a trajectory of a knife of the first medical device.
Clause 25. The method of any of the Clauses 2-24, further comprising:
determining an emplacement of an affected region based at least in part on the determined emplacement of the first virtual medical device,
wherein causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of:
-
- a 3D rendering of at least a portion of the affected region based at least in part on the determined emplacement of the virtual medical device, or
- an affected region intersection indicator indicating an intersection between a trajectory of the affected region along the drive path.
Clause 26. The method of Clause 25, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
Clause 27. The method of Clause 25, wherein the at least a portion of the affected region comprises a stapling path of at least one staple of the first medical device.
Clause 28. The method of Clause 25, wherein the at least a portion of the affected region comprises a cutting path of a knife of the first medical device.
Clause 29. The method of Clause 25, further comprising determining an intersection between the affected region and the medical image.
Clause 30. The method of Clause 29, wherein the at least a portion of the affected region comprises the intersection between the affected region and the medical image.
Clause 31. The method of Clause 25, further comprising receiving operating parameters corresponding to the first medical device, wherein determining the emplacement of an affected region is based at least in part on the received operating parameters.
Clause 32. The method of any of the Clauses 2-31, further comprising:
determining an emplacement of a plane-of-interest based at least in part on the determined emplacement of one of the medical image; and
determining an emplacement of a ghost medical device based at least in part on the determined emplacement of the first virtual medical device and the determined emplacement of the plane-of-interest,
wherein causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of:
-
- a 3D rendering of at least a portion of the ghost medical device on the plane-of-interest based at least in part on the determined emplacement of the ghost medical device.
Clause 33. The method of any of the Clauses 2-32, further comprising:
determining an emplacement of a plane-of-interest based at least in part on the determined emplacement of one of the first virtual medical device; and
determining an emplacement of a ghost medical image based at least in part on the determined emplacement of the medical image and the determined emplacement of the plane-of-interest,
wherein causing the one or more displays to concurrently display a perspective view further comprises causing concurrent display of:
-
- a 3D rendering of at least a portion of the ghost medical image on the plane-of-interest based at least in part on the determined emplacement of the ghost medical image.
Clause 34. The method of any of Clauses 32 or 33, further comprising projecting the display object on the plane-of-interest to display the ghost display object.
Clause 35. The method of Clause 34, wherein projecting comprises determining at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
Clause 36. The method of Clause 34, wherein projecting comprises determining an arc path corresponding to the display object and the plane-of-interest.
Clause 37. The method of Clause 34, wherein projecting comprises determining which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
Clause 38. The method of Clause 34, wherein projecting comprises using a virtual light source that is at least as large as the display object to be projected, wherein the light source is distal the plane-of-interest relative to the display object.
Clause 39. The method of Clause 32, wherein the ghost display object includes a cross-section and/or outline of display object.
Clause 40. The method of Clause 32, wherein the ghost display object includes a cross-section and/or outline of display object.
Clause 41. A system comprising:
a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
-
- receive first emplacement data associated with a first medical device, the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis,
- receive second emplacement data associated with a second medical device;
- determine an emplacement of a first virtual medical device corresponding to the first medical device based at least in part on the received first emplacement data,
- determine an emplacement of a second virtual medical device corresponding to the second medical device based at least in part on the received second emplacement data,
- determine an emplacement of a medical image based at least in part on the received second emplacement data,
- determine a first intersection between an image plane and a trajectory of the first limb member along a drive path that is parallel to the joint member axis, wherein the image plane corresponds to the medical image,
- determine a second intersection between the image plane and a trajectory of the second limb member along the drive path, and
- cause the display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member,
- a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device,
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image,
- a first trajectory indicator corresponding to the trajectory of the first limb member along the drive path,
- a second trajectory indicator corresponding to the trajectory of the second limb member along the drive path,
- a first intersection indicator corresponding to the first intersection, and
- a second intersection indicator corresponding to the second intersection;
and
the display.
Clause 42. A system comprising:
a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
-
- determine an emplacement of a first virtual medical device corresponding to a first medical device, the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis,
- determine an emplacement of a medical image,
- determine a first intersection between an image plane corresponding to the medical image and a trajectory of the first limb member along a drive path,
- determine a second intersection between the image plane and a trajectory of the second limb member along the drive path, and
- cause the display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image,
- a first intersection indicator corresponding to the first intersection, and
- a second intersection indicator corresponding to the second intersection; and
the display.
Clause 43. The system of the Clause 42, wherein the computer system is further configured to:
receive first emplacement data associated with the first medical device;
receive second emplacement data associated with a second medical device;
determine an emplacement of a second virtual medical device corresponding to the second medical device based at least in part on the received second emplacement data;
determine the emplacement of the first virtual medical device based at least in part on the received first emplacement data; and
determine the emplacement of the medical image is based at least in part on the received second emplacement data.
Clause 44. The system of any of clauses 42 or 43, wherein the computer system is further configured to cause the display to concurrently display a perspective view of:
a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member, or
a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device.
Clause 45. The system of any of clauses 42-44, wherein the computer system is further configured to cause the display to concurrently display a perspective view of:
a first trajectory indicator corresponding to the trajectory of the first limb member along the drive path, or
a second trajectory indicator corresponding to the trajectory of the second limb member along the drive path.
Clause 46. The system of any of clauses 42-45, wherein the drive path is parallel to at least one of the joint member axis, the first axis, or the second axis.
Clause 47. The system of any of clauses 42-46, wherein the drive path is perpendicular to at least one of the joint member axis, the first axis, or the second axis.
Clause 48. The system of any of clauses 42-47, wherein the drive path is predefined.
Clause 49. The system of any of clauses 42-48, wherein the drive path is dynamic and is based at least in part on detected movement of first medical device or the second medical device.
Clause 50. The system of any of clauses 42-49, wherein a primary axis of the first medical device is the joint member axis.
Clause 51. The system of any of clauses 42-50, wherein a secondary axis of the first medical device is the first axis and/or the second axis.
Clause 52. The system of any of clauses 42-51, wherein the first axis is parallel to the joint member axis.
Clause 53. The system of any of clauses 42-52, wherein the first axis is different from the joint member axis.
Clause 54. The system of any of clauses 42-53, wherein the first intersection indicator includes an outline of a cross-section of the first limb member.
Clause 55. The system of any of clauses 42-54, wherein the second intersection indicator includes an outline of a cross-section of the second limb member.
Clause 56. The system of any of clauses 42-55, wherein the first intersection indicator and/or the second intersection indicator includes a cross-sectional outline of a knife path and/or the first virtual medical device, wherein the knife path corresponds to a transecting knife of the first medical device.
Clause 57. The system of any of clauses 42-56, wherein the first intersection and/or the second intersection indicator includes an indication of at least a portion of the image plane.
Clause 58. The system of clause 57, wherein the indication of at least a portion of the image plane comprises a box representative of a portion of the image plane.
Clause 59. The system of clause 45, wherein the first trajectory indicator extends to at least a distal portion of the first limb member and corresponds to an axis along the drive path that is parallel to the joint member axis.
Clause 60. The system of clause 45, wherein the first trajectory indicator corresponds to an axis that intersects with the first axis at a distal portion of the first limb member and is parallel to the joint member axis and the drive path.
Clause 61. The system of clause 45, wherein the second trajectory indicator extends to at least a distal portion of the second limb member and corresponds to an axis along the drive path that is different from and parallel to the joint member axis.
Clause 62. The system of clause 45, wherein the second trajectory indicator corresponds to an axis that intersects with the second axis at a distal portion of the second limb member and is parallel to the joint member axis and the drive path.
Clause 63. The system of any of clauses 42-62, wherein the computer system is further configured to determine a cutting path associated with the first medical device and cause the display to concurrently display a perspective view of the cutting path.
Clause 64. The system of clause 63, wherein the cutting path corresponds to a trajectory of a knife of the first medical device.
Clause 65. The system of any of clauses 42-64, wherein the computer system is further configured to:
determine an emplacement of an affected region based at least in part on the determined emplacement of the first virtual medical device; and
causing the display to concurrently display a perspective view of:
-
- a 3D rendering of at least a portion of the affected region based at least in part on the determined emplacement of the virtual medical device, or
- an affected region intersection indicator indicating an intersection between a trajectory of the affected region along the drive path.
Clause 66. The system of clause 65, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
Clause 67. The system of clause 65, wherein the at least a portion of the affected region comprises a stapling path of at least one staple of the first medical device.
Clause 68. The system of clause 65, wherein the at least a portion of the affected region comprises a cutting path of a knife of the first medical device.
Clause 69. The system of clause 65, wherein the at least a portion of the affected region comprises the intersection between the affected region and the medical image.
Clause 70. The system of any of clauses 42-69, wherein the computer system is further configured to determine an intersection between the affected region and the medical image.
Clause 71. The system of any of clauses 42-70, wherein the computer system is further configured to receive operating parameters corresponding to the first medical device and determine the emplacement of an affected region based at least in part on the received operating parameters.
Clause 72. The system of any of clauses 42-70, wherein the computer system is further configured to:
determine an emplacement of a plane-of-interest based at least in part on the determined emplacement of one of the medical image;
determine an emplacement of a ghost medical device based at least in part on the determined emplacement of the first virtual medical device and the determined emplacement of the plane-of-interest; and
cause the display to concurrently display a perspective view of a 3D rendering of at least a portion of the ghost medical device on the plane-of-interest based at least in part on the determined emplacement of the ghost medical device.
Clause 73. The system of any of clauses 42-70, wherein the computer system is further configured to:
determine an emplacement of a plane-of-interest based at least in part on the determined emplacement of one of the first virtual medical device;
determine an emplacement of a ghost medical image based at least in part on the determined emplacement of the medical image and the determined emplacement of the plane-of-interest; and
cause the display to currently display a perspective view of a 3D rendering of at least a portion of the ghost medical image on the plane-of-interest based at least in part on the determined emplacement of the ghost medical image.
Clause 74. The system of any one of clause 72 or 73, wherein the computer system is further configured to project the display object on the plane-of-interest to display the ghost display object.
Clause 75. The system of the Clause 74, wherein the computer system is further configured to determine at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
Clause 76. The system of any of clauses 74 or 75, wherein the computer system is further configured to determine an arc path corresponding to the display object and the plane-of-interest.
Clause 77. The system of any of clauses 74-76, wherein the computer system is further configured to determine which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
Clause 78. The system of any of clauses 74-77, wherein the computer system further comprises a virtual light source that is at least as large as the display object to be projected, wherein the light source is distal the plane-of-interest relative to the display object.
Clause 79. The system of any of clauses 74-78, wherein the ghost display object includes a cross-section and/or outline of display object.
Clause 80. The system of any of clauses 74-79, wherein the ghost display object includes a cross-section and/or outline of display object.
Clause 81. A non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
-
- receive first emplacement data associated with a first medical device, the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis,
- receive second emplacement data associated with a second medical device;
- determine an emplacement of a first virtual medical device corresponding to the first medical device based at least in part on the received first emplacement data,
- determine an emplacement of a second virtual medical device corresponding to the second medical device based at least in part on the received second emplacement data,
- determine an emplacement of a medical image based at least in part on the received second emplacement data,
- determine a first intersection between an image plane and a trajectory of the first limb member along a drive path that is parallel to the joint member axis, wherein the image plane corresponds to the medical image,
- determine a second intersection between the image plane and a trajectory of the second limb member along the drive path, and
- cause the display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member,
- a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device,
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image,
- a first trajectory indicator corresponding to the trajectory of the first limb member along the drive path,
- a second trajectory indicator corresponding to the trajectory of the second limb member along the drive path,
- a first intersection indicator corresponding to the first intersection, and
- a second intersection indicator corresponding to the second intersection.
Clause 82. A non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
-
- determine an emplacement of a first virtual medical device corresponding to a first medical device, the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis,
- determine an emplacement of a medical image,
- determine a first intersection between an image plane corresponding to the medical image and a trajectory of the first limb member along a drive path,
- determine a second intersection between the image plane and a trajectory of the second limb member along the drive path, and
- cause the display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image,
- a first intersection indicator corresponding to the first intersection, and
- a second intersection indicator corresponding to the second intersection.
Clause 83. The non-transitory computer-readable medium of clause 82, wherein the computer executable instructions further cause the one or more processors to:
receive first emplacement data associated with the first medical device;
receive second emplacement data associated with a second medical device;
determine an emplacement of a second virtual medical device corresponding to the second medical device based at least in part on the received second emplacement data;
determine the emplacement of the first virtual medical device based at least in part on the received first emplacement data; and
determine the emplacement of the medical image is based at least in part on the received second emplacement data.
Clause 84. The non-transitory computer-readable medium of clauses 82 or 83, wherein the computer executable instructions further cause the one or more processors to cause the display to concurrently display a perspective view of:
a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member, or
a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device.
Clause 85. The non-transitory computer-readable medium of clauses 82-84, wherein the computer executable instructions further cause the one or more processors to cause the display to concurrently display a perspective view of:
a first trajectory indicator corresponding to the trajectory of the first limb member along the drive path, or
a second trajectory indicator corresponding to the trajectory of the second limb member along the drive path.
Clause 86. The non-transitory computer-readable medium of any of clauses 82-85, wherein the drive path is parallel to at least one of the joint member axis, the first axis, or the second axis.
Clause 87. The non-transitory computer-readable medium of any of clauses 82-86, wherein the drive path is perpendicular to at least one of the joint member axis, the first axis, or the second axis.
Clause 88. The non-transitory computer-readable medium of any of clauses 82-87, wherein the drive path is predefined.
Clause 89. The non-transitory computer-readable medium of any of clauses 82-88, wherein the drive path is dynamic and is based at least in part on detected movement of first medical device or the second medical device.
Clause 90. The non-transitory computer-readable medium of any of clauses 82-89, wherein a primary axis of the first medical device is the joint member axis.
Clause 91. The non-transitory computer-readable medium of any of clauses 82-90, wherein a secondary axis of the first medical device is the first axis and/or the second axis.
Clause 92. The non-transitory computer-readable medium of any of clauses 82-91, wherein the first axis is parallel to the joint member axis.
Clause 93. The non-transitory computer-readable medium of any of clauses 82-92, wherein the first axis is different from the joint member axis.
Clause 94. The non-transitory computer-readable medium of any of clauses 82-93, wherein the first intersection indicator includes an outline of a cross-section of the first limb member.
Clause 95. The non-transitory computer-readable medium of any of clauses 82-94, wherein the second intersection indicator includes an outline of a cross-section of the second limb member.
Clause 96. The non-transitory computer-readable medium of any of clauses 82-95, wherein the first intersection indicator and/or the second intersection indicator includes a cross-sectional outline of a knife path and/or the first virtual medical device, wherein the knife path corresponds to a transecting knife of the first medical device.
Clause 97. The non-transitory computer-readable medium of any of clauses 82-96, wherein the first intersection and/or the second intersection indicator includes an indication of at least a portion of the image plane.
Clause 98. The non-transitory computer-readable medium of clause 97, wherein the indication of at least a portion of the image plane comprises a box representative of a portion of the image plane.
Clause 99. The non-transitory computer-readable medium of clause 85, wherein the first trajectory indicator extends to at least a distal portion of the first limb member and corresponds to an axis along the drive path that is parallel to the joint member axis.
Clause 101. The non-transitory computer-readable medium of clause 85, wherein the second trajectory indicator extends to at least a distal portion of the second limb member and corresponds to an axis along the drive path that is different from and parallel to the joint member axis.
Clause 103. The non-transitory computer-readable medium of any of clauses 82-102, wherein the computer system is further configured to determine a cutting path associated with the first medical device and cause the display to concurrently display a perspective view of the cutting path.
Clause 104. The non-transitory computer-readable medium of clause 103, wherein the cutting path corresponds to a trajectory of a knife of the first medical device.
Clause 105. The non-transitory computer-readable medium of any of clauses 82-104, wherein the computer executable instructions further cause the one or more processors to:
determine an emplacement of an affected region based at least in part on the determined emplacement of the first virtual medical device; and
causing the display to concurrently display a perspective view of:
-
- a 3D rendering of at least a portion of the affected region based at least in part on the determined emplacement of the virtual medical device, or
- an affected region intersection indicator indicating an intersection between a trajectory of the affected region along the drive path.
Clause 106. The non-transitory computer-readable medium of clause 105, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
Clause 107. The non-transitory computer-readable medium of clause 105, wherein the at least a portion of the affected region comprises a stapling path of at least one staple of the first medical device.
Clause 108. The non-transitory computer-readable medium of clause 105, wherein the at least a portion of the affected region comprises a cutting path of a knife of the first medical device.
Clause 109. The non-transitory computer-readable medium of clause 105, wherein the at least a portion of the affected region comprises the intersection between the affected region and the medical image.
Clause 110. The non-transitory computer-readable medium of any of clauses 82-109, wherein the computer executable instructions further cause the one or more processors to determine an intersection between the affected region and the medical image.
Clause 111. The non-transitory computer-readable medium of any of clauses 82-110, wherein the computer executable instructions further cause the one or more processors to receive operating parameters corresponding to the first medical device and determine the emplacement of an affected region based at least in part on the received operating parameters.
Clause 112. The non-transitory computer-readable medium of any of clauses 82-110, wherein the computer executable instructions further cause the one or more processors to:
determine an emplacement of a plane-of-interest based at least in part on the determined emplacement of one of the medical image;
determine an emplacement of a ghost medical device based at least in part on the determined emplacement of the first virtual medical device and the determined emplacement of the plane-of-interest; and
cause the display to concurrently display a perspective view of a 3D rendering of at least a portion of the ghost medical device on the plane-of-interest based at least in part on the determined emplacement of the ghost medical device.
Clause 113. The non-transitory computer-readable medium of any of clauses 82-110, wherein the computer executable instructions further cause the one or more processors to:
determine an emplacement of a plane-of-interest based at least in part on the determined emplacement of one of the first virtual medical device;
determine an emplacement of a ghost medical image based at least in part on the determined emplacement of the medical image and the determined emplacement of the plane-of-interest; and
cause the display to currently display a perspective view of a 3D rendering of at least a portion of the ghost medical image on the plane-of-interest based at least in part on the determined emplacement of the ghost medical image.
Clause 114. The non-transitory computer-readable medium of any one of clause 112 or 73, wherein the computer executable instructions further cause the one or more processors to project the display object on the plane-of-interest to display the ghost display object.
Clause 115. The non-transitory computer-readable medium of Clause 114, wherein the computer executable instructions further cause the one or more processors to determine at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
Clause 116. The non-transitory computer-readable medium of any of clauses 114 or 115, wherein the computer executable instructions further cause the one or more processors to determine an arc path corresponding to the display object and the plane-of-interest.
Clause 117. The non-transitory computer-readable medium of any of clauses 114-116, wherein the computer executable instructions further cause the one or more processors to determine which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
Clause 118. The non-transitory computer-readable medium of any of clauses 114-117, wherein the computer executable instructions further cause the one or more processors to use a virtual light source that is at least as large as the display object to be projected, wherein the light source is distal the plane-of-interest relative to the display object.
Clause 119. The non-transitory computer-readable medium of any of clauses 114-118, wherein the ghost display object includes a cross-section and/or outline of display object.
Clause 120. The non-transitory computer-readable medium of any of clauses 114-119, wherein the ghost display object includes a cross-section and/or outline of display object.
Various example embodiments of methods, systems and non-transitory computer-readable medium relating to projecting a display object onto a plane-of-interest to display a ghost display object can be found in the following clauses:
receiving first emplacement data associated with a first medical device;
receiving second emplacement data associated with a second medical device;
receiving a medical image corresponding to the second medical device;
determining an emplacement of a first display object corresponding to the first medical device based at least in part on the received first emplacement data;
determining an emplacement of a second display object corresponding to the second medical device based at least in part on the received second emplacement data;
determining an emplacement of the medical image based at least in part on the received second emplacement data;
determining an emplacement of a plane-of-interest based at least in part on the emplacement of one of the first display object or the medical image;
determining an emplacement of a ghost display object based at least in part on the determined emplacement of the other of the first display object or the medical image display object and the determined emplacement of the plane-of-interest; and
causing one or more displays to concurrently display a perspective view of:
-
- a 3D rendering of at least a portion of the first display object based at least in part on the determined emplacement of the first display object,
- a 3D rendering of at least a portion of the second display object based at least in part on the determined emplacement of the second display object,
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image,
- a 3D rendering of at least a portion of the ghost display object on the plane-of-interest based at least in part on the determined emplacement of the ghost display object.
Clause 2. A method for medical instrument navigation, the method comprising:
determining an emplacement of a display object;
determining an emplacement of a plane-of-interest;
determining an emplacement of a ghost display object based at least in part on the determined emplacement of the display object and the determined emplacement the plane-of-interest; and
causing one or more displays to display concurrently display:
-
- a 3D rendering of at least a portion of the display object, and
- a 3D rendering of at least a portion of the ghost display object on the plane-of-interest.
Clause 3. The method of Clause 2, further comprising:
receiving first emplacement data associated with a first medical device;
receiving second emplacement data associated with a second medical device;
wherein said determining the emplacement of the display object is based at least in part on the received first emplacement data and said determining the emplacement of the plane-of-interest is based at least in part on the received second emplacement data.
Clause 4. The method of Clause 3, further comprising
receiving a medical image corresponding to the second medical device; and
determining an emplacement of the medical image based at least in part on the received second emplacement data,
wherein said determining the emplacement of the plane-of-interest is based at least in part on the determined emplacement of the medical image.
Clause 5. The method of Clause 3, further comprising
receiving a medical image corresponding to the first medical device; and
determining an emplacement of the medical image based at least in part on the received first emplacement data,
wherein said determining the emplacement of the display object is based at least in part on the determined emplacement of the medical image.
Clause 6. The method of any of clauses 2-5, further comprising projecting the display object on the plane-of-interest to display the ghost display object.
Clause 7. The method of any of clauses 6-6, wherein said projecting comprises determining at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
Clause 8. The method of any of clauses 6-7, wherein said projecting comprises determining an arc path corresponding to the display object and the plane-of-interest.
Clause 9. The method of any of clauses 6-8, wherein said projecting comprises determining which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
Clause 10. The method of any of clauses 6-9, wherein said projecting comprises using a virtual light source that is at least as large as the display object to be projected, wherein the virtual light source is distal to the plane-of-interest relative to the display object.
Clause 11. The method of any of clauses 2-10, wherein the ghost display object includes a cross-section and/or outline of display object.
Clause 12. The method of any of clauses 2-11, wherein the ghost display object corresponds to at least one of a virtual medical device, a medical image, or an affected region.
Clause 13. The method of any of clauses 2-12, wherein the plane-of-interest corresponds to at least one of a medical device plane or an image plane.
Clause 14. The method of any of clauses 2-13, wherein the display object is an affected region and wherein the affected region corresponds to a volume located between a first limb member and a second limb member of a medical device.
Clause 15. The method of any of clauses 2-14, wherein the display object is an affected region and wherein the affected region comprises a stapling path of at least one staple of a medical device.
Clause 16. The method of any of clauses 2-15, wherein the display object is an affected region and wherein the affected region comprises a cutting path of a knife of a medical device.
Clause 17. A system comprising:
a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
receive first emplacement data associated with a first medical device,
receive second emplacement data associated with a second medical device,
receive a medical image corresponding to the second medical device,
determine an emplacement of a first display object corresponding to the first medical device based at least in part on the received first emplacement data,
determine an emplacement of a second display object corresponding to the second medical device based at least in part on the received second emplacement data,
determine an emplacement of the medical image based at least in part on the received second emplacement data,
determine an emplacement of a plane-of-interest based at least in part on the emplacement of one of the first display object or the medical image,
determine an emplacement of a ghost display object based at least in part on the determined emplacement of the other of the first display object or the medical image display object and the determined emplacement of the plane-of-interest, and
cause a display to concurrently display a perspective view of:
-
- a 3D rendering of at least a portion of the first display object based at least in part on the determined emplacement of the first display object,
- a 3D rendering of at least a portion of the second display object based at least in part on the determined emplacement of the second display object,
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image,
- a 3D rendering of at least a portion of the ghost display object on the plane-of-interest based at least in part on the determined emplacement of the ghost display object; and
the display.
Clause 18. A system comprising:
a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
determine an emplacement of a display object,
determine an emplacement of a plane-of-interest,
determine an emplacement of a ghost display object based at least in part on the determined emplacement of the display object and the determined emplacement the plane-of-interest, and
cause one or more displays to display concurrently display:
-
- a 3D rendering of at least a portion of the display object, and
- a 3D rendering of at least a portion of the ghost display object on the plane-of-interest; and
the display.
Clause 19. The system of Clause 18, wherein the computer system is further configured to:
receive first emplacement data associated with a first medical device;
receiving second emplacement data associated with a second medical device;
wherein said determining the emplacement of the display object is based at least in part on the received first emplacement data and said determining the emplacement of the plane-of-interest is based at least in part on the received second emplacement data.
The system of Clause 19, wherein the computer system is further configured to:
receive a medical image corresponding to the second medical device;
determine an emplacement of the medical image based at least in part on the received second emplacement data; and
determine the emplacement of the plane-of-interest based at least in part on the determined emplacement of the medical image.
Clause 20. The system of Clause 19, wherein the computer system is further configured to:
receive a medical image corresponding to the first medical device;
determine an emplacement of the medical image based at least in part on the received first emplacement data; and
determine the emplacement of the display object based at least in part on the determined emplacement of the medical image.
Clause 21. The system of any of clauses 18-21, wherein the computer system is further configured to project the display object on the plane-of-interest to display the ghost display object.
Clause 22. The system of any of clauses 21-22, wherein the computer system is further configured to determine at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
Clause 23. The system of any of clauses 21-23, wherein the computer system is further configured to determine an arc path corresponding to the display object and the plane-of-interest.
Clause 24. The system of any of clauses 21-24, wherein the computer system is further configured to determine which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
Clause 25. The system of any of clauses 21-25, wherein the computer system is further configured to use a virtual light source that is at least as large as the display object to be projected, wherein the virtual light source is distal to the plane-of-interest relative to the display object.
Clause 26. The system of any of clauses 18-26, wherein the ghost display object includes a cross-section and/or outline of display object.
Clause 27. The system of any of clauses 18-27, wherein the ghost display object corresponds to at least one of a virtual medical device, a medical image, or an affected region.
Clause 28. The system of any of clauses 18-28, wherein the plane-of-interest corresponds to at least one of a medical device plane or an image plane.
Clause 29. The system of any of clauses 18-29, wherein the display object is an affected region and wherein the affected region corresponds to a volume located between a first limb member and a second limb member of a medical device.
Clause 30. The system of any of clauses 18-30, wherein the display object is an affected region and wherein the affected region comprises a stapling path of at least one staple of a medical device.
Clause 31. The system of any of clauses 18-31, wherein the display object is an affected region and wherein the affected region comprises a cutting path of a knife of a medical device.
Clause 32. A non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
receive first emplacement data associated with a first medical device,
receive second emplacement data associated with a second medical device,
receive a medical image corresponding to the second medical device,
determine an emplacement of a first display object corresponding to the first medical device based at least in part on the received first emplacement data,
determine an emplacement of a second display object corresponding to the second medical device based at least in part on the received second emplacement data,
determine an emplacement of the medical image based at least in part on the received second emplacement data,
determine an emplacement of a plane-of-interest based at least in part on the emplacement of one of the first display object or the medical image,
determine an emplacement of a ghost display object based at least in part on the determined emplacement of the other of the first display object or the medical image display object and the determined emplacement of the plane-of-interest, and
cause a display to concurrently display a perspective view of:
-
- a 3D rendering of at least a portion of the first display object based at least in part on the determined emplacement of the first display object,
- a 3D rendering of at least a portion of the second display object based at least in part on the determined emplacement of the second display object,
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image,
- a 3D rendering of at least a portion of the ghost display object on the plane-of-interest based at least in part on the determined emplacement of the ghost display object.
Clause 33. A non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
determine an emplacement of a display object,
determine an emplacement of a plane-of-interest,
determine an emplacement of a ghost display object based at least in part on the determined emplacement of the display object and the determined emplacement the plane-of-interest, and
cause one or more displays to display concurrently display:
-
- a 3D rendering of at least a portion of the display object, and
- a 3D rendering of at least a portion of the ghost display object on the plane-of-interest.
Clause 34. The non-transitory computer-readable medium of Clause 18, wherein the computer executable instructions further cause the one or more processors to:
receive first emplacement data associated with a first medical device;
receiving second emplacement data associated with a second medical device;
wherein said determining the emplacement of the display object is based at least in part on the received first emplacement data and said determining the emplacement of the plane-of-interest is based at least in part on the received second emplacement data.
Clause 35. The non-transitory computer-readable medium of Clause 19, wherein the computer executable instructions further cause the one or more processors to:
receive a medical image corresponding to the second medical device;
determine an emplacement of the medical image based at least in part on the received second emplacement data; and
determine the emplacement of the plane-of-interest based at least in part on the determined emplacement of the medical image.
Clause 36. The non-transitory computer-readable medium of Clause 19, wherein the computer executable instructions further cause the one or more processors to:
receive a medical image corresponding to the first medical device;
determine an emplacement of the medical image based at least in part on the received first emplacement data; and
determine the emplacement of the display object based at least in part on the determined emplacement of the medical image.
Clause 37. The non-transitory computer-readable medium of any of clauses 18-21, wherein the computer executable instructions further cause the one or more processors to project the display object on the plane-of-interest to display the ghost display object.
Clause 38. The non-transitory computer-readable medium of any of clauses 21-22, wherein the computer executable instructions further cause the one or more processors to determine at least one of portions, points, or coordinates of the plane-of-interest that are closest to the display object.
Clause 39. The non-transitory computer-readable medium of any of clauses 21-23, wherein the computer executable instructions further cause the one or more processors to determine an arc path corresponding to the display object and the plane-of-interest.
Clause 40. The non-transitory computer-readable medium of any of clauses 21-24, wherein the computer executable instructions further cause the one or more processors to determine which of a plurality of connecting lines intersect the display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the display object.
Clause 41. The non-transitory computer-readable medium of any of clauses 21-25, wherein the computer executable instructions further cause the one or more processors to use a virtual light source that is at least as large as the display object to be projected, wherein the virtual light source is distal to the plane-of-interest relative to the display object.
Clause 42. The non-transitory computer-readable medium of any of clauses 18-26, wherein the ghost display object includes a cross-section and/or outline of display object.
Clause 43. The non-transitory computer-readable medium of any of clauses 18-27, wherein the ghost display object corresponds to at least one of a virtual medical device, a medical image, or an affected region.
Clause 44. The non-transitory computer-readable medium of any of clauses 18-28, wherein the plane-of-interest corresponds to at least one of a medical device plane or an image plane.
Clause 45. The non-transitory computer-readable medium of any of clauses 18-29, wherein the display object is an affected region and wherein the affected region corresponds to a volume located between a first limb member and a second limb member of a medical device.
Clause 46. The non-transitory computer-readable medium of any of clauses 18-30, wherein the display object is an affected region and wherein the affected region comprises a stapling path of at least one staple of a medical device.
Clause 47. The non-transitory computer-readable medium of any of clauses 18-31, wherein the display object is an affected region and wherein the affected region comprises a cutting path of a knife of a medical device.
Various example embodiments of methods, systems and non-transitory computer-readable medium relating to displaying an affected region of a medical device can be found in the following clauses:
receiving emplacement data associated with a medical device, the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
receiving operating parameters corresponding to the medical device;
determining an emplacement of a virtual medical device corresponding to the medical device based at least in part on the received emplacement data;
determining an emplacement of an affected region based at least in part on the received emplacement data and the operating parameters, wherein the affected region corresponds to a volume located between the first limb member and the second limb member; and
causing one or more displays to concurrently display a perspective view of:
-
- a 3D rendering of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device, the 3D rendering of the virtual medical device including at least the first limb member and the second limb member, and
- a 3D rendering of at least a portion of the affected region based at least in part on the determined emplacement of the affected region, or
- an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path that is parallel to the joint member axis.
Clause 2. A method for medical instrument navigation, the method comprising:
determining an emplacement of a virtual medical device corresponding to a medical device, the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
determining an emplacement of an affected region based at least in part on the determined emplacement of the virtual medical device; and
causing one or more displays to display at least one of:
-
- a 3D rendering of at least a portion of the affected region based at least in part on the determined emplacement of the virtual medical device, or
- an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path that is parallel to the joint member axis.
Clause 3. The method of Clause 2, further comprising:
receiving emplacement data associated with the medical device; and
receiving operating parameters corresponding to the medical device,
wherein said determining the emplacement of the virtual medical device is based at least in part on the received emplacement data, wherein said determining the emplacement of the affected region is based at least in part on the received emplacement data and the operating parameters.
Clause 4. The method of Clause 2, wherein the 3D rendering of the virtual medical includes at least the first limb member and the second limb member.
Clause 5. The method of Clause 2, further comprising causing the one or more displays to display a 3D rendering of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device, wherein the 3D rendering of the virtual medical device includes at least a first virtual limb member corresponding to the first limb member and a second virtual limb member corresponding to the second limb member.
Clause 6. The method of Clause 2, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
Clause 7. The method of Clause 2, wherein the affected region comprises a stapling path of at least one staple of the medical device.
Clause 8. The method of Clause 2, wherein the affected region comprises a cutting path of a knife of the medical device.
Clause 9. A system comprising:
a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
-
- receive emplacement data associated with a medical device, the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
- receive operating parameters corresponding to the medical device,
- determine an emplacement of a virtual medical device corresponding to the medical device based at least in part on the received emplacement data,
- determine an emplacement of an affected region based at least in part on the received emplacement data and the operating parameters, wherein the affected region corresponds to a volume located between the first limb member and the second limb member, and
- cause a display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device, the 3D rendering of the virtual medical device including at least the first limb member and the second limb member, and
- a 3D rendering of at least a portion of the affected region based at least in part on the determined emplacement of the affected region, or
- an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path that is parallel to the joint member axis; and
the display.
Clause 10. a system comprising:
a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
-
- determine an emplacement of a virtual medical device corresponding to a medical device, the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis,
- determine an emplacement of an affected region based at least in part on the determined emplacement of the virtual medical device, and
- cause a display to display at least one of:
- a 3D rendering of at least a portion of the affected region based at least in part on the determined emplacement of the virtual medical device, or
- an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path that is parallel to the joint member axis; and
the display.
Clause 11. The system of Clause 10, wherein the computer system is further configured to:
receive emplacement data associated with the medical device;
receive operating parameters corresponding to the medical device;
determine the emplacement of the virtual medical device based at least in part on the received emplacement data; and
determine the emplacement of the affected region based at least in part on the received emplacement data and the operating parameters.
Clause 12. The system of any of clauses 10-11, wherein the 3D rendering of the virtual medical includes at least the first limb member and the second limb member.
Clause 13. The system of any of clauses 10-12, wherein the computer system is further configured to cause the display to display a 3D rendering of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device, wherein the 3D rendering of the virtual medical device includes at least a first virtual limb member corresponding to the first limb member and a second virtual limb member corresponding to the second limb member.
Clause 14. The system of any of clauses 10-13, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
Clause 15. The system of any of clauses 10-14, wherein the affected region comprises a stapling path of at least one staple of the medical device.
Clause 16. The system of any of clauses 10-15, wherein the affected region comprises a cutting path of a knife of the medical device.
Clause 17. A non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
-
- receive emplacement data associated with a medical device, the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
- receive operating parameters corresponding to the medical device,
- determine an emplacement of a virtual medical device corresponding to the medical device based at least in part on the received emplacement data,
- determine an emplacement of an affected region based at least in part on the received emplacement data and the operating parameters, wherein the affected region corresponds to a volume located between the first limb member and the second limb member, and
- cause a display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device, the 3D rendering of the virtual medical device including at least the first limb member and the second limb member, and
- a 3D rendering of at least a portion of the affected region based at least in part on the determined emplacement of the affected region, or
- an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path that is parallel to the joint member axis.
Clause 18. A non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
determine an emplacement of a virtual medical device corresponding to a medical device, the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis,
determine an emplacement of an affected region based at least in part on the determined emplacement of the virtual medical device, and
cause a display to display at least one of:
-
- a 3D rendering of at least a portion of the affected region based at least in part on the determined emplacement of the virtual medical device, or
- an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path that is parallel to the joint member axis.
Clause 19. The non-transitory computer-readable medium of Clause 18, wherein the computer executable instructions further cause the one or more processors to:
receive emplacement data associated with the medical device;
receive operating parameters corresponding to the medical device;
determine the emplacement of the virtual medical device based at least in part on the received emplacement data; and
determine the emplacement of the affected region based at least in part on the received emplacement data and the operating parameters.
Clause 20. The non-transitory computer-readable medium of any of clauses 18-19, wherein the 3D rendering of the virtual medical includes at least the first limb member and the second limb member.
Clause 21. The non-transitory computer-readable medium of any of clauses 18-20, wherein the computer executable instructions further cause the one or more processors to cause the display to display a 3D rendering of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device, wherein the 3D rendering of the virtual medical device includes at least a first virtual limb member corresponding to the first limb member and a second virtual limb member corresponding to the second limb member.
Clause 22. The non-transitory computer-readable medium of any of clauses 18-21, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
Clause 23. The non-transitory computer-readable medium of any of clauses 18-22, wherein the affected region comprises a stapling path of at least one staple of the medical device.
Clause 24. The non-transitory computer-readable medium of any of clauses 18-23, wherein the affected region comprises a cutting path of a knife of the medical device.
Various example embodiments of methods, systems and non-transitory computer-readable medium relating to determining emplacement of a virtual medical device and a medical image based at least in part on received emplacement data associated with a medical device can be found in the following clauses:
determining an emplacement of a first virtual medical device corresponding to the first medical device based at least in part on the received emplacement data;
determining an emplacement of a second virtual medical device corresponding to a second medical device based at least in part on the received emplacement data;
determining an emplacement of a medical image based at least in part on the received emplacement data; and
causing one or more displays to concurrently display a perspective view of:
-
- a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member,
- a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device, and
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image.
Clause 2. A method for medical instrument navigation, the method comprising:
determining an emplacement of a display object based at least in part on first emplacement data;
determining an emplacement of a medical image based at least in part on the first emplacement data; and
causing one or more displays to concurrently display:
-
- a 3D rendering of at least a portion of the display object, and
- a 3D rendering of at least a portion of the medical image.
Clause 3. The method of Clause 2, wherein the display object is a first virtual medical device corresponding to a first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis.
Clause 4. The method of Clause 3, further comprising:
receiving the first emplacement data, the first emplacement data associated with the first medical device; and
determining an emplacement of a second virtual medical device corresponding to a second medical device based at least in part on the received first emplacement data.
Clause 5. The method of any of clauses 2-4, wherein the display object is a first virtual medical device and wherein causing the one or more displays to concurrently display further comprises causing concurrent display a 3D rendering of at least a portion of the first virtual medical device.
Clause 6. The method of Clause 5, the 3D rendering of at least a portion of the first virtual medical device includes at least a first limb member and a second limb member of the virtual medical device.
Clause 7. The method of any of clauses 2-6, where causing one or more displays to concurrently display further comprises causing the one or more displays to display:
a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device; and
a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image.
Clause 8. A system comprising:
a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
-
- receive emplacement data associated with a first medical device, the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis,
- determine an emplacement of a first virtual medical device corresponding to the first medical device based at least in part on the received emplacement data,
- determine an emplacement of a second virtual medical device corresponding to a second medical device based at least in part on the received emplacement data,
- determine an emplacement of a medical image based at least in part on the received emplacement data, and
- cause a display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member,
- a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device, and
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image; and the display.
Clause 9. A system comprising:
a computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
-
- determine an emplacement of a display object based at least in part on first emplacement data,
- determine an emplacement of a medical image based at least in part on the first emplacement data, and
- cause one or more displays to concurrently display:
- a 3D rendering of at least a portion of the display object, and
- a 3D rendering of at least a portion of the medical image; and
the display.
Clause 10. The system of Clause 9, wherein the display object is a first virtual medical device corresponding to a first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis.
Clause 11. The system of Clause 10, wherein the computer system is further configured to:
receive the first emplacement data, the first emplacement data associated with the first medical device; and
determine an emplacement of a second virtual medical device corresponding to a second medical device based at least in part on the received first emplacement data.
Clause 12. The system of any of clauses 9-11, wherein the display object is a first virtual medical device and the computer system is further configured to cause the display to concurrently display a 3D rendering of at least a portion of the first virtual medical device.
Clause 13. The system of Clause 12, the 3D rendering of at least a portion of the first virtual medical device includes at least a first limb member and a second limb member of the virtual medical device.
Clause 14. The system of any of clauses 9-14, where the computer system is further configured to cause the display to concurrently display:
a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device; and
a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image.
Clause 15. A non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
-
- receive emplacement data associated with a first medical device, the first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis,
- determine an emplacement of a first virtual medical device corresponding to the first medical device based at least in part on the received emplacement data,
- determine an emplacement of a second virtual medical device corresponding to a second medical device based at least in part on the received emplacement data,
- determine an emplacement of a medical image based at least in part on the received emplacement data, and
- cause a display to concurrently display a perspective view of:
- a 3D rendering of at least a portion of the first virtual medical device based at least in part on the determined emplacement of the first virtual medical device, the 3D rendering of the first virtual medical device including at least the first limb member and the second limb member,
- a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device, and
- a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image.
Clause 16. A non-transitory computer-readable medium comprising computer executable instructions that when executed by one or more processors cause the one or more processors to:
-
- determine an emplacement of a display object based at least in part on first emplacement data,
- determine an emplacement of a medical image based at least in part on the first emplacement data, and
- cause one or more displays to concurrently display:
- a 3D rendering of at least a portion of the display object, and
- a 3D rendering of at least a portion of the medical image.
Clause 17. The system of Clause 16, wherein the display object is a first virtual medical device corresponding to a first medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis.
Clause 18. The non-transitory computer-readable medium of Clause 17, wherein the computer executable instructions further cause the one or more processors to:
receive the first emplacement data, the first emplacement data associated with the first medical device; and
determine an emplacement of a second virtual medical device corresponding to a second medical device based at least in part on the received first emplacement data.
Clause 19. The non-transitory computer-readable medium of any of clauses 16-18, wherein the display object is a first virtual medical device and wherein the computer executable instructions further cause the one or more processors to cause the display to concurrently display a 3D rendering of at least a portion of the first virtual medical device.
Clause 20. The non-transitory computer-readable medium of Clause 19, the 3D rendering of at least a portion of the first virtual medical device includes at least a first limb member and a second limb member of the virtual medical device.
Clause 21. The non-transitory computer-readable medium of any of clauses 16-20, wherein the computer executable instructions further cause the one or more processors to cause the display to concurrently display:
a 3D rendering of at least a portion of the second virtual medical device based at least in part on the determined emplacement of the second virtual medical device; and
a 3D rendering of at least a portion of the medical image based at least in part on the determined emplacement of the medical image.
Terminology
Conditional language, such as, among others, “can,” “could,” “might,” or “may,” unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements, and/or steps. Thus, such conditional language is not generally intended to imply that features, elements and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether these features, elements and/or steps are included or are to be performed in any particular embodiment.
Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to.” As used herein, the terms “connected,” “coupled,” or any variant thereof means any connection or coupling, either direct or indirect, between two or more elements; the coupling or connection between the elements can be physical, logical, or a combination thereof. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, refer to this application as a whole and not to any particular portions of this application. Where the context permits, words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively. The word “or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list. Likewise the term “and/or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list.
Depending on the embodiment, certain operations, acts, events, or functions of any of the algorithms described herein can be performed in a different sequence, can be added, merged, or left out altogether (non-limiting example: not all are necessary for the practice of the algorithms). Moreover, in certain embodiments, operations, acts, functions, or events can be performed concurrently, non-limiting examples: through multi-threaded processing, interrupt processing, or multiple processors or processor cores or on other parallel architectures, rather than sequentially.
The various illustrative logical blocks, modules, routines, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, or as a combination of electronic hardware and executable software. To clearly illustrate this interchangeability, various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware, or as software that runs on hardware, depends upon the particular application and design constraints imposed on the overall system 100. The described functionality can be implemented in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosure.
Moreover, the various illustrative logical blocks and modules described in connection with the embodiments disclosed herein can be implemented or performed by a machine, such as a processor device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor device can be a microprocessor, but in the alternative, the processor device can be a controller, microcontroller, or combinations of the same, or the like. A processor device can include electrical circuitry configured to process computer-executable instructions. In another embodiment, a processor device includes an FPGA or other programmable device that performs logic operations without processing computer-executable instructions. A processor device can also be implemented as a combination of computing devices, non-limiting examples: a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Although described herein primarily with respect to digital technology, a processor device may also include primarily analog components. For example, some or all of the signal processing algorithms described herein may be implemented in analog circuitry or mixed analog and digital circuitry. A computing environment can include any type of computer system, including, but not limited to, a computer system based on a microprocessor, a mainframe computer, a digital signal processor, a portable computing device, a device controller, or a computational engine within an appliance, to name a few.
The elements of a method, process, routine, or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor device, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of a non-transitory computer-readable storage medium. An exemplary storage medium can be coupled to the processor device such that the processor device can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor device. The processor device and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor device and the storage medium can reside as discrete components in a user terminal.
Further, the processing of the various components of the illustrated systems can be distributed across multiple machines, networks, and other computing resources. In addition, two or more components of a system can be combined into fewer components. Various components of the illustrated systems can be implemented in one or more virtual machines, rather than in dedicated computer hardware systems and/or computing devices.
Virtualization technologies allow a single physical computing device to host one or more instances of a virtual machine, which virtual machine instance appears to a user as an independent computing device. With virtualization, the host computing device can create, maintain, delete, or otherwise manage virtual machines instances in a dynamic manner. In turn, users can request computing resources, including single computing devices or a configuration of networked computing devices, and be provided with virtual machine instances that provide the requested computing resources.
An instance of a virtual machine may be configured to provide specific functionality. For example, a virtual machine instance may be associated with different combinations of software applications and operating systems or operating system configurations to enable a virtual machine to provide different desired functionalities, or to provide similar functionalities more efficiently.
Any patents and applications and other references noted above, including any that may be listed in accompanying filing papers, are incorporated herein by reference. Aspects of the invention can be modified, if necessary, to employ the systems, functions, and concepts of the various references described above to provide yet further implementations of the invention.
These and other changes can be made to the invention in light of the above Detailed Description. While the above description describes certain examples of the invention, and describes the best mode contemplated, no matter how detailed the above appears in text, the invention can be practiced in many ways. Details of the system may vary considerably in its specific implementation, while still being encompassed by the invention disclosed herein. As noted above, particular terminology used when describing certain features or aspects of the invention should not be taken to imply that the terminology is being redefined herein to be restricted to any specific characteristics, features, or aspects of the invention with which that terminology is associated. In general, the terms used in the following claims should not be construed to limit the invention to the specific examples disclosed in the specification, unless the above Detailed Description section explicitly defines such terms. Accordingly, the actual scope of the invention encompasses not only the disclosed examples, but also all equivalent ways of practicing or implementing the invention under the claims.
To reduce the number of claims, certain aspects of the invention are presented below in certain claim forms, but the applicant contemplates the various aspects of the invention in any number of claim forms. For example, while only one aspect of the invention may be recited as a means-plus-function claim under 35 U.S.C. sec. 112(f) (MA), other aspects may likewise be embodied as a means-plus-function claim, or in other forms, such as being embodied in a computer-readable medium. Any claims intended to be treated under 35 U.S.C. § 112(f) will begin with the words “means for”, but use of the term “for” in any other context is not intended to invoke treatment under 35 U.S.C. § 112(f). Accordingly, the applicant reserves the right to pursue additional claims after filing this application, in either this application or in a continuing application.
Disjunctive language such as the phrase “at least one of X, Y, or Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., may be either X, Y, or Z, or any combination thereof (non-limiting examples: X, Y, and/or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, or at least one of Z to each be present.
Unless otherwise explicitly stated, articles such a “a” or “an” should generally be interpreted to include one or more described items. Accordingly, phrases such as “a device configured to” are intended to include one or more recited devices. Such one or more recited devices can also be collectively configured to carry out the stated recitations. For example, “a processor configured to carry out recitations A, B and C” can include a first processor configured to carry out recitation A working in conjunction with a second processor configured to carry out recitations B and C.
While the above detailed description has shown, described, and pointed out novel features as applied to various embodiments, it can be understood that various omissions, substitutions, and changes in the form and details of the devices or algorithms illustrated can be made without departing from the spirit of the disclosure. As can be recognized, certain embodiments described herein can be embodied within a form that does not provide all of the features and benefits set forth herein, as some features can be used or practiced separately from others. The scope of certain embodiments disclosed herein is indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
The previous description of the disclosed implementations is provided to enable a person skilled in the art to make or use the present invention. Various modifications to these implementations will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other implementations without departing from the spirit or scope of the invention. Furthermore, although described above with reference to medical devices and procedures, it will be understood that the embodiments described herein can be applied to other systems in which non-medical objects are tracked and non-medical image streams are received, and virtual representations are displayed on a display and/or systems in which multiple objects are displayed on a display within a virtual space, such as within a virtual 3D space. Thus, the present invention is not intended to be limited to the implementations shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (28)
1. A method for medical device navigation, the method comprising:
determining an emplacement of a virtual medical device corresponding to a medical device, the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
determining an emplacement of a medical image slice;
determining a first intersection between an image plane corresponding to the medical image slice and a trajectory of the first limb member along a drive path;
determining a second intersection between the image plane and a trajectory of the second limb member along the drive path; and
causing one or more displays to concurrently display, in a virtual 3D space, a perspective view of:
at least a portion of the medical image slice based at least in part on the determined emplacement of the medical image slice,
a first intersection indicator corresponding to the first intersection, and
a second intersection indicator corresponding to the second intersection.
2. The method of claim 1 , wherein the virtual medical device is a first virtual medical device, wherein the medical device is a first medical device, the method further comprising:
receiving first emplacement data associated with the first medical device;
receiving second emplacement data associated with a second medical device; and
determining an emplacement of a second virtual medical device corresponding to the second medical device based at least in part on the received second emplacement data,
wherein said determining the emplacement of the first virtual medical device is based at least in part on the received first emplacement data, and wherein said determining the emplacement of the medical image slice is based at least in part on the received second emplacement data.
3. The method of claim 1 , further comprising further causing the one or more displays to concurrently display, in the virtual 3D space, a perspective view of:
a first trajectory indicator corresponding to the trajectory of the first limb member along the drive path, or
a second trajectory indicator corresponding to the trajectory of the second limb member along the drive path.
4. The method of claim 3 , wherein the second trajectory indicator corresponds to an axis that intersects with the second axis at a distal portion of the second limb member and is parallel to the joint member axis and the drive path.
5. The method of claim 1 , wherein the drive path is parallel to at least one of the joint member axis, the first axis, or the second axis.
6. The method of claim 1 , wherein at least one of the first intersection indicator or the second intersection indicator comprises an indication of at least a portion of a path of a transecting knife of the medical device.
7. The method of claim 1 , wherein the medical device comprises at least one of a transecting stapler, a grasper, or a vessel sealer, and wherein the medical image slice corresponds to a real-time medical image of a medical imaging device.
8. A method for medical instrument navigation, the method comprising:
determining an emplacement of a medical display object;
determining an emplacement of a plane-of-interest;
determining an emplacement of a ghost medical display object based at least in part on the determined emplacement of the medical display object and the determined emplacement of the plane-of-interest; and
causing one or more displays to concurrently display, in a virtual 3D space:
at least a portion of the medical display object, and
at least a portion of the ghost medical display object on the plane-of-interest.
9. The method of claim 8 , further comprising:
receiving first emplacement data associated with a first medical device; and
receiving second emplacement data associated with a second medical device,
wherein said determining the emplacement of the medical display object is based at least in part on the received first emplacement data and said determining the emplacement of the plane-of-interest is based at least in part on the received second emplacement data.
10. The method of claim 9 , further comprising:
receiving a medical image corresponding to the second medical device; and
determining an emplacement of the medical image based at least in part on the received second emplacement data,
wherein said determining the emplacement of the plane-of-interest is based at least in part on the determined emplacement of the medical image.
11. The method of claim 8 , wherein said determining the emplacement of the ghost medical display object comprises at least one of:
determining at least one of portions, points, or coordinates of the plane-of-interest that are closest to the medical display object,
determining an arc path corresponding to the medical display object and the plane-of-interest,
determining which of a plurality of connecting lines intersect the medical display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the medical display object, or
using a virtual light source that is at least as large as the medical display object to be projected, wherein the virtual light source is distal to the plane-of-interest relative to the medical display object.
12. The method of claim 8 , wherein the ghost medical display object comprises at least one of a virtual medical device that corresponds to a medical device, a medical image slice that corresponds to a real-time medical image of a medical imaging device, or an affected region of the medical device, wherein the medical device comprises a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
13. The method of claim 8 , wherein the plane-of-interest corresponds to at least one of a medical device plane or an image plane.
14. The method of claim 8 , wherein the medical display object corresponds to an affected region of a medical device, wherein the medical device comprises a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis, and wherein the affected region corresponds to at least one of:
a volume located between the first limb member and the second limb member,
a stapling path of at least one staple of the medical device, or
a cutting path of a knife of the medical device.
15. The method of claim 8 , wherein the medical display object corresponds to at least one of a transecting stapler, a grasper, or a vessel sealer, and wherein the plane-of-interest corresponds to an imaging plane of a medical imaging device.
16. A method for medical instrument navigation, the method comprising:
determining an emplacement of a virtual medical device corresponding to a medical device, the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
determining an emplacement of an affected region based at least in part on the determined emplacement of the virtual medical device; and
causing one or more displays to display, in a virtual 3D space, a perspective view of at least one of:
at least a portion of the affected region based at least in part on the determined emplacement of the virtual medical device, or
an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path that is parallel to the joint member axis.
17. The method of claim 16 , further comprising:
receiving emplacement data associated with the medical device, wherein said determining the emplacement of the virtual medical device is based at least in part on the received emplacement data; and
receiving operating parameters corresponding to the medical device, wherein said determining the emplacement of the affected region is further based at least in part on the received operating parameters.
18. The method of claim 16 , further comprising further causing the one or more displays to display, in the virtual 3D space, a perspective view of at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device.
19. The method of claim 16 , wherein the affected region corresponds to a volume located between the first limb member and the second limb member.
20. The method of claim 16 , wherein the medical device comprises a medical stapler, wherein the affected region comprises a stapling path of the medical stapler.
21. The method of claim 16 , wherein the affected region comprises a cutting path of a knife of the medical device.
22. A computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:
determine an emplacement of a virtual medical device corresponding to a medical device, the medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;
determine an emplacement of a medical image slice;
determine a first intersection between an image plane corresponding to the medical image slice and a trajectory of the first limb member along a drive path;
determine a second intersection between the image plane and a trajectory of the second limb member along the drive path; and
cause the display to concurrently display, in a virtual 3D space, a perspective view of:
at least a portion of the medical image slice based at least in part on the determined emplacement of the medical image slice,
a first intersection indicator corresponding to the first intersection, and
a second intersection indicator corresponding to the second intersection.
23. The computer system of claim 22 , wherein the virtual medical device is a first virtual medical device, wherein the medical device is a first medical device, wherein the computer system is further configured to:
receive first emplacement data associated with the first medical device;
receive second emplacement data associated with a second medical device; and
determine an emplacement of a second virtual medical device corresponding to the second medical device based at least in part on the received second emplacement data,
wherein to determine the emplacement of the first virtual medical device, the computer system is further configured to determine the emplacement of the first virtual medical device based at least in part on the received first emplacement data,
wherein to determine the emplacement of the medical image slice, the computer system is further configured to determine the emplacement of the medical image slice based at least in part on the received second emplacement data.
24. The computer system of claim 22 , wherein the computer system is further configured to further cause the display to display, in the virtual 3D space, a perspective view of:
a first trajectory indicator corresponding to the trajectory of the first limb member along the drive path, or
a second trajectory indicator corresponding to the trajectory of the second limb member along the drive path.
25. The computer system of claim 24 , wherein the second trajectory indicator corresponds to an axis that intersects with the second axis at a distal portion of the second limb member and is parallel to the joint member axis and the drive path.
26. The computer system of claim 22 , wherein the drive path is parallel to at least one of the joint member axis, the first axis, or the second axis.
27. The computer system of claim 22 , wherein at least one of the first intersection indicator or the second intersection indicator comprises an indication of at least a portion of a path of transecting knife of the medical device.
28. The computer system of claim 22 , wherein the medical device comprises at least one of a transecting stapler, grasper, or vessel sealer, and wherein the medical image slice corresponds to a real-time medical image of a medical imaging device.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/415,398 US10278778B2 (en) | 2016-10-27 | 2017-01-25 | Medical device navigation using a virtual 3D space |
PCT/US2017/057042 WO2018080844A1 (en) | 2016-10-27 | 2017-10-17 | Medical instrument navigation |
US16/366,537 US10772686B2 (en) | 2016-10-27 | 2019-03-27 | Medical device navigation using a virtual 3D space |
US16/985,580 US11369439B2 (en) | 2016-10-27 | 2020-08-05 | Medical device navigation using a virtual 3D space |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662413925P | 2016-10-27 | 2016-10-27 | |
US15/415,398 US10278778B2 (en) | 2016-10-27 | 2017-01-25 | Medical device navigation using a virtual 3D space |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/366,537 Continuation US10772686B2 (en) | 2016-10-27 | 2019-03-27 | Medical device navigation using a virtual 3D space |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180116731A1 US20180116731A1 (en) | 2018-05-03 |
US10278778B2 true US10278778B2 (en) | 2019-05-07 |
Family
ID=62020776
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/415,398 Active 2037-02-10 US10278778B2 (en) | 2016-10-27 | 2017-01-25 | Medical device navigation using a virtual 3D space |
US16/366,537 Active US10772686B2 (en) | 2016-10-27 | 2019-03-27 | Medical device navigation using a virtual 3D space |
US16/985,580 Active 2037-04-16 US11369439B2 (en) | 2016-10-27 | 2020-08-05 | Medical device navigation using a virtual 3D space |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/366,537 Active US10772686B2 (en) | 2016-10-27 | 2019-03-27 | Medical device navigation using a virtual 3D space |
US16/985,580 Active 2037-04-16 US11369439B2 (en) | 2016-10-27 | 2020-08-05 | Medical device navigation using a virtual 3D space |
Country Status (2)
Country | Link |
---|---|
US (3) | US10278778B2 (en) |
WO (1) | WO2018080844A1 (en) |
Cited By (159)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190216548A1 (en) * | 2017-06-23 | 2019-07-18 | Auris Health, Inc. | Robotic systems for determining a roll of a medical device in luminal networks |
US10595887B2 (en) | 2017-12-28 | 2020-03-24 | Ethicon Llc | Systems for adjusting end effector parameters based on perioperative information |
US10695081B2 (en) | 2017-12-28 | 2020-06-30 | Ethicon Llc | Controlling a surgical instrument according to sensed closure parameters |
US20200214682A1 (en) * | 2019-01-07 | 2020-07-09 | Butterfly Network, Inc. | Methods and apparatuses for tele-medicine |
US10733700B2 (en) | 2006-08-02 | 2020-08-04 | Inneroptic Technology, Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US10755813B2 (en) | 2017-12-28 | 2020-08-25 | Ethicon Llc | Communication of smoke evacuation system parameters to hub or cloud in smoke evacuation module for interactive surgical platform |
US10758310B2 (en) | 2017-12-28 | 2020-09-01 | Ethicon Llc | Wireless pairing of a surgical device with another device within a sterile surgical field based on the usage and situational awareness of devices |
US10772651B2 (en) | 2017-10-30 | 2020-09-15 | Ethicon Llc | Surgical instruments comprising a system for articulation and rotation compensation |
US10772686B2 (en) | 2016-10-27 | 2020-09-15 | Inneroptic Technology, Inc. | Medical device navigation using a virtual 3D space |
US10796432B2 (en) | 2015-09-18 | 2020-10-06 | Auris Health, Inc. | Navigation of tubular networks |
US10806535B2 (en) | 2015-11-30 | 2020-10-20 | Auris Health, Inc. | Robot-assisted driving systems and methods |
US10820946B2 (en) | 2014-12-12 | 2020-11-03 | Inneroptic Technology, Inc. | Surgical guidance intersection display |
US10827913B2 (en) | 2018-03-28 | 2020-11-10 | Auris Health, Inc. | Systems and methods for displaying estimated location of instrument |
US10849697B2 (en) | 2017-12-28 | 2020-12-01 | Ethicon Llc | Cloud interface for coupled surgical devices |
US10892995B2 (en) | 2017-12-28 | 2021-01-12 | Ethicon Llc | Surgical network determination of prioritization of communication, interaction, or processing based on system or device needs |
US10892899B2 (en) | 2017-12-28 | 2021-01-12 | Ethicon Llc | Self describing data packets generated at an issuing instrument |
US10898286B2 (en) | 2018-05-31 | 2021-01-26 | Auris Health, Inc. | Path-based navigation of tubular networks |
US10898622B2 (en) | 2017-12-28 | 2021-01-26 | Ethicon Llc | Surgical evacuation system with a communication circuit for communication between a filter and a smoke evacuation device |
US10898277B2 (en) | 2018-03-28 | 2021-01-26 | Auris Health, Inc. | Systems and methods for registration of location sensors |
US10898275B2 (en) | 2018-05-31 | 2021-01-26 | Auris Health, Inc. | Image-based airway analysis and mapping |
US10905499B2 (en) | 2018-05-30 | 2021-02-02 | Auris Health, Inc. | Systems and methods for location sensor-based branch prediction |
US10932872B2 (en) | 2017-12-28 | 2021-03-02 | Ethicon Llc | Cloud-based medical analytics for linking of local usage trends with the resource acquisition behaviors of larger data set |
US10944728B2 (en) | 2017-12-28 | 2021-03-09 | Ethicon Llc | Interactive surgical systems with encrypted communication capabilities |
US10943454B2 (en) | 2017-12-28 | 2021-03-09 | Ethicon Llc | Detection and escalation of security responses of surgical instruments to increasing severity threats |
US10966791B2 (en) | 2017-12-28 | 2021-04-06 | Ethicon Llc | Cloud-based medical analytics for medical facility segmented individualization of instrument function |
US10973520B2 (en) | 2018-03-28 | 2021-04-13 | Ethicon Llc | Surgical staple cartridge with firing member driven camming assembly that has an onboard tissue cutting feature |
US10987178B2 (en) | 2017-12-28 | 2021-04-27 | Ethicon Llc | Surgical hub control arrangements |
US11013563B2 (en) | 2017-12-28 | 2021-05-25 | Ethicon Llc | Drive arrangements for robot-assisted surgical platforms |
US11020016B2 (en) | 2013-05-30 | 2021-06-01 | Auris Health, Inc. | System and method for displaying anatomy and devices on a movable display |
US11026687B2 (en) | 2017-10-30 | 2021-06-08 | Cilag Gmbh International | Clip applier comprising clip advancing systems |
US20210177402A1 (en) * | 2019-12-13 | 2021-06-17 | Dinesh Vyas | Stapler apparatus and methods for use |
US11056244B2 (en) | 2017-12-28 | 2021-07-06 | Cilag Gmbh International | Automated data scaling, alignment, and organizing based on predefined parameters within surgical networks |
US11051876B2 (en) | 2017-12-28 | 2021-07-06 | Cilag Gmbh International | Surgical evacuation flow paths |
US11051681B2 (en) | 2010-06-24 | 2021-07-06 | Auris Health, Inc. | Methods and devices for controlling a shapeable medical device |
US11058498B2 (en) | 2017-12-28 | 2021-07-13 | Cilag Gmbh International | Cooperative surgical actions for robot-assisted surgical platforms |
US11058493B2 (en) | 2017-10-13 | 2021-07-13 | Auris Health, Inc. | Robotic system configured for navigation path tracing |
US11069012B2 (en) | 2017-12-28 | 2021-07-20 | Cilag Gmbh International | Interactive surgical systems with condition handling of devices and data capabilities |
US11076921B2 (en) | 2017-12-28 | 2021-08-03 | Cilag Gmbh International | Adaptive control program updates for surgical hubs |
US11090047B2 (en) | 2018-03-28 | 2021-08-17 | Cilag Gmbh International | Surgical instrument comprising an adaptive control system |
US11096693B2 (en) | 2017-12-28 | 2021-08-24 | Cilag Gmbh International | Adjustment of staple height of at least one row of staples based on the sensed tissue thickness or force in closing |
US11096688B2 (en) | 2018-03-28 | 2021-08-24 | Cilag Gmbh International | Rotary driven firing members with different anvil and channel engagement features |
US11100631B2 (en) | 2017-12-28 | 2021-08-24 | Cilag Gmbh International | Use of laser light and red-green-blue coloration to determine properties of back scattered light |
US11103200B2 (en) | 2015-07-22 | 2021-08-31 | Inneroptic Technology, Inc. | Medical device approaches |
US11109866B2 (en) | 2017-12-28 | 2021-09-07 | Cilag Gmbh International | Method for circular stapler control algorithm adjustment based on situational awareness |
US11114195B2 (en) | 2017-12-28 | 2021-09-07 | Cilag Gmbh International | Surgical instrument with a tissue marking assembly |
US11129602B2 (en) | 2013-03-15 | 2021-09-28 | Auris Health, Inc. | Systems and methods for tracking robotically controlled medical instruments |
US11132462B2 (en) | 2017-12-28 | 2021-09-28 | Cilag Gmbh International | Data stripping method to interrogate patient records and create anonymized record |
US11129611B2 (en) | 2018-03-28 | 2021-09-28 | Cilag Gmbh International | Surgical staplers with arrangements for maintaining a firing member thereof in a locked configuration unless a compatible cartridge has been installed therein |
US11147633B2 (en) | 2019-08-30 | 2021-10-19 | Auris Health, Inc. | Instrument image reliability systems and methods |
US11147607B2 (en) | 2017-12-28 | 2021-10-19 | Cilag Gmbh International | Bipolar combination device that automatically adjusts pressure based on energy modality |
US11160615B2 (en) | 2017-12-18 | 2021-11-02 | Auris Health, Inc. | Methods and systems for instrument tracking and navigation within luminal networks |
US11160605B2 (en) | 2017-12-28 | 2021-11-02 | Cilag Gmbh International | Surgical evacuation sensing and motor control |
US11166772B2 (en) | 2017-12-28 | 2021-11-09 | Cilag Gmbh International | Surgical hub coordination of control and communication of operating room devices |
US11179208B2 (en) | 2017-12-28 | 2021-11-23 | Cilag Gmbh International | Cloud-based medical analytics for security and authentication trends and reactive measures |
US11179175B2 (en) | 2017-12-28 | 2021-11-23 | Cilag Gmbh International | Controlling an ultrasonic surgical instrument according to tissue location |
US11179136B2 (en) | 2016-02-17 | 2021-11-23 | Inneroptic Technology, Inc. | Loupe display |
US11202570B2 (en) | 2017-12-28 | 2021-12-21 | Cilag Gmbh International | Communication hub and storage device for storing parameters and status of a surgical device to be shared with cloud based analytics systems |
US11207141B2 (en) | 2019-08-30 | 2021-12-28 | Auris Health, Inc. | Systems and methods for weight-based registration of location sensors |
US11207067B2 (en) | 2018-03-28 | 2021-12-28 | Cilag Gmbh International | Surgical stapling device with separate rotary driven closure and firing systems and firing member that engages both jaws while firing |
US11219453B2 (en) | 2018-03-28 | 2022-01-11 | Cilag Gmbh International | Surgical stapling devices with cartridge compatible closure and firing lockout arrangements |
US11229436B2 (en) | 2017-10-30 | 2022-01-25 | Cilag Gmbh International | Surgical system comprising a surgical tool and a surgical hub |
US11234756B2 (en) | 2017-12-28 | 2022-02-01 | Cilag Gmbh International | Powered surgical tool with predefined adjustable control algorithm for controlling end effector parameter |
US11241203B2 (en) | 2013-03-13 | 2022-02-08 | Auris Health, Inc. | Reducing measurement sensor error |
US11253315B2 (en) | 2017-12-28 | 2022-02-22 | Cilag Gmbh International | Increasing radio frequency to create pad-less monopolar loop |
US11257589B2 (en) | 2017-12-28 | 2022-02-22 | Cilag Gmbh International | Real-time analysis of comprehensive cost of all instrumentation used in surgery utilizing data fluidity to track instruments through stocking and in-house processes |
US11259830B2 (en) | 2018-03-08 | 2022-03-01 | Cilag Gmbh International | Methods for controlling temperature in ultrasonic device |
US11259806B2 (en) | 2018-03-28 | 2022-03-01 | Cilag Gmbh International | Surgical stapling devices with features for blocking advancement of a camming assembly of an incompatible cartridge installed therein |
US11259879B2 (en) | 2017-08-01 | 2022-03-01 | Inneroptic Technology, Inc. | Selective transparency to assist medical device navigation |
US11259807B2 (en) | 2019-02-19 | 2022-03-01 | Cilag Gmbh International | Staple cartridges with cam surfaces configured to engage primary and secondary portions of a lockout of a surgical stapling device |
US11266468B2 (en) | 2017-12-28 | 2022-03-08 | Cilag Gmbh International | Cooperative utilization of data derived from secondary sources by intelligent surgical hubs |
US11273001B2 (en) | 2017-12-28 | 2022-03-15 | Cilag Gmbh International | Surgical hub and modular device response adjustment based on situational awareness |
US11278281B2 (en) | 2017-12-28 | 2022-03-22 | Cilag Gmbh International | Interactive surgical system |
US11278280B2 (en) | 2018-03-28 | 2022-03-22 | Cilag Gmbh International | Surgical instrument comprising a jaw closure lockout |
US11284936B2 (en) | 2017-12-28 | 2022-03-29 | Cilag Gmbh International | Surgical instrument having a flexible electrode |
US11291510B2 (en) | 2017-10-30 | 2022-04-05 | Cilag Gmbh International | Method of hub communication with surgical instrument systems |
US11291495B2 (en) | 2017-12-28 | 2022-04-05 | Cilag Gmbh International | Interruption of energy due to inadvertent capacitive coupling |
US11298195B2 (en) | 2019-12-31 | 2022-04-12 | Auris Health, Inc. | Anatomical feature identification and targeting |
US11298148B2 (en) | 2018-03-08 | 2022-04-12 | Cilag Gmbh International | Live time tissue classification using electrical parameters |
US11304763B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Image capturing of the areas outside the abdomen to improve placement and control of a surgical device in use |
US11304720B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Activation of energy devices |
US11308075B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Surgical network, instrument, and cloud responses based on validation of received dataset and authentication of its source and integrity |
US11304745B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Surgical evacuation sensing and display |
US11304699B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Method for adaptive control schemes for surgical network control and interaction |
US11311342B2 (en) | 2017-10-30 | 2022-04-26 | Cilag Gmbh International | Method for communicating with surgical instrument systems |
US11311306B2 (en) | 2017-12-28 | 2022-04-26 | Cilag Gmbh International | Surgical systems for detecting end effector tissue distribution irregularities |
US11317915B2 (en) | 2019-02-19 | 2022-05-03 | Cilag Gmbh International | Universal cartridge based key feature that unlocks multiple lockout arrangements in different surgical staplers |
US11317919B2 (en) | 2017-10-30 | 2022-05-03 | Cilag Gmbh International | Clip applier comprising a clip crimping system |
USD950728S1 (en) | 2019-06-25 | 2022-05-03 | Cilag Gmbh International | Surgical staple cartridge |
US11317937B2 (en) | 2018-03-08 | 2022-05-03 | Cilag Gmbh International | Determining the state of an ultrasonic end effector |
US11324557B2 (en) | 2017-12-28 | 2022-05-10 | Cilag Gmbh International | Surgical instrument with a sensing array |
USD952144S1 (en) | 2019-06-25 | 2022-05-17 | Cilag Gmbh International | Surgical staple cartridge retainer with firing system authentication key |
US11337746B2 (en) | 2018-03-08 | 2022-05-24 | Cilag Gmbh International | Smart blade and power pulsing |
US11357503B2 (en) | 2019-02-19 | 2022-06-14 | Cilag Gmbh International | Staple cartridge retainers with frangible retention features and methods of using same |
US11364075B2 (en) | 2017-12-28 | 2022-06-21 | Cilag Gmbh International | Radio frequency energy device for delivering combined electrical signals |
US11369377B2 (en) | 2019-02-19 | 2022-06-28 | Cilag Gmbh International | Surgical stapling assembly with cartridge based retainer configured to unlock a firing lockout |
US11376002B2 (en) | 2017-12-28 | 2022-07-05 | Cilag Gmbh International | Surgical instrument cartridge sensor assemblies |
US11382697B2 (en) | 2017-12-28 | 2022-07-12 | Cilag Gmbh International | Surgical instruments comprising button circuits |
US11389164B2 (en) | 2017-12-28 | 2022-07-19 | Cilag Gmbh International | Method of using reinforced flexible circuits with multiple sensors to optimize performance of radio frequency devices |
US11410259B2 (en) | 2017-12-28 | 2022-08-09 | Cilag Gmbh International | Adaptive control program updates for surgical devices |
US11419667B2 (en) | 2017-12-28 | 2022-08-23 | Cilag Gmbh International | Ultrasonic energy device which varies pressure applied by clamp arm to provide threshold control pressure at a cut progression location |
US11419630B2 (en) | 2017-12-28 | 2022-08-23 | Cilag Gmbh International | Surgical system distributed processing |
US11423007B2 (en) | 2017-12-28 | 2022-08-23 | Cilag Gmbh International | Adjustment of device control programs based on stratified contextual data in addition to the data |
US11424027B2 (en) | 2017-12-28 | 2022-08-23 | Cilag Gmbh International | Method for operating surgical instrument systems |
US11426095B2 (en) | 2013-03-15 | 2022-08-30 | Auris Health, Inc. | Flexible instrument localization from both remote and elongation sensors |
US11432885B2 (en) | 2017-12-28 | 2022-09-06 | Cilag Gmbh International | Sensing arrangements for robot-assisted surgical platforms |
US11446052B2 (en) | 2017-12-28 | 2022-09-20 | Cilag Gmbh International | Variation of radio frequency and ultrasonic power level in cooperation with varying clamp arm pressure to achieve predefined heat flux or power applied to tissue |
USD964564S1 (en) | 2019-06-25 | 2022-09-20 | Cilag Gmbh International | Surgical staple cartridge retainer with a closure system authentication key |
US11464575B2 (en) | 2009-02-17 | 2022-10-11 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image guided surgery |
US11464578B2 (en) | 2009-02-17 | 2022-10-11 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image management in image-guided medical procedures |
US11464535B2 (en) | 2017-12-28 | 2022-10-11 | Cilag Gmbh International | Detection of end effector emersion in liquid |
US11464559B2 (en) | 2017-12-28 | 2022-10-11 | Cilag Gmbh International | Estimating state of ultrasonic end effector and control system therefor |
US11471156B2 (en) | 2018-03-28 | 2022-10-18 | Cilag Gmbh International | Surgical stapling devices with improved rotary driven closure systems |
US11490782B2 (en) | 2017-03-31 | 2022-11-08 | Auris Health, Inc. | Robotic systems for navigation of luminal networks that compensate for physiological noise |
US11504187B2 (en) | 2013-03-15 | 2022-11-22 | Auris Health, Inc. | Systems and methods for localizing, tracking and/or controlling medical instruments |
US11503986B2 (en) | 2018-05-31 | 2022-11-22 | Auris Health, Inc. | Robotic systems and methods for navigation of luminal network that detect physiological noise |
US11504192B2 (en) | 2014-10-30 | 2022-11-22 | Cilag Gmbh International | Method of hub communication with surgical instrument systems |
US11510736B2 (en) | 2017-12-14 | 2022-11-29 | Auris Health, Inc. | System and method for estimating instrument location |
US11510741B2 (en) | 2017-10-30 | 2022-11-29 | Cilag Gmbh International | Method for producing a surgical instrument comprising a smart electrical system |
US11529187B2 (en) | 2017-12-28 | 2022-12-20 | Cilag Gmbh International | Surgical evacuation sensor arrangements |
US11540855B2 (en) | 2017-12-28 | 2023-01-03 | Cilag Gmbh International | Controlling activation of an ultrasonic surgical instrument according to the presence of tissue |
US11559308B2 (en) | 2017-12-28 | 2023-01-24 | Cilag Gmbh International | Method for smart energy device infrastructure |
US11559307B2 (en) | 2017-12-28 | 2023-01-24 | Cilag Gmbh International | Method of robotic hub communication, detection, and control |
US11564756B2 (en) | 2017-10-30 | 2023-01-31 | Cilag Gmbh International | Method of hub communication with surgical instrument systems |
US11571234B2 (en) | 2017-12-28 | 2023-02-07 | Cilag Gmbh International | Temperature control of ultrasonic end effector and control system therefor |
US11576677B2 (en) | 2017-12-28 | 2023-02-14 | Cilag Gmbh International | Method of hub communication, processing, display, and cloud analytics |
US20230056943A1 (en) * | 2019-12-13 | 2023-02-23 | Dinesh Vyas | Stapler apparatus and methods for use |
US11589932B2 (en) | 2017-12-28 | 2023-02-28 | Cilag Gmbh International | Usage and technique analysis of surgeon / staff performance against a baseline to optimize device utilization and performance for both current and future procedures |
US11589888B2 (en) | 2017-12-28 | 2023-02-28 | Cilag Gmbh International | Method for controlling smart energy devices |
US11596291B2 (en) | 2017-12-28 | 2023-03-07 | Cilag Gmbh International | Method of compressing tissue within a stapling device and simultaneously displaying of the location of the tissue within the jaws |
US11602372B2 (en) | 2019-12-31 | 2023-03-14 | Auris Health, Inc. | Alignment interfaces for percutaneous access |
US11602393B2 (en) | 2017-12-28 | 2023-03-14 | Cilag Gmbh International | Surgical evacuation sensing and generator control |
US20230087202A1 (en) * | 2021-09-17 | 2023-03-23 | Ford Global Technologies, Llc | Augmented Reality And Touch-Based User Engagement Parking Assist |
US11612444B2 (en) | 2017-12-28 | 2023-03-28 | Cilag Gmbh International | Adjustment of a surgical device function based on situational awareness |
US11659023B2 (en) | 2017-12-28 | 2023-05-23 | Cilag Gmbh International | Method of hub communication |
US11660147B2 (en) | 2019-12-31 | 2023-05-30 | Auris Health, Inc. | Alignment techniques for percutaneous access |
US11666331B2 (en) | 2017-12-28 | 2023-06-06 | Cilag Gmbh International | Systems for detecting proximity of surgical end effector to cancerous tissue |
US11684429B2 (en) | 2014-10-02 | 2023-06-27 | Inneroptic Technology, Inc. | Affected region display associated with a medical device |
US11707330B2 (en) * | 2017-01-03 | 2023-07-25 | Mako Surgical Corp. | Systems and methods for surgical navigation |
US11744604B2 (en) | 2017-12-28 | 2023-09-05 | Cilag Gmbh International | Surgical instrument with a hardware-only control circuit |
US11751872B2 (en) | 2019-02-19 | 2023-09-12 | Cilag Gmbh International | Insertable deactivator element for surgical stapler lockouts |
US11771309B2 (en) | 2016-12-28 | 2023-10-03 | Auris Health, Inc. | Detecting endolumenal buckling of flexible instruments |
US11771487B2 (en) | 2017-12-28 | 2023-10-03 | Cilag Gmbh International | Mechanisms for controlling different electromechanical systems of an electrosurgical instrument |
US11786245B2 (en) | 2017-12-28 | 2023-10-17 | Cilag Gmbh International | Surgical systems with prioritized data transmission capabilities |
US11786251B2 (en) | 2017-12-28 | 2023-10-17 | Cilag Gmbh International | Method for adaptive control schemes for surgical network control and interaction |
US11801098B2 (en) | 2017-10-30 | 2023-10-31 | Cilag Gmbh International | Method of hub communication with surgical instrument systems |
US11818052B2 (en) | 2017-12-28 | 2023-11-14 | Cilag Gmbh International | Surgical network determination of prioritization of communication, interaction, or processing based on system or device needs |
US11832899B2 (en) | 2017-12-28 | 2023-12-05 | Cilag Gmbh International | Surgical systems with autonomously adjustable control programs |
US11832840B2 (en) | 2017-12-28 | 2023-12-05 | Cilag Gmbh International | Surgical instrument having a flexible circuit |
US11850008B2 (en) | 2017-10-13 | 2023-12-26 | Auris Health, Inc. | Image-based branch detection and mapping for navigation |
US11857152B2 (en) | 2017-12-28 | 2024-01-02 | Cilag Gmbh International | Surgical hub spatial awareness to determine devices in operating theater |
US11864728B2 (en) | 2017-12-28 | 2024-01-09 | Cilag Gmbh International | Characterization of tissue irregularities through the use of mono-chromatic light refractivity |
US11871901B2 (en) | 2012-05-20 | 2024-01-16 | Cilag Gmbh International | Method for situational awareness for surgical network or surgical network connected device capable of adjusting function based on a sensed situation or usage |
US11896322B2 (en) | 2017-12-28 | 2024-02-13 | Cilag Gmbh International | Sensing the patient position and contact utilizing the mono-polar return pad electrode to provide situational awareness to the hub |
US11896443B2 (en) | 2017-12-28 | 2024-02-13 | Cilag Gmbh International | Control of a surgical system through a surgical barrier |
US11903601B2 (en) | 2017-12-28 | 2024-02-20 | Cilag Gmbh International | Surgical instrument comprising a plurality of drive systems |
US11911045B2 (en) | 2017-10-30 | 2024-02-27 | Cllag GmbH International | Method for operating a powered articulating multi-clip applier |
US11937769B2 (en) | 2017-12-28 | 2024-03-26 | Cilag Gmbh International | Method of hub communication, processing, storage and display |
US11969216B2 (en) | 2017-12-28 | 2024-04-30 | Cilag Gmbh International | Surgical network recommendations from real time analysis of procedure variables against a baseline highlighting differences from the optimal solution |
US11998193B2 (en) | 2018-12-19 | 2024-06-04 | Cilag Gmbh International | Method for usage of the shroud as an aspect of sensing or controlling a powered surgical device, and a control algorithm to adjust its default operation |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8641621B2 (en) | 2009-02-17 | 2014-02-04 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image management in image-guided medical procedures |
US10314559B2 (en) | 2013-03-14 | 2019-06-11 | Inneroptic Technology, Inc. | Medical device guidance |
US11779192B2 (en) * | 2017-05-03 | 2023-10-10 | Covidien Lp | Medical image viewer control from surgeon's camera |
CA3056260C (en) * | 2017-05-09 | 2022-04-12 | Brainlab Ag | Generation of augmented reality image of a medical device |
US11354818B2 (en) * | 2017-06-21 | 2022-06-07 | Sony Corporation | Medical imaging system, method and computer program product |
JP7291697B2 (en) * | 2017-11-09 | 2023-06-15 | アセッサー・ヘルス・インコーポレーテッド | A system for controlling ablation therapy and visualization |
US11484365B2 (en) | 2018-01-23 | 2022-11-01 | Inneroptic Technology, Inc. | Medical image guidance |
US20210219978A1 (en) * | 2018-05-10 | 2021-07-22 | Franklin Institute of Innovation, LLC | Surgical staplers and related methods |
CN110554784B (en) * | 2018-05-31 | 2023-07-14 | 广东虚拟现实科技有限公司 | Input method, input device, display device and storage medium |
CN110179502A (en) * | 2019-06-06 | 2019-08-30 | 深圳大学 | Surgical instrument and application method |
WO2021058087A1 (en) * | 2019-09-24 | 2021-04-01 | Brainlab Ag | Method and system for projecting an incision marker onto a patient |
US20210153959A1 (en) * | 2019-11-26 | 2021-05-27 | Intuitive Surgical Operations, Inc. | Physical medical element affixation systems, methods, and materials |
CN114746012A (en) * | 2019-12-03 | 2022-07-12 | 医疗视野Xr有限公司 | Holographic augmented reality ultrasound needle guide for percutaneous surgical procedure insertion |
NL2026208B1 (en) * | 2020-08-04 | 2022-04-08 | Stichting Het Nederlands Kanker Inst Antoni Van Leeuwenhoek Ziekenhuis | Surgical forceps and stapler |
US20240013679A1 (en) * | 2020-10-02 | 2024-01-11 | The Johns Hopkins University | Smart sensor-equipped phantom for performance assessment of medical procedures |
Citations (392)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3556079A (en) | 1967-05-16 | 1971-01-19 | Haruo Omizo | Method of puncturing a medical instrument under guidance of ultrasound |
US4058114A (en) | 1974-09-11 | 1977-11-15 | Siemens Aktiengesellschaft | Ultrasonic arrangement for puncturing internal body organs, vessels and the like |
USRE30397E (en) | 1976-04-27 | 1980-09-09 | Three-dimensional ultrasonic imaging of animal soft tissue | |
US4249539A (en) | 1979-02-09 | 1981-02-10 | Technicare Corporation | Ultrasound needle tip localization system |
US4294544A (en) | 1979-08-03 | 1981-10-13 | Altschuler Bruce R | Topographic comparator |
US4390025A (en) | 1980-04-24 | 1983-06-28 | Tokyo Shibaura Denki Kabushiki Kaisha | Ultrasonic display apparatus having means for detecting the position of an ultrasonic probe |
US4407294A (en) | 1982-01-07 | 1983-10-04 | Technicare Corporation | Ultrasound tissue probe localization system |
US4431006A (en) | 1982-01-07 | 1984-02-14 | Technicare Corporation | Passive ultrasound needle probe locator |
US4567896A (en) | 1984-01-20 | 1986-02-04 | Elscint, Inc. | Method and apparatus for calibrating a biopsy attachment for ultrasonic imaging apparatus |
US4583538A (en) | 1984-05-04 | 1986-04-22 | Onik Gary M | Method and apparatus for stereotaxic placement of probes in the body utilizing CT scanner localization |
US4620546A (en) | 1984-06-30 | 1986-11-04 | Kabushiki Kaisha Toshiba | Ultrasound hyperthermia apparatus |
US4671292A (en) | 1985-04-30 | 1987-06-09 | Dymax Corporation | Concentric biopsy probe |
JPS63290550A (en) | 1987-05-22 | 1988-11-28 | Hitachi Medical Corp | Ultrasonic tomographic apparatus for guiding puncture needle |
US4839836A (en) | 1986-03-11 | 1989-06-13 | U.S. Philips Corporation | Signal transient improvement arrangement |
US4862873A (en) | 1987-05-27 | 1989-09-05 | Olympus Optical Co., Ltd. | Stereo endoscope |
US4884219A (en) | 1987-01-21 | 1989-11-28 | W. Industries Limited | Method and apparatus for the perception of computer-generated imagery |
US4899756A (en) | 1988-07-18 | 1990-02-13 | Sonek Jiri D | Articulated needle guide for ultrasound imaging and method of using same |
US4911173A (en) | 1987-11-13 | 1990-03-27 | Diasonics, Inc. | Biopsy attachment for ultrasound probe |
US4945305A (en) | 1986-10-09 | 1990-07-31 | Ascension Technology Corporation | Device for quantitatively measuring the relative position and orientation of two bodies in the presence of metals utilizing direct current magnetic fields |
EP0427358A1 (en) | 1989-11-08 | 1991-05-15 | George S. Allen | Mechanical arm for and interactive image-guided surgical system |
US5076279A (en) | 1990-07-17 | 1991-12-31 | Acuson Corporation | Needle guide for assembly upon an ultrasound imaging transducer |
US5078140A (en) | 1986-05-08 | 1992-01-07 | Kwoh Yik S | Imaging device - aided robotic stereotaxis system |
US5078142A (en) | 1989-11-21 | 1992-01-07 | Fischer Imaging Corporation | Precision mammographic needle biopsy system |
US5095910A (en) | 1990-04-18 | 1992-03-17 | Advanced Technology Laboratories, Inc. | Ultrasonic imaging of biopsy needle |
US5109276A (en) | 1988-05-27 | 1992-04-28 | The University Of Connecticut | Multi-dimensional multi-spectral imaging system |
US5158088A (en) | 1990-11-14 | 1992-10-27 | Advanced Technology Laboratories, Inc. | Ultrasonic diagnostic systems for imaging medical instruments within the body |
US5161536A (en) | 1991-03-22 | 1992-11-10 | Catheter Technology | Ultrasonic position indicating apparatus and methods |
US5193120A (en) | 1991-02-27 | 1993-03-09 | Mechanical Technology Incorporated | Machine vision three dimensional profiling system |
US5209235A (en) | 1991-09-13 | 1993-05-11 | Cardiovascular Imaging Systems, Inc. | Ultrasonic imaging catheter assembly and method for identification of the same |
US5251127A (en) | 1988-02-01 | 1993-10-05 | Faro Medical Technologies Inc. | Computer-aided surgery apparatus |
US5249581A (en) | 1991-07-15 | 1993-10-05 | Horbal Mark T | Precision bone alignment |
US5261404A (en) | 1991-07-08 | 1993-11-16 | Mick Peter R | Three-dimensional mammal anatomy imaging system and method |
US5265610A (en) | 1991-09-03 | 1993-11-30 | General Electric Company | Multi-planar X-ray fluoroscopy system using radiofrequency fields |
US5271400A (en) | 1992-04-01 | 1993-12-21 | General Electric Company | Tracking system to monitor the position and orientation of a device using magnetic resonance detection of a sample contained within the device |
US5307153A (en) | 1990-06-19 | 1994-04-26 | Fujitsu Limited | Three-dimensional measuring apparatus |
US5309913A (en) | 1992-11-30 | 1994-05-10 | The Cleveland Clinic Foundation | Frameless stereotaxy system |
US5323002A (en) | 1992-03-25 | 1994-06-21 | Texas Instruments Incorporated | Spatial light modulator based optical calibration system |
US5371543A (en) | 1993-03-03 | 1994-12-06 | Texas Instruments Incorporated | Monolithic color wheel |
US5383454A (en) | 1990-10-19 | 1995-01-24 | St. Louis University | System for indicating the position of a surgical probe within a head on an image of the head |
US5394875A (en) | 1993-10-21 | 1995-03-07 | Lewis; Judith T. | Automatic ultrasonic localization of targets implanted in a portion of the anatomy |
US5411026A (en) | 1993-10-08 | 1995-05-02 | Nomos Corporation | Method and apparatus for lesion position verification |
JPH07116164A (en) | 1993-10-25 | 1995-05-09 | Hitachi Medical Corp | Ultrasonic tomographic apparatus |
US5433198A (en) | 1993-03-11 | 1995-07-18 | Desai; Jawahar M. | Apparatus and method for cardiac ablation |
US5433739A (en) | 1993-11-02 | 1995-07-18 | Sluijter; Menno E. | Method and apparatus for heating an intervertebral disc for relief of back pain |
US5443489A (en) | 1993-07-20 | 1995-08-22 | Biosense, Inc. | Apparatus and method for ablation |
US5446798A (en) | 1989-06-20 | 1995-08-29 | Fujitsu Limited | Method and apparatus for measuring position and orientation of an object based on a sequence of projected points |
US5447154A (en) | 1992-07-31 | 1995-09-05 | Universite Joseph Fourier | Method for determining the position of an organ |
US5452024A (en) | 1993-11-01 | 1995-09-19 | Texas Instruments Incorporated | DMD display system |
US5457493A (en) | 1993-09-15 | 1995-10-10 | Texas Instruments Incorporated | Digital micro-mirror based image simulation system |
US5474073A (en) | 1994-11-22 | 1995-12-12 | Advanced Technology Laboratories, Inc. | Ultrasonic diagnostic scanning for three dimensional display |
US5476096A (en) | 1994-02-09 | 1995-12-19 | Vingmed Sound A/S | Analysis and measurement of temporal tissue variations |
US5483961A (en) | 1993-03-19 | 1996-01-16 | Kelly; Patrick J. | Magnetic field digitizer for stereotactic surgery |
US5488431A (en) | 1993-11-04 | 1996-01-30 | Texas Instruments Incorporated | Video data formatter for a multi-channel digital television system without overlap |
US5489952A (en) | 1993-07-14 | 1996-02-06 | Texas Instruments Incorporated | Method and device for multi-format television |
US5491510A (en) | 1993-12-03 | 1996-02-13 | Texas Instruments Incorporated | System and method for simultaneously viewing a scene and an obscured object |
US5494039A (en) | 1993-07-16 | 1996-02-27 | Cryomedical Sciences, Inc. | Biopsy needle insertion guide and method of use in prostate cryosurgery |
WO1996005768A1 (en) | 1994-08-19 | 1996-02-29 | Biosense, Inc. | Medical diagnosis, treatment and imaging systems |
US5503152A (en) | 1994-09-28 | 1996-04-02 | Tetrad Corporation | Ultrasonic transducer assembly and method for three-dimensional imaging |
US5505204A (en) | 1993-05-13 | 1996-04-09 | University Hospital (London) Development Corporation | Ultrasonic blood volume flow rate meter |
US5515856A (en) | 1994-08-30 | 1996-05-14 | Vingmed Sound A/S | Method for generating anatomical M-mode displays |
US5517990A (en) | 1992-11-30 | 1996-05-21 | The Cleveland Clinic Foundation | Stereotaxy wand and tool guide |
US5526051A (en) | 1993-10-27 | 1996-06-11 | Texas Instruments Incorporated | Digital television system |
US5526812A (en) | 1993-06-21 | 1996-06-18 | General Electric Company | Display system for enhancing visualization of body structures during medical procedures |
US5529070A (en) | 1990-11-22 | 1996-06-25 | Advanced Technology Laboratories, Inc. | Acquisition and display of ultrasonic images from sequentially oriented image planes |
US5532997A (en) | 1990-06-06 | 1996-07-02 | Texas Instruments Incorporated | Optical tracking system |
US5531227A (en) | 1994-01-28 | 1996-07-02 | Schneider Medical Technologies, Inc. | Imaging device and method |
US5541723A (en) | 1993-06-21 | 1996-07-30 | Minolta Camera Kabushiki Kaisha | Distance measuring device |
US5558091A (en) | 1993-10-06 | 1996-09-24 | Biosense, Inc. | Magnetic determination of position and orientation |
US5568811A (en) | 1994-10-04 | 1996-10-29 | Vingmed Sound A/S | Method for motion encoding of tissue structures in ultrasonic imaging |
US5579026A (en) | 1993-05-14 | 1996-11-26 | Olympus Optical Co., Ltd. | Image display apparatus of head mounted type |
US5581271A (en) | 1994-12-05 | 1996-12-03 | Hughes Aircraft Company | Head mounted visual display |
US5588948A (en) | 1993-02-17 | 1996-12-31 | Olympus Optical Co. Ltd. | Stereoscopic endoscope |
US5608849A (en) | 1991-08-27 | 1997-03-04 | King, Jr.; Donald | Method of visual guidance for positioning images or data in three-dimensional space |
US5611345A (en) | 1995-04-24 | 1997-03-18 | Hibbeln; John F. | Medical instrument with improved ultrasonic visibility |
US5612753A (en) | 1995-01-27 | 1997-03-18 | Texas Instruments Incorporated | Full-color projection display system using two light modulators |
US5611353A (en) | 1993-06-21 | 1997-03-18 | Osteonics Corp. | Method and apparatus for locating functional structures of the lower leg during knee surgery |
US5625408A (en) | 1993-06-24 | 1997-04-29 | Canon Kabushiki Kaisha | Three-dimensional image recording/reconstructing method and apparatus therefor |
WO1997015249A1 (en) | 1995-10-23 | 1997-05-01 | Cypromed A.S | Method for control of prostheses and other aid means |
US5629794A (en) | 1995-05-31 | 1997-05-13 | Texas Instruments Incorporated | Spatial light modulator having an analog beam for steering light |
US5628327A (en) | 1994-12-15 | 1997-05-13 | Imarx Pharmaceutical Corp. | Apparatus for performing biopsies and the like |
US5630027A (en) | 1994-12-28 | 1997-05-13 | Texas Instruments Incorporated | Method and apparatus for compensating horizontal and vertical alignment errors in display systems |
WO1997017014A1 (en) | 1995-11-10 | 1997-05-15 | Nyfotek As | Surgical instrument |
US5647361A (en) | 1992-09-28 | 1997-07-15 | Fonar Corporation | Magnetic resonance imaging method and apparatus for guiding invasive therapy |
US5647373A (en) | 1993-11-07 | 1997-07-15 | Ultra-Guide Ltd. | Articulated needle guide for ultrasound imaging and method of using same |
WO1997029682A1 (en) | 1996-02-15 | 1997-08-21 | Biosense Inc. | Locatable biopsy needle |
US5660185A (en) | 1995-04-13 | 1997-08-26 | Neovision Corporation | Image-guided biopsy apparatus with enhanced imaging and methods |
US5662111A (en) | 1991-01-28 | 1997-09-02 | Cosman; Eric R. | Process of stereotactic optical navigation |
US5699444A (en) | 1995-03-31 | 1997-12-16 | Synthonics Incorporated | Methods and apparatus for using image data to determine camera location and orientation |
US5701900A (en) | 1995-05-01 | 1997-12-30 | Cedars-Sinai Medical Center | Ultrasonic transducer orientation sensing and display apparatus and method |
US5701898A (en) | 1994-09-02 | 1997-12-30 | The United States Of America As Represented By The Department Of Health And Human Services | Method and system for Doppler ultrasound measurement of blood flow |
US5726670A (en) | 1992-07-20 | 1998-03-10 | Olympus Optical Co., Ltd. | Display apparatus to be mounted on the head or face of an individual |
US5728044A (en) | 1995-03-10 | 1998-03-17 | Shan; Yansong | Sensor device for spacial imaging of endoscopes |
US5758650A (en) | 1996-09-30 | 1998-06-02 | Siemens Medical Systems, Inc. | Universal needle guide for ultrasonic transducers |
US5766135A (en) | 1995-03-08 | 1998-06-16 | Terwilliger; Richard A. | Echogenic needle tip |
US5784098A (en) | 1995-08-28 | 1998-07-21 | Olympus Optical Co., Ltd. | Apparatus for measuring three-dimensional configurations |
US5792147A (en) | 1994-03-17 | 1998-08-11 | Roke Manor Research Ltd. | Video-based systems for computer assisted surgery and localisation |
US5793701A (en) | 1995-04-07 | 1998-08-11 | Acuson Corporation | Method and apparatus for coherent image formation |
US5797849A (en) | 1995-03-28 | 1998-08-25 | Sonometrics Corporation | Method for carrying out a medical procedure using a three-dimensional tracking and imaging system |
US5807395A (en) | 1993-08-27 | 1998-09-15 | Medtronic, Inc. | Method and apparatus for RF ablation and hyperthermia |
US5810008A (en) | 1996-12-03 | 1998-09-22 | Isg Technologies Inc. | Apparatus and method for visualizing ultrasonic images |
US5817022A (en) | 1995-03-28 | 1998-10-06 | Sonometrics Corporation | System for displaying a 2-D ultrasound image within a 3-D viewing environment |
US5820554A (en) | 1993-08-31 | 1998-10-13 | Medtronic, Inc. | Ultrasound biopsy needle |
US5820561A (en) | 1996-07-30 | 1998-10-13 | Vingmed Sound A/S | Analysis and measurement of temporal tissue velocity information |
US5829439A (en) | 1995-06-28 | 1998-11-03 | Hitachi Medical Corporation | Needle-like ultrasonic probe for ultrasonic diagnosis apparatus, method of producing same, and ultrasonic diagnosis apparatus using same |
US5829444A (en) | 1994-09-15 | 1998-11-03 | Visualization Technology, Inc. | Position tracking and imaging system for use in medical applications |
US5870136A (en) | 1997-12-05 | 1999-02-09 | The University Of North Carolina At Chapel Hill | Dynamic generation of imperceptible structured light for tracking and acquisition of three dimensional scene geometry and surface characteristics in interactive three dimensional computer graphics applications |
US5891034A (en) | 1990-10-19 | 1999-04-06 | St. Louis University | System for indicating the position of a surgical probe within a head on an image of the head |
WO1999026534A1 (en) | 1995-07-26 | 1999-06-03 | Burdette Medical Systems, Inc. | Virtual reality 3d visualization for surgical procedures |
US5920395A (en) | 1993-04-22 | 1999-07-06 | Image Guided Technologies, Inc. | System for locating relative positions of objects in three dimensional space |
US5961527A (en) | 1997-01-22 | 1999-10-05 | Barzell Whitmore Maroon Bells, Inc. | Omni-directional precision instrument platform |
US5967991A (en) | 1996-12-03 | 1999-10-19 | Echocath, Inc. | Drive apparatus for an interventional medical device used in an ultrasonic imaging system |
US5991085A (en) | 1995-04-21 | 1999-11-23 | I-O Display Systems Llc | Head-mounted personal visual display apparatus with image generator and holder |
US6016439A (en) | 1996-10-15 | 2000-01-18 | Biosense, Inc. | Method and apparatus for synthetic viewpoint imaging |
US6019724A (en) | 1995-02-22 | 2000-02-01 | Gronningsaeter; Aage | Method for ultrasound guidance during clinical procedures |
US6048312A (en) | 1998-04-23 | 2000-04-11 | Ishrak; Syed Omar | Method and apparatus for three-dimensional ultrasound imaging of biopsy needle |
US6064749A (en) | 1996-08-02 | 2000-05-16 | Hirota; Gentaro | Hybrid tracking for augmented reality using both camera motion detection and landmark tracking |
US6091546A (en) | 1997-10-30 | 2000-07-18 | The Microoptical Corporation | Eyeglass interface system |
US6095982A (en) | 1995-03-14 | 2000-08-01 | Board Of Regents, The University Of Texas System | Spectroscopic method and apparatus for optically detecting abnormal mammalian epithelial tissue |
US6099471A (en) | 1997-10-07 | 2000-08-08 | General Electric Company | Method and apparatus for real-time calculation and display of strain in ultrasound imaging |
US6108130A (en) | 1999-09-10 | 2000-08-22 | Intel Corporation | Stereoscopic image sensor |
US6122538A (en) | 1997-01-16 | 2000-09-19 | Acuson Corporation | Motion--Monitoring method and system for medical devices |
US6122541A (en) | 1995-05-04 | 2000-09-19 | Radionics, Inc. | Head band for frameless stereotactic registration |
US6160666A (en) | 1994-02-07 | 2000-12-12 | I-O Display Systems Llc | Personal visual display system |
US6167296A (en) | 1996-06-28 | 2000-12-26 | The Board Of Trustees Of The Leland Stanford Junior University | Method for volumetric image navigation |
US6181371B1 (en) | 1995-05-30 | 2001-01-30 | Francis J Maguire, Jr. | Apparatus for inducing attitudinal head movements for passive virtual reality |
US6216029B1 (en) | 1995-07-16 | 2001-04-10 | Ultraguide Ltd. | Free-hand aiming of a needle guide |
US6241725B1 (en) | 1993-12-15 | 2001-06-05 | Sherwood Services Ag | High frequency thermal ablation of cancerous tumors and functional targets with image data assistance |
WO2001039683A1 (en) | 1999-12-03 | 2001-06-07 | Sinvent As | Tool navigator |
US6245017B1 (en) | 1998-10-30 | 2001-06-12 | Kabushiki Kaisha Toshiba | 3D ultrasonic diagnostic apparatus |
US6246898B1 (en) | 1995-03-28 | 2001-06-12 | Sonometrics Corporation | Method for carrying out a medical procedure using a three-dimensional tracking and imaging system |
US6248101B1 (en) | 1997-01-22 | 2001-06-19 | Barzell Whitmore Maroon Bells, Inc. | Omni-directional precision instrument platform |
US6261234B1 (en) | 1998-05-07 | 2001-07-17 | Diasonics Ultrasound, Inc. | Method and apparatus for ultrasound imaging with biplane instrument guidance |
US20010016804A1 (en) | 1996-09-04 | 2001-08-23 | Cunningham Richard L. | Surgical simulation interface device and method |
US20010045979A1 (en) | 1995-03-29 | 2001-11-29 | Sanyo Electric Co., Ltd. | Methods for creating an image for a three-dimensional display, for calculating depth information, and for image processing using the depth information |
US6341016B1 (en) | 1999-08-06 | 2002-01-22 | Michael Malione | Method and apparatus for measuring three-dimensional shape of object |
US20020010384A1 (en) | 2000-03-30 | 2002-01-24 | Ramin Shahidi | Apparatus and method for calibrating an endoscope |
US6348058B1 (en) | 1997-12-12 | 2002-02-19 | Surgical Navigation Technologies, Inc. | Image guided spinal surgery guide, system, and method for use thereof |
US6350238B1 (en) | 1999-11-02 | 2002-02-26 | Ge Medical Systems Global Technology Company, Llc | Real-time display of ultrasound in slow motion |
US6352507B1 (en) | 1999-08-23 | 2002-03-05 | G.E. Vingmed Ultrasound As | Method and apparatus for providing real-time calculation and display of tissue deformation in ultrasound imaging |
US20020032772A1 (en) | 2000-09-14 | 2002-03-14 | Bjorn Olstad | Method for searching and analysing information in data networks |
US20020049375A1 (en) | 1999-05-18 | 2002-04-25 | Mediguide Ltd. | Method and apparatus for real time quantitative three-dimensional image reconstruction of a moving organ and intra-body navigation |
US6379302B1 (en) | 1999-10-28 | 2002-04-30 | Surgical Navigation Technologies Inc. | Navigation information overlay onto ultrasound imagery |
US6385475B1 (en) | 1997-03-11 | 2002-05-07 | Philippe Cinquin | Process and device for the preoperative determination of the positioning data of endoprosthetic parts |
US20020077543A1 (en) | 2000-06-27 | 2002-06-20 | Robert Grzeszczuk | Method and apparatus for tracking a medical instrument based on image registration |
US20020077540A1 (en) | 2000-11-17 | 2002-06-20 | Kienzle Thomas C. | Enhanced graphic features for computer assisted surgery system |
US20020103431A1 (en) | 2001-01-30 | 2002-08-01 | Emre Toker | Medical instrument guidance using stereo radiolocation |
US20020105484A1 (en) | 2000-09-25 | 2002-08-08 | Nassir Navab | System and method for calibrating a monocular optical see-through head-mounted display system for augmented reality |
US6442417B1 (en) | 1999-11-29 | 2002-08-27 | The Board Of Trustees Of The Leland Stanford Junior University | Method and apparatus for transforming view orientations in image-guided surgery |
US6447450B1 (en) | 1999-11-02 | 2002-09-10 | Ge Medical Systems Global Technology Company, Llc | ECG gated ultrasonic image compounding |
US6456868B2 (en) | 1999-03-30 | 2002-09-24 | Olympus Optical Co., Ltd. | Navigation apparatus and surgical operation image acquisition/display apparatus using the same |
US20020138008A1 (en) | 2000-01-13 | 2002-09-26 | Kazuhiro Tsujita | Method and apparatus for displaying fluorescence images and method and apparatus for acquiring endoscope images |
US20020135673A1 (en) | 2000-11-03 | 2002-09-26 | Favalora Gregg E. | Three-dimensional display systems |
US20020140814A1 (en) | 2001-03-28 | 2002-10-03 | Koninkiijke Philips Electronics N.V. | Method for assisting an automated video tracking system in reaquiring a target |
US6470207B1 (en) | 1999-03-23 | 2002-10-22 | Surgical Navigation Technologies, Inc. | Navigational guidance via computer-assisted fluoroscopic imaging |
US6471366B1 (en) | 2001-07-30 | 2002-10-29 | The United States Of America As Represented By The Secretary Of The Navy | Depth-compensated underwater light |
US6477400B1 (en) | 1998-08-20 | 2002-11-05 | Sofamor Danek Holdings, Inc. | Fluoroscopic image guided orthopaedic surgery system with intraoperative registration |
US6478793B1 (en) | 1999-06-11 | 2002-11-12 | Sherwood Services Ag | Ablation treatment of bone metastases |
US20020198451A1 (en) | 2001-02-27 | 2002-12-26 | Carson Christopher P. | Surgical navigation systems and processes for high tibial osteotomy |
US6503195B1 (en) | 1999-05-24 | 2003-01-07 | University Of North Carolina At Chapel Hill | Methods and systems for real-time structured light depth extraction and endoscope using real-time structured light depth extraction |
US6518939B1 (en) | 1996-11-08 | 2003-02-11 | Olympus Optical Co., Ltd. | Image observation apparatus |
US6527443B1 (en) | 1999-04-20 | 2003-03-04 | Brainlab Ag | Process and apparatus for image guided treatment with an integration of X-ray detection and navigation system |
US6537217B1 (en) | 2001-08-24 | 2003-03-25 | Ge Medical Systems Global Technology Company, Llc | Method and apparatus for improved spatial and temporal resolution in ultrasound imaging |
US6545706B1 (en) | 1999-07-30 | 2003-04-08 | Electric Planet, Inc. | System, method and article of manufacture for tracking a head of a camera-generated image of a person |
US6546279B1 (en) | 2001-10-12 | 2003-04-08 | University Of Florida | Computer controlled guidance of a biopsy needle |
US6551325B2 (en) | 2000-09-26 | 2003-04-22 | Brainlab Ag | Device, system and method for determining the position of an incision block |
WO2003034705A2 (en) | 2001-10-19 | 2003-04-24 | University Of North Carolina At Chapel Hill | Methods and systems for dynamic virtual convergence and head mountable display |
US6570566B1 (en) | 1999-06-10 | 2003-05-27 | Sony Corporation | Image processing apparatus, image processing method, and program providing medium |
US6575969B1 (en) | 1995-05-04 | 2003-06-10 | Sherwood Services Ag | Cool-tip radiofrequency thermosurgery electrode system for tumor ablation |
US6579240B2 (en) | 2001-06-12 | 2003-06-17 | Ge Medical Systems Global Technology Company, Llc | Ultrasound display of selected movement parameter values |
US6587711B1 (en) | 1999-07-22 | 2003-07-01 | The Research Foundation Of Cuny | Spectral polarizing tomographic dermatoscope |
US6592522B2 (en) | 2001-06-12 | 2003-07-15 | Ge Medical Systems Global Technology Company, Llc | Ultrasound display of displacement |
US6594517B1 (en) | 1998-05-15 | 2003-07-15 | Robin Medical, Inc. | Method and apparatus for generating controlled torques on objects particularly objects inside a living body |
US20030135119A1 (en) | 2001-12-31 | 2003-07-17 | Medison Co., Ltd. | Method and apparatus for enabling a biopsy needle to be observed |
US6597818B2 (en) | 1997-05-09 | 2003-07-22 | Sarnoff Corporation | Method and apparatus for performing geo-spatial registration of imagery |
US6604404B2 (en) | 1997-12-31 | 2003-08-12 | Ultraguide Ltd. | Calibration method and apparatus for calibrating position sensors on scanning transducers |
US20030163142A1 (en) | 1997-11-27 | 2003-08-28 | Yoav Paltieli | System and method for guiding the movements of a device to a target particularly for medical applications |
US20030164172A1 (en) | 2000-06-09 | 2003-09-04 | Chumas Nicole Jane | Method and apparatus for guiding a surgical instrument |
US6616610B2 (en) | 2000-11-16 | 2003-09-09 | Ge Medical Systems Kretztechnik Gmbh & Co. Ohg | Method for determination of the direction of introduction and for controlling the introduction path of biopsy needles |
US6626832B1 (en) | 1999-04-15 | 2003-09-30 | Ultraguide Ltd. | Apparatus and method for detecting the bending of medical invasive tools in medical interventions |
US6652462B2 (en) | 2001-06-12 | 2003-11-25 | Ge Medical Systems Global Technology Company, Llc. | Ultrasound display of movement parameter gradients |
WO2003105289A2 (en) | 2002-06-07 | 2003-12-18 | University Of North Carolina At Chapel Hill | Methods and systems for laser based real-time structured light depth extraction |
US20030231789A1 (en) | 2002-06-18 | 2003-12-18 | Scimed Life Systems, Inc. | Computer generated representation of the imaging pattern of an imaging device |
US6689067B2 (en) | 2001-11-28 | 2004-02-10 | Siemens Corporate Research, Inc. | Method and apparatus for ultrasound guidance of needle biopsies |
US20040034313A1 (en) | 2000-12-15 | 2004-02-19 | Aesculap Ag & Co. Kg | Method and device for determining the mechanical axis of a femur |
US6695786B2 (en) | 2001-03-16 | 2004-02-24 | U-Systems, Inc. | Guide and position monitor for invasive medical instrument |
US6711429B1 (en) | 1998-09-24 | 2004-03-23 | Super Dimension Ltd. | System and method for determining the location of a catheter during an intra-body medical procedure |
US6725082B2 (en) | 1999-03-17 | 2004-04-20 | Synthes U.S.A. | System and method for ligament graft placement |
US20040078036A1 (en) | 2002-10-21 | 2004-04-22 | Yaron Keidar | Real-time monitoring and mapping of ablation lesion formation in the heart |
US6733458B1 (en) | 2001-09-25 | 2004-05-11 | Acuson Corporation | Diagnostic medical ultrasound systems and methods using image based freehand needle guidance |
US20040095507A1 (en) | 2002-11-18 | 2004-05-20 | Medicapture, Inc. | Apparatus and method for capturing, processing and storing still images captured inline from an analog video stream and storing in a digital format on removable non-volatile memory |
US20040116810A1 (en) | 2002-12-17 | 2004-06-17 | Bjorn Olstad | Ultrasound location of anatomical landmarks |
US6766184B2 (en) | 2000-03-28 | 2004-07-20 | Board Of Regents, The University Of Texas System | Methods and apparatus for diagnostic multispectral digital imaging |
US6768496B2 (en) | 2000-03-30 | 2004-07-27 | Siemens Aktiengesellschaft | System and method for generating an image from an image dataset and a video image |
US20040147920A1 (en) | 2002-10-21 | 2004-07-29 | Yaron Keidar | Prediction and assessment of ablation of cardiac tissue |
US20040152970A1 (en) | 2003-01-30 | 2004-08-05 | Mark Hunter | Six degree of freedom alignment display for medical procedures |
US6775404B1 (en) | 1999-03-18 | 2004-08-10 | University Of Washington | Apparatus and method for interactive 3D registration of ultrasound and magnetic resonance images based on a magnetic position sensor |
US6783524B2 (en) | 2001-04-19 | 2004-08-31 | Intuitive Surgical, Inc. | Robotic surgical tool with ultrasound cauterizing and cutting instrument |
US20040215071A1 (en) | 2003-04-25 | 2004-10-28 | Frank Kevin J. | Method and apparatus for performing 2D to 3D registration |
US20040243148A1 (en) | 2003-04-08 | 2004-12-02 | Wasielewski Ray C. | Use of micro- and miniature position sensing devices for use in TKA and THA |
US20040243146A1 (en) | 2002-11-18 | 2004-12-02 | Chesbrough Richard M | Method and apparatus for supporting a medical device |
US20040249281A1 (en) | 2003-06-09 | 2004-12-09 | Bjorn Olstad | Method and apparatus for extracting wall function information relative to ultrasound-located landmarks |
US20040249282A1 (en) | 2003-06-09 | 2004-12-09 | Bjorn Olstad | System and method for extracting information based on ultrasound-located landmarks |
US20040254454A1 (en) | 2001-06-13 | 2004-12-16 | Kockro Ralf Alfons | Guide system and a probe therefor |
US20050010098A1 (en) | 2003-04-11 | 2005-01-13 | Sigmund Frigstad | Method and apparatus for knowledge based diagnostic imaging |
WO2005010711A2 (en) | 2003-07-21 | 2005-02-03 | Johns Hopkins University | Robotic 5-dimensional ultrasound |
US6863655B2 (en) | 2001-06-12 | 2005-03-08 | Ge Medical Systems Global Technology Company, Llc | Ultrasound display of tissue, tracking and tagging |
JP2005058584A (en) | 2003-08-19 | 2005-03-10 | Toshiba Corp | Ultrasonic diagnostic equipment |
US6873867B2 (en) | 2000-04-05 | 2005-03-29 | Brainlab Ag | Referencing or registering a patient or a patient body part in a medical navigation system by means of irradiation of light points |
US6875179B2 (en) | 2002-06-17 | 2005-04-05 | Board Of Trustees Of The University Of Arkansas | Ultrasonic guided catheter deployment system |
US20050085718A1 (en) | 2003-10-21 | 2005-04-21 | Ramin Shahidi | Systems and methods for intraoperative targetting |
US20050085717A1 (en) | 2003-10-21 | 2005-04-21 | Ramin Shahidi | Systems and methods for intraoperative targetting |
US20050090742A1 (en) | 2003-08-19 | 2005-04-28 | Yoshitaka Mine | Ultrasonic diagnostic apparatus |
US6895268B1 (en) | 1999-06-28 | 2005-05-17 | Siemens Aktiengesellschaft | Medical workstation, imaging system, and method for mixing two images |
US20050107679A1 (en) | 2003-07-11 | 2005-05-19 | Bernhard Geiger | System and method for endoscopic path planning |
US20050111733A1 (en) | 2003-11-26 | 2005-05-26 | Fors Steven L. | Automated digitized film slicing and registration tool |
US20050159641A1 (en) | 2004-01-15 | 2005-07-21 | Pentax Corporation | Optical system for stereoscopic rigid endoscope |
US20050182316A1 (en) | 2002-08-29 | 2005-08-18 | Burdette Everette C. | Method and system for localizing a medical tool |
US6936048B2 (en) | 2003-01-16 | 2005-08-30 | Charlotte-Mecklenburg Hospital Authority | Echogenic needle for transvaginal ultrasound directed reduction of uterine fibroids and an associated method |
US6947783B2 (en) | 2000-09-26 | 2005-09-20 | Brainlab Ag | System for the navigation-assisted positioning of elements |
US20050222574A1 (en) | 2002-09-27 | 2005-10-06 | Aesculap Ag & Co. Kg | Method and apparatus for determining the position of the tibial exit point of the anterior cruciate ligament |
US20050251148A1 (en) | 2002-11-05 | 2005-11-10 | Aesculap Ag & Co. Kg | Method and device for determining the position of a knee-joint endoprosthesis |
JP2005323669A (en) | 2004-05-12 | 2005-11-24 | Toshiba Corp | Ultrasonic diagnosing apparatus for puncture treatment |
US6978167B2 (en) | 2002-07-01 | 2005-12-20 | Claron Technology Inc. | Video pose tracking system and method |
US20060004275A1 (en) | 2004-06-30 | 2006-01-05 | Vija A H | Systems and methods for localized image registration and fusion |
US20060020204A1 (en) | 2004-07-01 | 2006-01-26 | Bracco Imaging, S.P.A. | System and method for three-dimensional space management and visualization of ultrasound data ("SonoDEX") |
US20060036162A1 (en) | 2004-02-02 | 2006-02-16 | Ramin Shahidi | Method and apparatus for guiding a medical instrument to a subsurface target site in a patient |
US7008373B2 (en) | 2001-11-08 | 2006-03-07 | The Johns Hopkins University | System and method for robot targeting under fluoroscopy based on image servoing |
US20060052792A1 (en) | 2003-02-26 | 2006-03-09 | Aesculap Ag & Co. Kg | Patella reference device |
US20060058675A1 (en) | 2004-08-31 | 2006-03-16 | General Electric Company | Three dimensional atrium-ventricle plane detection |
US20060058610A1 (en) | 2004-08-31 | 2006-03-16 | General Electric Company | Increasing the efficiency of quantitation in stress echo |
US20060058674A1 (en) | 2004-08-31 | 2006-03-16 | General Electric Company | Optimizing ultrasound acquisition based on ultrasound-located landmarks |
US20060058609A1 (en) | 2004-08-31 | 2006-03-16 | General Electric Company | Extracting ultrasound summary information useful for inexperienced users of ultrasound |
US20060100505A1 (en) | 2004-10-26 | 2006-05-11 | Viswanathan Raju R | Surgical navigation using a three-dimensional user interface |
US20060122495A1 (en) | 2002-11-14 | 2006-06-08 | Kienzle Thomas C Iii | Interchangeable localizing devices for use with tracking systems |
US7072707B2 (en) | 2001-06-27 | 2006-07-04 | Vanderbilt University | Method and apparatus for collecting and processing physical space data for use while performing image-guided surgery |
US20060184040A1 (en) | 2004-12-09 | 2006-08-17 | Keller Kurtis P | Apparatus, system and method for optically analyzing a substrate |
US20060193504A1 (en) | 2003-03-27 | 2006-08-31 | Koninklijke Philips Electronics N.V. | Guidance of invasive medical devices by three dimensional ultrasonic imaging |
US7110013B2 (en) | 2000-03-15 | 2006-09-19 | Information Decision Technology | Augmented reality display integrated with self-contained breathing apparatus |
US20060229594A1 (en) | 2000-01-19 | 2006-10-12 | Medtronic, Inc. | Method for guiding a medical device |
US20060235538A1 (en) | 2005-04-13 | 2006-10-19 | Tornier | Surgical apparatus for implantation of a partial of total knee prosthesis |
US20060235290A1 (en) | 2005-04-04 | 2006-10-19 | Aesculap Ag & Co. Kg | Method and apparatus for positioning a cutting tool for orthopedic surgery using a localization system |
US20060241450A1 (en) | 2003-03-17 | 2006-10-26 | Biotelligent Inc. | Ultrasound guided tissue measurement system |
US20060253032A1 (en) | 2005-04-26 | 2006-11-09 | Altmann Andres C | Display of catheter tip with beam direction for ultrasound system |
US20060271056A1 (en) | 2005-05-10 | 2006-11-30 | Smith & Nephew, Inc. | System and method for modular navigated osteotome |
US20060282023A1 (en) | 2002-12-03 | 2006-12-14 | Aesculap Ag & Co. Kg | Method of determining the position of the articular point of a joint |
US20060293643A1 (en) | 2004-03-05 | 2006-12-28 | Wallace Daniel T | Robotic catheter system |
US20070002582A1 (en) | 2003-03-14 | 2007-01-04 | Light Sciences Corporation | Medical apparatus employing flexible light structures and methods for manufacturing same |
US20070016035A1 (en) | 2005-05-17 | 2007-01-18 | Ge Medical Systems Global Technology Company, Llc | Ultrasonic diagnostic apparatus and ultrasonic image generating method |
US20070032906A1 (en) | 2002-08-13 | 2007-02-08 | Sutherland Garnette R | Microsurgical robot system |
WO2007019216A1 (en) | 2005-08-04 | 2007-02-15 | Teratech Corporation | Integrated ultrasound and magnetic resonance imaging system |
US20070073455A1 (en) | 2005-09-29 | 2007-03-29 | Toyota Jidosha Kabushiki Kaisha | Vehicle preferential treatment system, electrically powered vehicle, server used for vehicle preferential treatment system, and vehicle preferential treatment method |
US20070073155A1 (en) | 2005-09-02 | 2007-03-29 | Ultrasound Ventures, Llc | Ultrasound guidance system |
WO2007067323A2 (en) | 2005-12-02 | 2007-06-14 | Abbott Cardiovascular Systems Inc. | Image-guidance in medical image processing |
US7245746B2 (en) | 2001-06-12 | 2007-07-17 | Ge Medical Systems Global Technology Company, Llc | Ultrasound color characteristic mapping |
US20070167701A1 (en) | 2005-12-26 | 2007-07-19 | Depuy Products, Inc. | Computer assisted orthopaedic surgery system with light source and associated method |
US20070167699A1 (en) | 2005-12-20 | 2007-07-19 | Fabienne Lathuiliere | Methods and systems for segmentation and surface matching |
US7248232B1 (en) | 1998-02-25 | 2007-07-24 | Semiconductor Energy Laboratory Co., Ltd. | Information processing device |
US20070225553A1 (en) | 2003-10-21 | 2007-09-27 | The Board Of Trustees Of The Leland Stanford Junio | Systems and Methods for Intraoperative Targeting |
US20070239281A1 (en) | 2006-01-10 | 2007-10-11 | Brainlab Ab | Femur head center localization |
US20070244488A1 (en) | 2006-03-03 | 2007-10-18 | Robert Metzger | Tensor for use in surgical navigation |
US20070255136A1 (en) | 2006-04-27 | 2007-11-01 | General Electric Company | Method and system for measuring flow through a heart valve |
US20070270718A1 (en) | 2005-04-13 | 2007-11-22 | Tornier | Surgical apparatus for implantation of a partial or total knee prosthesis |
US20070291000A1 (en) | 2006-05-16 | 2007-12-20 | Cheng-Chung Liang | Joy-stick like graphical user interface to adjust 3D cross sectional plane in 3D volume |
US7313430B2 (en) | 2003-08-28 | 2007-12-25 | Medtronic Navigation, Inc. | Method and apparatus for performing stereotactic surgery |
US20080004516A1 (en) | 2006-06-30 | 2008-01-03 | Disilvestro Mark R | Registration pointer and method for registering a bone of a patient to a computer assisted orthopaedic surgery system |
US20080004481A1 (en) | 2006-06-28 | 2008-01-03 | Jeffrey Bax | Apparatus and method for guiding insertion of a medical tool |
WO2008017051A2 (en) | 2006-08-02 | 2008-02-07 | Inneroptic Technology Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US20080039723A1 (en) | 2006-05-18 | 2008-02-14 | Suri Jasjit S | System and method for 3-d biopsy |
US20080051910A1 (en) | 2006-08-08 | 2008-02-28 | Aesculap Ag & Co. Kg | Method and apparatus for positioning a bone prosthesis using a localization system |
US7351205B2 (en) | 2003-01-03 | 2008-04-01 | Civco Medical Instruments Co., Inc. | Shallow angle needle guide apparatus and method |
US20080091106A1 (en) | 2006-10-17 | 2008-04-17 | Medison Co., Ltd. | Ultrasound system for fusing an ultrasound image and an external medical image |
US20080114235A1 (en) | 2006-11-15 | 2008-05-15 | Wisconsin Alumni Research Foundation | System and method for simultaneous 3DPR device tracking and imaging under MR-guidance for therapeutic endovascular interventions |
US7379769B2 (en) | 2003-09-30 | 2008-05-27 | Sunnybrook Health Sciences Center | Hybrid imaging method to monitor medical device delivery and patient support for use in the method |
US20080146939A1 (en) | 2002-06-07 | 2008-06-19 | Mcmorrow Gerald | Apparatus and method for image guided insertion and removal of a cannula or needle |
US7392076B2 (en) | 2003-11-04 | 2008-06-24 | Stryker Leibinger Gmbh & Co. Kg | System and method of registering image data to intra-operatively digitized landmarks |
US20080161824A1 (en) | 2006-12-27 | 2008-07-03 | Howmedica Osteonics Corp. | System and method for performing femoral sizing through navigation |
US7398116B2 (en) | 2003-08-11 | 2008-07-08 | Veran Medical Technologies, Inc. | Methods, apparatuses, and systems useful in conducting image guided interventions |
US20080183080A1 (en) | 2006-10-12 | 2008-07-31 | Innoscion, Llc | Image guided catheter having deployable balloons and pericardial access procedure |
US20080200794A1 (en) | 2007-02-19 | 2008-08-21 | Robert Teichman | Multi-configuration tracknig array and related method |
US20080208031A1 (en) | 2007-02-28 | 2008-08-28 | Wisconsin Alumni Research Foundation | Voltage standing wave suppression for MR-guided therapeutic interventions |
US20080208081A1 (en) | 2005-05-02 | 2008-08-28 | Smith & Nephew, Inc. | System and Method For Determining Tibial Rotation |
US20080214932A1 (en) | 2005-06-15 | 2008-09-04 | Aesculap Ag & Co. Kg | Method and surgical navigation system for creating a recess to receive an acetabulum |
US20080232679A1 (en) | 2005-08-17 | 2008-09-25 | Hahn Daniel V | Apparatus and Method for 3-Dimensional Scanning of an Object |
US20080287794A1 (en) | 2007-05-16 | 2008-11-20 | General Electric Company | Method for implementing an imaging nd navigation system |
US20080287837A1 (en) | 2004-05-21 | 2008-11-20 | Ethicon Endo-Surgery, Inc. | Ultrasound medical system and method |
US20080287805A1 (en) | 2007-05-16 | 2008-11-20 | General Electric Company | System and method to guide an instrument through an imaged subject |
US7466303B2 (en) | 2004-02-10 | 2008-12-16 | Sunnybrook Health Sciences Center | Device and process for manipulating real and virtual objects in three-dimensional space |
US20090024030A1 (en) | 2007-07-20 | 2009-01-22 | Martin Lachaine | Methods and systems for guiding the acquisition of ultrasound images |
US20090036902A1 (en) | 2006-06-06 | 2009-02-05 | Intuitive Surgical, Inc. | Interactive user interfaces for robotic minimally invasive surgical systems |
US7505809B2 (en) | 2003-01-13 | 2009-03-17 | Mediguide Ltd. | Method and system for registering a first image with a second image relative to the body of a patient |
US20090105597A1 (en) | 2006-10-12 | 2009-04-23 | Innoscion, Llc | Image guided catheter having remotely controlled surfaces-mounted and internal ultrasound transducers |
US20090118613A1 (en) | 2007-11-01 | 2009-05-07 | Tyco Healthcare Group Lp | Method for Volume Determination and Geometric Reconstruction |
US20090118724A1 (en) | 2005-10-05 | 2009-05-07 | Roni Zvuloni | Method and Apparatus for Positioning a Medical Instrument |
US20090131783A1 (en) | 2007-11-21 | 2009-05-21 | Surgi-Vision, Inc. | Methods, systems and computer program products for positioning a guidance apparatus relative to a patient |
WO2009063423A1 (en) | 2007-11-16 | 2009-05-22 | Koninklijke Philips Electronics, N.V. | Interventional navigation using 3d contrast-enhanced ultrasound |
US20090137907A1 (en) | 2007-11-22 | 2009-05-28 | Kabushiki Kaisha Toshiba | Imaging diagnosis apparatus having needling navigation control system and a needling navigation controlling method |
WO2009094646A2 (en) | 2008-01-24 | 2009-07-30 | The University Of North Carolina At Chapel Hill | Methods, systems, and computer readable media for image guided ablation |
US20090196480A1 (en) | 2008-02-04 | 2009-08-06 | Alba-Tx, Inc. | Methods And Apparatuses For Planning, Performing, Monitoring And Assessing Thermal Ablation |
US7588541B2 (en) | 2003-12-10 | 2009-09-15 | Sonosite, Inc. | Method and system for positioning a medical device at one or more angles relative to an imaging probe |
US20090234369A1 (en) | 2006-06-19 | 2009-09-17 | Robarts Research Institute | Apparatus for guiding a medical tool |
US7596267B2 (en) | 2003-02-28 | 2009-09-29 | Cedara Software Corp. | Image region segmentation system and method |
US20090312629A1 (en) | 2008-06-13 | 2009-12-17 | Inneroptic Technology Inc. | Correction of relative tracking errors based on a fiducial |
US7652259B2 (en) | 2000-08-21 | 2010-01-26 | Spectrum Dynamics Llc | Apparatus and methods for imaging and attenuation correction |
US7662128B2 (en) | 2002-12-23 | 2010-02-16 | Salcudean Septimiu E | Steerable needle |
US7678052B2 (en) | 2004-04-13 | 2010-03-16 | General Electric Company | Method and apparatus for detecting anatomic structures |
WO2010057315A1 (en) | 2008-11-24 | 2010-05-27 | The University Of British Columbia | Apparatus and method for imaging a medical instrument |
US20100152570A1 (en) | 2006-04-12 | 2010-06-17 | Nassir Navab | Virtual Penetrating Mirror Device for Visualizing Virtual Objects in Angiographic Applications |
US7747305B2 (en) | 2003-06-11 | 2010-06-29 | Case Western Reserve University | Computer-aided-design of skeletal implants |
US20100185087A1 (en) | 2006-08-04 | 2010-07-22 | Nields Morgan W | Methods and apparatuses for performing and monitoring thermal ablation |
US20100208963A1 (en) | 2006-11-27 | 2010-08-19 | Koninklijke Philips Electronics N. V. | System and method for fusing real-time ultrasound images with pre-acquired medical images |
WO2010096419A2 (en) | 2009-02-17 | 2010-08-26 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image guided surgery |
US7797032B2 (en) | 1999-10-28 | 2010-09-14 | Medtronic Navigation, Inc. | Method and system for navigating a catheter probe in the presence of field-influencing objects |
US20100268072A1 (en) | 2007-11-15 | 2010-10-21 | Koninklijke Philips Electronics N.V. | Method and apparatus for positional tracking of therapeutic ultrasound transducer |
US7833168B2 (en) | 2003-08-13 | 2010-11-16 | Envisioneering Medical Technologies, Llc | Targeted biopsy delivery system |
US7833221B2 (en) | 2004-10-22 | 2010-11-16 | Ethicon Endo-Surgery, Inc. | System and method for treatment of tissue using the tissue as a fiducial |
US20100296718A1 (en) | 2009-05-20 | 2010-11-25 | Algotec Systems Ltd. | Soft tissue segmentation using a bone atlas |
US20100298705A1 (en) | 2009-05-20 | 2010-11-25 | Laurent Pelissier | Freehand ultrasound imaging systems and methods for guiding fine elongate instruments |
US20100305448A1 (en) | 2009-05-26 | 2010-12-02 | Anne Cecile Dagonneau | Apparatus and method for indicating ultrasound probe orientation and activation status |
US7846103B2 (en) | 2004-09-17 | 2010-12-07 | Medical Equipment Diversified Services, Inc. | Probe guide for use with medical imaging systems |
US20100312121A1 (en) | 2009-06-09 | 2010-12-09 | Zhonghui Guan | Apparatus for a needle director for an ultrasound transducer probe |
US20100331252A1 (en) | 2009-06-25 | 2010-12-30 | Hamrick Mark W | Myostatin inhibitor enhancement of musculoskeletal repair |
US7876942B2 (en) | 2006-03-30 | 2011-01-25 | Activiews Ltd. | System and method for optical position measurement and guidance of a rigid or semi-flexible tool to a target |
WO2011014687A2 (en) | 2009-07-31 | 2011-02-03 | Inneroptic Technology, Inc. | Dual-tube stereoscope |
US7889905B2 (en) | 2005-05-23 | 2011-02-15 | The Penn State Research Foundation | Fast 3D-2D image registration method with application to continuously guided endoscopy |
US20110046486A1 (en) | 2009-08-18 | 2011-02-24 | Medison Co., Ltd. | Ultrasound image enhancement in an ultrasound system |
US20110057930A1 (en) | 2006-07-26 | 2011-03-10 | Inneroptic Technology Inc. | System and method of using high-speed, high-resolution depth extraction to provide three-dimensional imagery for endoscopy |
US7912849B2 (en) | 2005-05-06 | 2011-03-22 | Microsoft Corporation | Method for determining contextual summary information across documents |
US7920909B2 (en) | 2005-09-13 | 2011-04-05 | Veran Medical Technologies, Inc. | Apparatus and method for automatic image guided accuracy verification |
US20110082351A1 (en) | 2009-10-07 | 2011-04-07 | Inneroptic Technology, Inc. | Representing measurement information during a medical procedure |
US7976469B2 (en) | 2007-06-04 | 2011-07-12 | Medtronic, Inc. | Percutaneous needle guide |
US20110201915A1 (en) | 2008-10-23 | 2011-08-18 | Koninklijke Philips Electronics N.V. | Cardiac and or respiratory gated image acquisition system and method for virtual anatomy enriched real time 2d imaging in interventional radiofrequency ablation or pace maker replacement procecure |
US20110201976A1 (en) | 2005-06-01 | 2011-08-18 | Focus Surgery, Inc. | Laparoscopic hifu probe |
US8023712B2 (en) | 2007-12-27 | 2011-09-20 | Olympus Medical Systems Corp. | Medical system and method for generating medical guide image |
US20110230351A1 (en) | 2010-02-10 | 2011-09-22 | Bayer Cropscience Ag | Spiroheterocyclically Substituted Tetramic Acid Derivatives |
US20110238043A1 (en) | 2010-03-25 | 2011-09-29 | Sonowand As | Attachment device for attaching a rigid body to an arbitrary medical instrument |
US20110251483A1 (en) | 2010-04-12 | 2011-10-13 | Inneroptic Technology, Inc. | Image annotation in image-guided medical procedures |
US8041413B2 (en) | 2006-10-02 | 2011-10-18 | Hansen Medical, Inc. | Systems and methods for three-dimensional ultrasound mapping |
US8052636B2 (en) | 2004-03-05 | 2011-11-08 | Hansen Medical, Inc. | Robotic catheter system and methods |
US20110274324A1 (en) | 2010-05-04 | 2011-11-10 | Logan Clements | System and method for abdominal surface matching using pseudo-features |
US20110282188A1 (en) | 2007-11-26 | 2011-11-17 | C.R. Bard, Inc. | Insertion guidance system for needles and medical components |
US20110288412A1 (en) | 2006-04-20 | 2011-11-24 | Gynesonics, Inc. | Devices and methods for treatment of tissue |
US8066644B2 (en) | 2007-05-17 | 2011-11-29 | Vanderbilt University | System, method and device for positioning a target located within soft tissue in a path of an instrument |
US20110295108A1 (en) | 2007-11-26 | 2011-12-01 | C.R. Bard, Inc. | Apparatus for use with needle insertion guidance system |
US8073528B2 (en) | 2007-09-30 | 2011-12-06 | Intuitive Surgical Operations, Inc. | Tool tracking systems, methods and computer products for image guided surgery |
US8086298B2 (en) | 2008-09-29 | 2011-12-27 | Civco Medical Instruments Co., Inc. | EM tracking systems for use with ultrasound and other imaging modalities |
US20120035473A1 (en) | 2003-03-10 | 2012-02-09 | Focus Surgery, Inc. | Laparoscopic hifu probe |
US20120059260A1 (en) | 2009-04-28 | 2012-03-08 | Koninklijke Philips Electronics N.V. | Biopsy guide system with an ultrasound transducer and method of using same |
US8135669B2 (en) | 2005-10-13 | 2012-03-13 | Microsoft Corporation | Information access with usage-driven metadata feedback |
US8137281B2 (en) | 2005-12-20 | 2012-03-20 | Shenzhen Mindray Bio-Medical Electronics Co., Ltd. | Structure for attaching needle guide to ultrasound probe |
US20120071759A1 (en) | 2010-09-20 | 2012-03-22 | Soma Access Systems, Llc | Virtual image formation method for an ultrasound device |
US20120078094A1 (en) | 2008-09-08 | 2012-03-29 | Olympus Medical Systems Corp. | Ultrasound-Guided Ablation Method and Ultrasound-Guided Ablation System |
US8147408B2 (en) | 2005-08-31 | 2012-04-03 | Sonosite, Inc. | Medical device guide locator |
US8152724B2 (en) | 2003-11-11 | 2012-04-10 | Soma Access Systems, Llc | Ultrasound guided probe device and method of using same |
US8167805B2 (en) | 2005-10-20 | 2012-05-01 | Kona Medical, Inc. | Systems and methods for ultrasound applicator station keeping |
US20120143029A1 (en) | 2007-11-26 | 2012-06-07 | Bard Access Systems, Inc. | Systems and methods for guiding a medical instrument |
US20120138658A1 (en) * | 2010-12-07 | 2012-06-07 | Immersion Corporation | Surgical stapler having haptic feedback |
US20120143055A1 (en) | 2010-12-01 | 2012-06-07 | General Electric Company | Method and system for ultrasound imaging |
US20120165679A1 (en) | 2010-12-22 | 2012-06-28 | C. R. Bard. Inc. | Selectable Angle Needle Guide |
US8216149B2 (en) | 2005-08-11 | 2012-07-10 | Kabushiki Kaisha Toshiba | Puncture adaptor, ultrasonic probe for puncture, ultrasonic diagnostic apparatus for puncture, method for detecting angle of puncture needle |
US8221322B2 (en) | 2002-06-07 | 2012-07-17 | Verathon Inc. | Systems and methods to improve clarity in ultrasound images |
US8228028B2 (en) | 2007-06-05 | 2012-07-24 | Ascension Technology Corporation | Systems and methods for compensating for large moving objects in magnetic-tracking environments |
US20120237105A1 (en) | 2009-12-08 | 2012-09-20 | Koninklijke Philips Electronics N.V. | Ablation treatment planning and device |
US20120259210A1 (en) | 2011-04-11 | 2012-10-11 | Harhen Edward P | Ultrasound guided positioning of cardiac replacement valves with 3d visualization |
US8296797B2 (en) | 2005-10-19 | 2012-10-23 | Microsoft International Holdings B.V. | Intelligent video summaries in information access |
US8340379B2 (en) | 2008-03-07 | 2012-12-25 | Inneroptic Technology, Inc. | Systems and methods for displaying guidance data based on updated deformable imaging data |
US20130030286A1 (en) | 2011-07-28 | 2013-01-31 | Alouani Ali T | Image guided surgery trackers using multiple asynchronous sensors |
US20130044930A1 (en) | 2011-04-18 | 2013-02-21 | Senhu Li | Image segmentation of organs and anatomical structures |
US20130079770A1 (en) | 2011-09-22 | 2013-03-28 | Pathfinder Therapeutics, Inc. | Ire guidance collar |
US20130096497A1 (en) | 2011-10-14 | 2013-04-18 | Intuitive Surgical Operations, Inc. | Catheters with control modes for interchangeable probes |
US20130151533A1 (en) | 2011-12-07 | 2013-06-13 | Microsoft Corporation | Provision of query suggestions independent of query logs |
US20130178745A1 (en) | 2011-09-22 | 2013-07-11 | Pathfinder Therapeutics, Inc. | Ultrasound tracking adapter |
WO2013116240A1 (en) | 2012-01-30 | 2013-08-08 | Inneroptic Technology, Inc. | Multiple medical device guidance |
US20130218024A1 (en) | 2011-10-09 | 2013-08-22 | Clear Guide Medical, Llc | Interventional In-Situ Image-Guidance by Fusing Ultrasound and Video |
US20130249787A1 (en) | 2012-03-23 | 2013-09-26 | Sony Corporation | Head-mounted display |
US8641621B2 (en) | 2009-02-17 | 2014-02-04 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image management in image-guided medical procedures |
US20140051987A1 (en) | 2012-08-14 | 2014-02-20 | Intuitive Surgical Operations, Inc. | Systems and Methods for Configuring Components in a Minimally Invasive Instrument |
US20140078138A1 (en) | 2012-03-06 | 2014-03-20 | Pathfinder Therapeutics, Inc. | Apparatus and methods for localization and relative positioning of a surgical instrument |
US20140201669A1 (en) | 2011-07-28 | 2014-07-17 | Koninklijke Philips N.V. | Ablation planning system |
US20140275997A1 (en) | 2013-03-15 | 2014-09-18 | Intuitive Surgical Operations, Inc. | Shape sensor systems for tracking interventional instruments and mehods of use |
US20140275760A1 (en) | 2013-03-13 | 2014-09-18 | Samsung Electronics Co., Ltd. | Augmented reality image display system and surgical robot system comprising the same |
US20140275810A1 (en) | 2013-03-15 | 2014-09-18 | Inneroptic Technology, Inc. | Sensor mount |
US20140343404A1 (en) | 2013-03-14 | 2014-11-20 | Inneroptic Technology, Inc. | Medical device guidance |
US20140350390A1 (en) | 2012-01-18 | 2014-11-27 | Koninklijke Philips N.V. | Ultrasonic guidance of a needle path during biopsy |
US9282947B2 (en) | 2009-12-01 | 2016-03-15 | Inneroptic Technology, Inc. | Imager focusing based on intraoperative data |
US20160117857A1 (en) | 2014-10-02 | 2016-04-28 | Inneroptic Technology, Inc. | Affected region display |
US20160166336A1 (en) | 2014-12-12 | 2016-06-16 | Inneroptic Technology, Inc. | Surgical guidance intersection display |
US20160196694A1 (en) | 2015-01-05 | 2016-07-07 | Worcester Polytechnic Institute | System and method for controlling immersiveness of head-worn displays |
US20170024903A1 (en) | 2015-07-22 | 2017-01-26 | Inneroptic Technology, Inc. | Medical device approaches |
US20170099479A1 (en) | 2014-05-20 | 2017-04-06 | University Of Washington Through Its Center For Commercialization | Systems and methods for mediated-reality surgical visualization |
US9675319B1 (en) | 2016-02-17 | 2017-06-13 | Inneroptic Technology, Inc. | Loupe display |
US20170348067A1 (en) * | 2016-06-02 | 2017-12-07 | Covidien Lp | Assessment of suture or staple line integrity and localization of potential tissue defects along the suture or staple line |
WO2018080844A1 (en) | 2016-10-27 | 2018-05-03 | Inneroptic Technology, Inc. | Medical instrument navigation |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5806521A (en) | 1996-03-26 | 1998-09-15 | Sandia Corporation | Composite ultrasound imaging apparatus and method |
US7194117B2 (en) | 1999-06-29 | 2007-03-20 | The Research Foundation Of State University Of New York | System and method for performing a three-dimensional virtual examination of objects, such as internal organs |
US6246784B1 (en) | 1997-08-19 | 2001-06-12 | The United States Of America As Represented By The Department Of Health And Human Services | Method for segmenting medical images and detecting surface anomalies in anatomical structures |
DE60208009T2 (en) | 2002-06-17 | 2006-08-24 | Nucletron B.V. | System for real-time planning of radiotherapy |
EP1524011B1 (en) | 2003-10-14 | 2011-05-25 | Nucletron B.V. | Method and apparatus for determining the position of a surgical tool relative to a target volume inside an animal body |
JP4522129B2 (en) | 2004-03-31 | 2010-08-11 | キヤノン株式会社 | Image processing method and image processing apparatus |
US7447344B2 (en) | 2004-04-16 | 2008-11-04 | Siemens Medical Solutions Usa, Inc. | System and method for visualization of pulmonary emboli from high-resolution computed tomography images |
US7379062B2 (en) | 2005-08-01 | 2008-05-27 | Barco Nv | Method for determining a path along a biological object with a lumen |
US9037215B2 (en) | 2007-01-31 | 2015-05-19 | The Penn State Research Foundation | Methods and apparatus for 3D route planning through hollow organs |
US8560118B2 (en) | 2007-04-16 | 2013-10-15 | Neuroarm Surgical Ltd. | Methods, devices, and systems for non-mechanically restricting and/or programming movement of a tool of a manipulator along a single axis |
JP5948057B2 (en) | 2008-11-04 | 2016-07-06 | コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. | Method and system for ultrasound therapy |
US11464578B2 (en) | 2009-02-17 | 2022-10-11 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image management in image-guided medical procedures |
CN102598088A (en) | 2009-11-11 | 2012-07-18 | 艾克提维尤斯有限公司 | Systems & methods for planning and performing percutaneous needle procedures |
EP3406299B1 (en) | 2010-06-09 | 2021-08-04 | Regents of the University of Minnesota | Dual mode ultrasound transducer (dmut) system for controlling delivery of ultrasound therapy |
CN103209653B (en) | 2010-11-18 | 2017-04-26 | 皇家飞利浦电子股份有限公司 | System for probabilistic ablation planning |
JP2012187161A (en) | 2011-03-09 | 2012-10-04 | Fujifilm Corp | Image processing apparatus, image processing method, and image processing program |
US9078665B2 (en) | 2011-09-28 | 2015-07-14 | Angiodynamics, Inc. | Multiple treatment zone ablation probe |
US20150238259A1 (en) | 2014-02-27 | 2015-08-27 | Dan David Albeck | Nerve sparing treatment systems and methods |
US11259879B2 (en) | 2017-08-01 | 2022-03-01 | Inneroptic Technology, Inc. | Selective transparency to assist medical device navigation |
US11484365B2 (en) | 2018-01-23 | 2022-11-01 | Inneroptic Technology, Inc. | Medical image guidance |
-
2017
- 2017-01-25 US US15/415,398 patent/US10278778B2/en active Active
- 2017-10-17 WO PCT/US2017/057042 patent/WO2018080844A1/en active Application Filing
-
2019
- 2019-03-27 US US16/366,537 patent/US10772686B2/en active Active
-
2020
- 2020-08-05 US US16/985,580 patent/US11369439B2/en active Active
Patent Citations (477)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3556079A (en) | 1967-05-16 | 1971-01-19 | Haruo Omizo | Method of puncturing a medical instrument under guidance of ultrasound |
US4058114A (en) | 1974-09-11 | 1977-11-15 | Siemens Aktiengesellschaft | Ultrasonic arrangement for puncturing internal body organs, vessels and the like |
USRE30397E (en) | 1976-04-27 | 1980-09-09 | Three-dimensional ultrasonic imaging of animal soft tissue | |
US4249539A (en) | 1979-02-09 | 1981-02-10 | Technicare Corporation | Ultrasound needle tip localization system |
US4294544A (en) | 1979-08-03 | 1981-10-13 | Altschuler Bruce R | Topographic comparator |
US4390025A (en) | 1980-04-24 | 1983-06-28 | Tokyo Shibaura Denki Kabushiki Kaisha | Ultrasonic display apparatus having means for detecting the position of an ultrasonic probe |
US4407294A (en) | 1982-01-07 | 1983-10-04 | Technicare Corporation | Ultrasound tissue probe localization system |
US4431006A (en) | 1982-01-07 | 1984-02-14 | Technicare Corporation | Passive ultrasound needle probe locator |
US4567896A (en) | 1984-01-20 | 1986-02-04 | Elscint, Inc. | Method and apparatus for calibrating a biopsy attachment for ultrasonic imaging apparatus |
US4583538A (en) | 1984-05-04 | 1986-04-22 | Onik Gary M | Method and apparatus for stereotaxic placement of probes in the body utilizing CT scanner localization |
US4620546A (en) | 1984-06-30 | 1986-11-04 | Kabushiki Kaisha Toshiba | Ultrasound hyperthermia apparatus |
US4671292A (en) | 1985-04-30 | 1987-06-09 | Dymax Corporation | Concentric biopsy probe |
US4839836A (en) | 1986-03-11 | 1989-06-13 | U.S. Philips Corporation | Signal transient improvement arrangement |
US5078140A (en) | 1986-05-08 | 1992-01-07 | Kwoh Yik S | Imaging device - aided robotic stereotaxis system |
US4945305A (en) | 1986-10-09 | 1990-07-31 | Ascension Technology Corporation | Device for quantitatively measuring the relative position and orientation of two bodies in the presence of metals utilizing direct current magnetic fields |
US4884219A (en) | 1987-01-21 | 1989-11-28 | W. Industries Limited | Method and apparatus for the perception of computer-generated imagery |
JPS63290550A (en) | 1987-05-22 | 1988-11-28 | Hitachi Medical Corp | Ultrasonic tomographic apparatus for guiding puncture needle |
US4862873A (en) | 1987-05-27 | 1989-09-05 | Olympus Optical Co., Ltd. | Stereo endoscope |
US4911173A (en) | 1987-11-13 | 1990-03-27 | Diasonics, Inc. | Biopsy attachment for ultrasound probe |
US5251127A (en) | 1988-02-01 | 1993-10-05 | Faro Medical Technologies Inc. | Computer-aided surgery apparatus |
US5109276A (en) | 1988-05-27 | 1992-04-28 | The University Of Connecticut | Multi-dimensional multi-spectral imaging system |
US4899756A (en) | 1988-07-18 | 1990-02-13 | Sonek Jiri D | Articulated needle guide for ultrasound imaging and method of using same |
US5446798A (en) | 1989-06-20 | 1995-08-29 | Fujitsu Limited | Method and apparatus for measuring position and orientation of an object based on a sequence of projected points |
EP0427358A1 (en) | 1989-11-08 | 1991-05-15 | George S. Allen | Mechanical arm for and interactive image-guided surgical system |
US5078142A (en) | 1989-11-21 | 1992-01-07 | Fischer Imaging Corporation | Precision mammographic needle biopsy system |
US5095910A (en) | 1990-04-18 | 1992-03-17 | Advanced Technology Laboratories, Inc. | Ultrasonic imaging of biopsy needle |
US5532997A (en) | 1990-06-06 | 1996-07-02 | Texas Instruments Incorporated | Optical tracking system |
US5307153A (en) | 1990-06-19 | 1994-04-26 | Fujitsu Limited | Three-dimensional measuring apparatus |
US5076279A (en) | 1990-07-17 | 1991-12-31 | Acuson Corporation | Needle guide for assembly upon an ultrasound imaging transducer |
US5891034A (en) | 1990-10-19 | 1999-04-06 | St. Louis University | System for indicating the position of a surgical probe within a head on an image of the head |
US5851183A (en) | 1990-10-19 | 1998-12-22 | St. Louis University | System for indicating the position of a surgical probe within a head on an image of the head |
US5383454B1 (en) | 1990-10-19 | 1996-12-31 | Univ St Louis | System for indicating the position of a surgical probe within a head on an image of the head |
US5383454A (en) | 1990-10-19 | 1995-01-24 | St. Louis University | System for indicating the position of a surgical probe within a head on an image of the head |
US5158088A (en) | 1990-11-14 | 1992-10-27 | Advanced Technology Laboratories, Inc. | Ultrasonic diagnostic systems for imaging medical instruments within the body |
US5529070A (en) | 1990-11-22 | 1996-06-25 | Advanced Technology Laboratories, Inc. | Acquisition and display of ultrasonic images from sequentially oriented image planes |
US5662111A (en) | 1991-01-28 | 1997-09-02 | Cosman; Eric R. | Process of stereotactic optical navigation |
US5193120A (en) | 1991-02-27 | 1993-03-09 | Mechanical Technology Incorporated | Machine vision three dimensional profiling system |
US5161536A (en) | 1991-03-22 | 1992-11-10 | Catheter Technology | Ultrasonic position indicating apparatus and methods |
US5261404A (en) | 1991-07-08 | 1993-11-16 | Mick Peter R | Three-dimensional mammal anatomy imaging system and method |
US5249581A (en) | 1991-07-15 | 1993-10-05 | Horbal Mark T | Precision bone alignment |
US5608849A (en) | 1991-08-27 | 1997-03-04 | King, Jr.; Donald | Method of visual guidance for positioning images or data in three-dimensional space |
US5265610A (en) | 1991-09-03 | 1993-11-30 | General Electric Company | Multi-planar X-ray fluoroscopy system using radiofrequency fields |
US5209235A (en) | 1991-09-13 | 1993-05-11 | Cardiovascular Imaging Systems, Inc. | Ultrasonic imaging catheter assembly and method for identification of the same |
US5323002A (en) | 1992-03-25 | 1994-06-21 | Texas Instruments Incorporated | Spatial light modulator based optical calibration system |
US5271400A (en) | 1992-04-01 | 1993-12-21 | General Electric Company | Tracking system to monitor the position and orientation of a device using magnetic resonance detection of a sample contained within the device |
US5726670A (en) | 1992-07-20 | 1998-03-10 | Olympus Optical Co., Ltd. | Display apparatus to be mounted on the head or face of an individual |
US5447154A (en) | 1992-07-31 | 1995-09-05 | Universite Joseph Fourier | Method for determining the position of an organ |
US5647361A (en) | 1992-09-28 | 1997-07-15 | Fonar Corporation | Magnetic resonance imaging method and apparatus for guiding invasive therapy |
US5309913A (en) | 1992-11-30 | 1994-05-10 | The Cleveland Clinic Foundation | Frameless stereotaxy system |
US5517990A (en) | 1992-11-30 | 1996-05-21 | The Cleveland Clinic Foundation | Stereotaxy wand and tool guide |
US5588948A (en) | 1993-02-17 | 1996-12-31 | Olympus Optical Co. Ltd. | Stereoscopic endoscope |
US5371543A (en) | 1993-03-03 | 1994-12-06 | Texas Instruments Incorporated | Monolithic color wheel |
US5433198A (en) | 1993-03-11 | 1995-07-18 | Desai; Jawahar M. | Apparatus and method for cardiac ablation |
US5483961A (en) | 1993-03-19 | 1996-01-16 | Kelly; Patrick J. | Magnetic field digitizer for stereotactic surgery |
US5920395A (en) | 1993-04-22 | 1999-07-06 | Image Guided Technologies, Inc. | System for locating relative positions of objects in three dimensional space |
US5505204A (en) | 1993-05-13 | 1996-04-09 | University Hospital (London) Development Corporation | Ultrasonic blood volume flow rate meter |
US5579026A (en) | 1993-05-14 | 1996-11-26 | Olympus Optical Co., Ltd. | Image display apparatus of head mounted type |
US5611353A (en) | 1993-06-21 | 1997-03-18 | Osteonics Corp. | Method and apparatus for locating functional structures of the lower leg during knee surgery |
US5526812A (en) | 1993-06-21 | 1996-06-18 | General Electric Company | Display system for enhancing visualization of body structures during medical procedures |
US5541723A (en) | 1993-06-21 | 1996-07-30 | Minolta Camera Kabushiki Kaisha | Distance measuring device |
US5625408A (en) | 1993-06-24 | 1997-04-29 | Canon Kabushiki Kaisha | Three-dimensional image recording/reconstructing method and apparatus therefor |
US5489952A (en) | 1993-07-14 | 1996-02-06 | Texas Instruments Incorporated | Method and device for multi-format television |
US5608468A (en) | 1993-07-14 | 1997-03-04 | Texas Instruments Incorporated | Method and device for multi-format television |
US5570135A (en) | 1993-07-14 | 1996-10-29 | Texas Instruments Incorporated | Method and device for multi-format television |
US5494039A (en) | 1993-07-16 | 1996-02-27 | Cryomedical Sciences, Inc. | Biopsy needle insertion guide and method of use in prostate cryosurgery |
US5443489A (en) | 1993-07-20 | 1995-08-22 | Biosense, Inc. | Apparatus and method for ablation |
US5807395A (en) | 1993-08-27 | 1998-09-15 | Medtronic, Inc. | Method and apparatus for RF ablation and hyperthermia |
US5820554A (en) | 1993-08-31 | 1998-10-13 | Medtronic, Inc. | Ultrasound biopsy needle |
US5457493A (en) | 1993-09-15 | 1995-10-10 | Texas Instruments Incorporated | Digital micro-mirror based image simulation system |
US5558091A (en) | 1993-10-06 | 1996-09-24 | Biosense, Inc. | Magnetic determination of position and orientation |
US5411026A (en) | 1993-10-08 | 1995-05-02 | Nomos Corporation | Method and apparatus for lesion position verification |
US5394875A (en) | 1993-10-21 | 1995-03-07 | Lewis; Judith T. | Automatic ultrasonic localization of targets implanted in a portion of the anatomy |
JPH07116164A (en) | 1993-10-25 | 1995-05-09 | Hitachi Medical Corp | Ultrasonic tomographic apparatus |
US5526051A (en) | 1993-10-27 | 1996-06-11 | Texas Instruments Incorporated | Digital television system |
US5452024A (en) | 1993-11-01 | 1995-09-19 | Texas Instruments Incorporated | DMD display system |
US5433739A (en) | 1993-11-02 | 1995-07-18 | Sluijter; Menno E. | Method and apparatus for heating an intervertebral disc for relief of back pain |
US5488431A (en) | 1993-11-04 | 1996-01-30 | Texas Instruments Incorporated | Video data formatter for a multi-channel digital television system without overlap |
US5647373A (en) | 1993-11-07 | 1997-07-15 | Ultra-Guide Ltd. | Articulated needle guide for ultrasound imaging and method of using same |
US5491510A (en) | 1993-12-03 | 1996-02-13 | Texas Instruments Incorporated | System and method for simultaneously viewing a scene and an obscured object |
US6241725B1 (en) | 1993-12-15 | 2001-06-05 | Sherwood Services Ag | High frequency thermal ablation of cancerous tumors and functional targets with image data assistance |
US5531227A (en) | 1994-01-28 | 1996-07-02 | Schneider Medical Technologies, Inc. | Imaging device and method |
US6160666A (en) | 1994-02-07 | 2000-12-12 | I-O Display Systems Llc | Personal visual display system |
US5476096A (en) | 1994-02-09 | 1995-12-19 | Vingmed Sound A/S | Analysis and measurement of temporal tissue variations |
US5792147A (en) | 1994-03-17 | 1998-08-11 | Roke Manor Research Ltd. | Video-based systems for computer assisted surgery and localisation |
WO1996005768A1 (en) | 1994-08-19 | 1996-02-29 | Biosense, Inc. | Medical diagnosis, treatment and imaging systems |
USRE37088E1 (en) | 1994-08-30 | 2001-03-06 | Vingmed Sound A/S | Method for generating anatomical M-mode displays |
US5515856A (en) | 1994-08-30 | 1996-05-14 | Vingmed Sound A/S | Method for generating anatomical M-mode displays |
US5701898A (en) | 1994-09-02 | 1997-12-30 | The United States Of America As Represented By The Department Of Health And Human Services | Method and system for Doppler ultrasound measurement of blood flow |
US5829444A (en) | 1994-09-15 | 1998-11-03 | Visualization Technology, Inc. | Position tracking and imaging system for use in medical applications |
US5967980A (en) | 1994-09-15 | 1999-10-19 | Visualization Technology, Inc. | Position tracking and imaging system for use in medical applications |
US5503152A (en) | 1994-09-28 | 1996-04-02 | Tetrad Corporation | Ultrasonic transducer assembly and method for three-dimensional imaging |
US5568811A (en) | 1994-10-04 | 1996-10-29 | Vingmed Sound A/S | Method for motion encoding of tissue structures in ultrasonic imaging |
US5474073A (en) | 1994-11-22 | 1995-12-12 | Advanced Technology Laboratories, Inc. | Ultrasonic diagnostic scanning for three dimensional display |
US5581271A (en) | 1994-12-05 | 1996-12-03 | Hughes Aircraft Company | Head mounted visual display |
US5628327A (en) | 1994-12-15 | 1997-05-13 | Imarx Pharmaceutical Corp. | Apparatus for performing biopsies and the like |
US5630027A (en) | 1994-12-28 | 1997-05-13 | Texas Instruments Incorporated | Method and apparatus for compensating horizontal and vertical alignment errors in display systems |
US5612753A (en) | 1995-01-27 | 1997-03-18 | Texas Instruments Incorporated | Full-color projection display system using two light modulators |
US6019724A (en) | 1995-02-22 | 2000-02-01 | Gronningsaeter; Aage | Method for ultrasound guidance during clinical procedures |
US5766135A (en) | 1995-03-08 | 1998-06-16 | Terwilliger; Richard A. | Echogenic needle tip |
US5728044A (en) | 1995-03-10 | 1998-03-17 | Shan; Yansong | Sensor device for spacial imaging of endoscopes |
US6095982A (en) | 1995-03-14 | 2000-08-01 | Board Of Regents, The University Of Texas System | Spectroscopic method and apparatus for optically detecting abnormal mammalian epithelial tissue |
US5797849A (en) | 1995-03-28 | 1998-08-25 | Sonometrics Corporation | Method for carrying out a medical procedure using a three-dimensional tracking and imaging system |
US6246898B1 (en) | 1995-03-28 | 2001-06-12 | Sonometrics Corporation | Method for carrying out a medical procedure using a three-dimensional tracking and imaging system |
US5817022A (en) | 1995-03-28 | 1998-10-06 | Sonometrics Corporation | System for displaying a 2-D ultrasound image within a 3-D viewing environment |
US20010045979A1 (en) | 1995-03-29 | 2001-11-29 | Sanyo Electric Co., Ltd. | Methods for creating an image for a three-dimensional display, for calculating depth information, and for image processing using the depth information |
US5699444A (en) | 1995-03-31 | 1997-12-16 | Synthonics Incorporated | Methods and apparatus for using image data to determine camera location and orientation |
US5793701A (en) | 1995-04-07 | 1998-08-11 | Acuson Corporation | Method and apparatus for coherent image formation |
US5660185A (en) | 1995-04-13 | 1997-08-26 | Neovision Corporation | Image-guided biopsy apparatus with enhanced imaging and methods |
US5991085A (en) | 1995-04-21 | 1999-11-23 | I-O Display Systems Llc | Head-mounted personal visual display apparatus with image generator and holder |
US5611345A (en) | 1995-04-24 | 1997-03-18 | Hibbeln; John F. | Medical instrument with improved ultrasonic visibility |
US5701900A (en) | 1995-05-01 | 1997-12-30 | Cedars-Sinai Medical Center | Ultrasonic transducer orientation sensing and display apparatus and method |
US6122541A (en) | 1995-05-04 | 2000-09-19 | Radionics, Inc. | Head band for frameless stereotactic registration |
US6575969B1 (en) | 1995-05-04 | 2003-06-10 | Sherwood Services Ag | Cool-tip radiofrequency thermosurgery electrode system for tumor ablation |
US6181371B1 (en) | 1995-05-30 | 2001-01-30 | Francis J Maguire, Jr. | Apparatus for inducing attitudinal head movements for passive virtual reality |
US5629794A (en) | 1995-05-31 | 1997-05-13 | Texas Instruments Incorporated | Spatial light modulator having an analog beam for steering light |
US5829439A (en) | 1995-06-28 | 1998-11-03 | Hitachi Medical Corporation | Needle-like ultrasonic probe for ultrasonic diagnosis apparatus, method of producing same, and ultrasonic diagnosis apparatus using same |
US6216029B1 (en) | 1995-07-16 | 2001-04-10 | Ultraguide Ltd. | Free-hand aiming of a needle guide |
WO1999026534A1 (en) | 1995-07-26 | 1999-06-03 | Burdette Medical Systems, Inc. | Virtual reality 3d visualization for surgical procedures |
US7171255B2 (en) | 1995-07-26 | 2007-01-30 | Computerized Medical Systems, Inc. | Virtual reality 3D visualization for surgical procedures |
US20010041838A1 (en) | 1995-07-26 | 2001-11-15 | Holupka Edward J. | Virtual reality 3D visualization for surgical procedures |
US5784098A (en) | 1995-08-28 | 1998-07-21 | Olympus Optical Co., Ltd. | Apparatus for measuring three-dimensional configurations |
WO1997015249A1 (en) | 1995-10-23 | 1997-05-01 | Cypromed A.S | Method for control of prostheses and other aid means |
WO1997017014A1 (en) | 1995-11-10 | 1997-05-15 | Nyfotek As | Surgical instrument |
WO1997029682A1 (en) | 1996-02-15 | 1997-08-21 | Biosense Inc. | Locatable biopsy needle |
US20010007919A1 (en) | 1996-06-28 | 2001-07-12 | Ramin Shahidi | Method and apparatus for volumetric image navigation |
US6591130B2 (en) | 1996-06-28 | 2003-07-08 | The Board Of Trustees Of The Leland Stanford Junior University | Method of image-enhanced endoscopy at a patient site |
US6167296A (en) | 1996-06-28 | 2000-12-26 | The Board Of Trustees Of The Leland Stanford Junior University | Method for volumetric image navigation |
US6529758B2 (en) | 1996-06-28 | 2003-03-04 | The Board Of Trustees Of The Leland Stanford Junior University | Method and apparatus for volumetric image navigation |
US5820561A (en) | 1996-07-30 | 1998-10-13 | Vingmed Sound A/S | Analysis and measurement of temporal tissue velocity information |
US6064749A (en) | 1996-08-02 | 2000-05-16 | Hirota; Gentaro | Hybrid tracking for augmented reality using both camera motion detection and landmark tracking |
US20010016804A1 (en) | 1996-09-04 | 2001-08-23 | Cunningham Richard L. | Surgical simulation interface device and method |
US5758650A (en) | 1996-09-30 | 1998-06-02 | Siemens Medical Systems, Inc. | Universal needle guide for ultrasonic transducers |
US6016439A (en) | 1996-10-15 | 2000-01-18 | Biosense, Inc. | Method and apparatus for synthetic viewpoint imaging |
US6518939B1 (en) | 1996-11-08 | 2003-02-11 | Olympus Optical Co., Ltd. | Image observation apparatus |
US5967991A (en) | 1996-12-03 | 1999-10-19 | Echocath, Inc. | Drive apparatus for an interventional medical device used in an ultrasonic imaging system |
US5810008A (en) | 1996-12-03 | 1998-09-22 | Isg Technologies Inc. | Apparatus and method for visualizing ultrasonic images |
US6122538A (en) | 1997-01-16 | 2000-09-19 | Acuson Corporation | Motion--Monitoring method and system for medical devices |
US5961527A (en) | 1997-01-22 | 1999-10-05 | Barzell Whitmore Maroon Bells, Inc. | Omni-directional precision instrument platform |
US6248101B1 (en) | 1997-01-22 | 2001-06-19 | Barzell Whitmore Maroon Bells, Inc. | Omni-directional precision instrument platform |
US7033360B2 (en) | 1997-03-11 | 2006-04-25 | Aesculap Ag & Co. Kg | Process and device for the preoperative determination of the positioning data endoprosthetic parts |
US6385475B1 (en) | 1997-03-11 | 2002-05-07 | Philippe Cinquin | Process and device for the preoperative determination of the positioning data of endoprosthetic parts |
US20040181144A1 (en) | 1997-03-11 | 2004-09-16 | Aesculap Ag & Co. Kg | Process and device for the preoperative determination of the positioning data of endoprosthetic parts |
US6915150B2 (en) | 1997-03-11 | 2005-07-05 | Aesculap Ag & Co. Kg | Process and device for the preoperative determination of the positioning data of endoprosthetic parts |
US6597818B2 (en) | 1997-05-09 | 2003-07-22 | Sarnoff Corporation | Method and apparatus for performing geo-spatial registration of imagery |
US6099471A (en) | 1997-10-07 | 2000-08-08 | General Electric Company | Method and apparatus for real-time calculation and display of strain in ultrasound imaging |
US6091546A (en) | 1997-10-30 | 2000-07-18 | The Microoptical Corporation | Eyeglass interface system |
US20030163142A1 (en) | 1997-11-27 | 2003-08-28 | Yoav Paltieli | System and method for guiding the movements of a device to a target particularly for medical applications |
US5870136A (en) | 1997-12-05 | 1999-02-09 | The University Of North Carolina At Chapel Hill | Dynamic generation of imperceptible structured light for tracking and acquisition of three dimensional scene geometry and surface characteristics in interactive three dimensional computer graphics applications |
US6348058B1 (en) | 1997-12-12 | 2002-02-19 | Surgical Navigation Technologies, Inc. | Image guided spinal surgery guide, system, and method for use thereof |
US6604404B2 (en) | 1997-12-31 | 2003-08-12 | Ultraguide Ltd. | Calibration method and apparatus for calibrating position sensors on scanning transducers |
US7248232B1 (en) | 1998-02-25 | 2007-07-24 | Semiconductor Energy Laboratory Co., Ltd. | Information processing device |
US6048312A (en) | 1998-04-23 | 2000-04-11 | Ishrak; Syed Omar | Method and apparatus for three-dimensional ultrasound imaging of biopsy needle |
US6261234B1 (en) | 1998-05-07 | 2001-07-17 | Diasonics Ultrasound, Inc. | Method and apparatus for ultrasound imaging with biplane instrument guidance |
US6594517B1 (en) | 1998-05-15 | 2003-07-15 | Robin Medical, Inc. | Method and apparatus for generating controlled torques on objects particularly objects inside a living body |
US6477400B1 (en) | 1998-08-20 | 2002-11-05 | Sofamor Danek Holdings, Inc. | Fluoroscopic image guided orthopaedic surgery system with intraoperative registration |
US6711429B1 (en) | 1998-09-24 | 2004-03-23 | Super Dimension Ltd. | System and method for determining the location of a catheter during an intra-body medical procedure |
US6245017B1 (en) | 1998-10-30 | 2001-06-12 | Kabushiki Kaisha Toshiba | 3D ultrasonic diagnostic apparatus |
US6725082B2 (en) | 1999-03-17 | 2004-04-20 | Synthes U.S.A. | System and method for ligament graft placement |
US6775404B1 (en) | 1999-03-18 | 2004-08-10 | University Of Washington | Apparatus and method for interactive 3D registration of ultrasound and magnetic resonance images based on a magnetic position sensor |
US20030073901A1 (en) | 1999-03-23 | 2003-04-17 | Simon David A. | Navigational guidance via computer-assisted fluoroscopic imaging |
US6470207B1 (en) | 1999-03-23 | 2002-10-22 | Surgical Navigation Technologies, Inc. | Navigational guidance via computer-assisted fluoroscopic imaging |
US6456868B2 (en) | 1999-03-30 | 2002-09-24 | Olympus Optical Co., Ltd. | Navigation apparatus and surgical operation image acquisition/display apparatus using the same |
US6626832B1 (en) | 1999-04-15 | 2003-09-30 | Ultraguide Ltd. | Apparatus and method for detecting the bending of medical invasive tools in medical interventions |
US6527443B1 (en) | 1999-04-20 | 2003-03-04 | Brainlab Ag | Process and apparatus for image guided treatment with an integration of X-ray detection and navigation system |
US20020049375A1 (en) | 1999-05-18 | 2002-04-25 | Mediguide Ltd. | Method and apparatus for real time quantitative three-dimensional image reconstruction of a moving organ and intra-body navigation |
US6503195B1 (en) | 1999-05-24 | 2003-01-07 | University Of North Carolina At Chapel Hill | Methods and systems for real-time structured light depth extraction and endoscope using real-time structured light depth extraction |
US6570566B1 (en) | 1999-06-10 | 2003-05-27 | Sony Corporation | Image processing apparatus, image processing method, and program providing medium |
US7480533B2 (en) | 1999-06-11 | 2009-01-20 | Covidien Ag | Ablation treatment of bone metastases |
US6478793B1 (en) | 1999-06-11 | 2002-11-12 | Sherwood Services Ag | Ablation treatment of bone metastases |
US20030040743A1 (en) | 1999-06-11 | 2003-02-27 | Cosman Eric R. | Ablation treatment of bone metastases |
US20050192564A1 (en) | 1999-06-11 | 2005-09-01 | Cosman Eric R. | Ablation treatment of bone metastases |
US6881214B2 (en) | 1999-06-11 | 2005-04-19 | Sherwood Services Ag | Ablation treatment of bone metastases |
US6895268B1 (en) | 1999-06-28 | 2005-05-17 | Siemens Aktiengesellschaft | Medical workstation, imaging system, and method for mixing two images |
US6587711B1 (en) | 1999-07-22 | 2003-07-01 | The Research Foundation Of Cuny | Spectral polarizing tomographic dermatoscope |
US6545706B1 (en) | 1999-07-30 | 2003-04-08 | Electric Planet, Inc. | System, method and article of manufacture for tracking a head of a camera-generated image of a person |
US6341016B1 (en) | 1999-08-06 | 2002-01-22 | Michael Malione | Method and apparatus for measuring three-dimensional shape of object |
US7077807B2 (en) | 1999-08-23 | 2006-07-18 | G.E. Vingmed Ultrasound As | Method and apparatus for providing real-time calculation and display of tissue deformation in ultrasound imaging |
US7798965B2 (en) | 1999-08-23 | 2010-09-21 | G.E. Vingmed Ultrasound As | Method and apparatus for providing real-time calculation and display of tissue deformation in ultrasound imaging |
US6352507B1 (en) | 1999-08-23 | 2002-03-05 | G.E. Vingmed Ultrasound As | Method and apparatus for providing real-time calculation and display of tissue deformation in ultrasound imaging |
US7261694B2 (en) | 1999-08-23 | 2007-08-28 | G.E. Vingmed Ultrasound As | Method and apparatus for providing real-time calculation and display of tissue deformation in ultrasound imaging |
US6676599B2 (en) | 1999-08-23 | 2004-01-13 | G.E. Vingmed Ultrasound As | Method and apparatus for providing real-time calculation and display of tissue deformation in ultrasound imaging |
US6517485B2 (en) | 1999-08-23 | 2003-02-11 | G.E. Vingmed Ultrasound As | Method and apparatus for providing real-time calculation and display of tissue deformation in ultrasound imaging |
US6108130A (en) | 1999-09-10 | 2000-08-22 | Intel Corporation | Stereoscopic image sensor |
US6669635B2 (en) | 1999-10-28 | 2003-12-30 | Surgical Navigation Technologies, Inc. | Navigation information overlay onto ultrasound imagery |
US20020156375A1 (en) | 1999-10-28 | 2002-10-24 | Paul Kessman | Navigation information overlay onto ultrasound imagery |
US6968224B2 (en) | 1999-10-28 | 2005-11-22 | Surgical Navigation Technologies, Inc. | Method of detecting organ matter shift in a patient |
US7797032B2 (en) | 1999-10-28 | 2010-09-14 | Medtronic Navigation, Inc. | Method and system for navigating a catheter probe in the presence of field-influencing objects |
US6379302B1 (en) | 1999-10-28 | 2002-04-30 | Surgical Navigation Technologies Inc. | Navigation information overlay onto ultrasound imagery |
US6447450B1 (en) | 1999-11-02 | 2002-09-10 | Ge Medical Systems Global Technology Company, Llc | ECG gated ultrasonic image compounding |
US6350238B1 (en) | 1999-11-02 | 2002-02-26 | Ge Medical Systems Global Technology Company, Llc | Real-time display of ultrasound in slow motion |
US6442417B1 (en) | 1999-11-29 | 2002-08-27 | The Board Of Trustees Of The Leland Stanford Junior University | Method and apparatus for transforming view orientations in image-guided surgery |
WO2001039683A1 (en) | 1999-12-03 | 2001-06-07 | Sinvent As | Tool navigator |
US20020138008A1 (en) | 2000-01-13 | 2002-09-26 | Kazuhiro Tsujita | Method and apparatus for displaying fluorescence images and method and apparatus for acquiring endoscope images |
US20060229594A1 (en) | 2000-01-19 | 2006-10-12 | Medtronic, Inc. | Method for guiding a medical device |
US7110013B2 (en) | 2000-03-15 | 2006-09-19 | Information Decision Technology | Augmented reality display integrated with self-contained breathing apparatus |
US6766184B2 (en) | 2000-03-28 | 2004-07-20 | Board Of Regents, The University Of Texas System | Methods and apparatus for diagnostic multispectral digital imaging |
US20020010384A1 (en) | 2000-03-30 | 2002-01-24 | Ramin Shahidi | Apparatus and method for calibrating an endoscope |
US6768496B2 (en) | 2000-03-30 | 2004-07-27 | Siemens Aktiengesellschaft | System and method for generating an image from an image dataset and a video image |
US6511418B2 (en) | 2000-03-30 | 2003-01-28 | The Board Of Trustees Of The Leland Stanford Junior University | Apparatus and method for calibrating and endoscope |
US6873867B2 (en) | 2000-04-05 | 2005-03-29 | Brainlab Ag | Referencing or registering a patient or a patient body part in a medical navigation system by means of irradiation of light points |
US20030164172A1 (en) | 2000-06-09 | 2003-09-04 | Chumas Nicole Jane | Method and apparatus for guiding a surgical instrument |
US6782287B2 (en) | 2000-06-27 | 2004-08-24 | The Board Of Trustees Of The Leland Stanford Junior University | Method and apparatus for tracking a medical instrument based on image registration |
US20020077543A1 (en) | 2000-06-27 | 2002-06-20 | Robert Grzeszczuk | Method and apparatus for tracking a medical instrument based on image registration |
US7652259B2 (en) | 2000-08-21 | 2010-01-26 | Spectrum Dynamics Llc | Apparatus and methods for imaging and attenuation correction |
US20020032772A1 (en) | 2000-09-14 | 2002-03-14 | Bjorn Olstad | Method for searching and analysing information in data networks |
US7093012B2 (en) | 2000-09-14 | 2006-08-15 | Overture Services, Inc. | System and method for enhancing crawling by extracting requests for webpages in an information flow |
US20020105484A1 (en) | 2000-09-25 | 2002-08-08 | Nassir Navab | System and method for calibrating a monocular optical see-through head-mounted display system for augmented reality |
US6551325B2 (en) | 2000-09-26 | 2003-04-22 | Brainlab Ag | Device, system and method for determining the position of an incision block |
US6947783B2 (en) | 2000-09-26 | 2005-09-20 | Brainlab Ag | System for the navigation-assisted positioning of elements |
US20020135673A1 (en) | 2000-11-03 | 2002-09-26 | Favalora Gregg E. | Three-dimensional display systems |
US6616610B2 (en) | 2000-11-16 | 2003-09-09 | Ge Medical Systems Kretztechnik Gmbh & Co. Ohg | Method for determination of the direction of introduction and for controlling the introduction path of biopsy needles |
US20020077540A1 (en) | 2000-11-17 | 2002-06-20 | Kienzle Thomas C. | Enhanced graphic features for computer assisted surgery system |
US6917827B2 (en) | 2000-11-17 | 2005-07-12 | Ge Medical Systems Global Technology Company, Llc | Enhanced graphic features for computer assisted surgery system |
US7331932B2 (en) | 2000-12-15 | 2008-02-19 | Aesculap Ag & Co. Kg | Method and device for determining the mechanical axis of a femur |
US20040034313A1 (en) | 2000-12-15 | 2004-02-19 | Aesculap Ag & Co. Kg | Method and device for determining the mechanical axis of a femur |
US20020103431A1 (en) | 2001-01-30 | 2002-08-01 | Emre Toker | Medical instrument guidance using stereo radiolocation |
US6923817B2 (en) | 2001-02-27 | 2005-08-02 | Smith & Nephew, Inc. | Total knee arthroplasty systems and processes |
US20020198451A1 (en) | 2001-02-27 | 2002-12-26 | Carson Christopher P. | Surgical navigation systems and processes for high tibial osteotomy |
US6827723B2 (en) | 2001-02-27 | 2004-12-07 | Smith & Nephew, Inc. | Surgical navigation systems and processes for unicompartmental knee arthroplasty |
US6695786B2 (en) | 2001-03-16 | 2004-02-24 | U-Systems, Inc. | Guide and position monitor for invasive medical instrument |
US20020140814A1 (en) | 2001-03-28 | 2002-10-03 | Koninkiijke Philips Electronics N.V. | Method for assisting an automated video tracking system in reaquiring a target |
US6783524B2 (en) | 2001-04-19 | 2004-08-31 | Intuitive Surgical, Inc. | Robotic surgical tool with ultrasound cauterizing and cutting instrument |
US6579240B2 (en) | 2001-06-12 | 2003-06-17 | Ge Medical Systems Global Technology Company, Llc | Ultrasound display of selected movement parameter values |
US6652462B2 (en) | 2001-06-12 | 2003-11-25 | Ge Medical Systems Global Technology Company, Llc. | Ultrasound display of movement parameter gradients |
US6592522B2 (en) | 2001-06-12 | 2003-07-15 | Ge Medical Systems Global Technology Company, Llc | Ultrasound display of displacement |
US7245746B2 (en) | 2001-06-12 | 2007-07-17 | Ge Medical Systems Global Technology Company, Llc | Ultrasound color characteristic mapping |
US6863655B2 (en) | 2001-06-12 | 2005-03-08 | Ge Medical Systems Global Technology Company, Llc | Ultrasound display of tissue, tracking and tagging |
US20040254454A1 (en) | 2001-06-13 | 2004-12-16 | Kockro Ralf Alfons | Guide system and a probe therefor |
US7072707B2 (en) | 2001-06-27 | 2006-07-04 | Vanderbilt University | Method and apparatus for collecting and processing physical space data for use while performing image-guided surgery |
US6471366B1 (en) | 2001-07-30 | 2002-10-29 | The United States Of America As Represented By The Secretary Of The Navy | Depth-compensated underwater light |
US6537217B1 (en) | 2001-08-24 | 2003-03-25 | Ge Medical Systems Global Technology Company, Llc | Method and apparatus for improved spatial and temporal resolution in ultrasound imaging |
US6733458B1 (en) | 2001-09-25 | 2004-05-11 | Acuson Corporation | Diagnostic medical ultrasound systems and methods using image based freehand needle guidance |
US6546279B1 (en) | 2001-10-12 | 2003-04-08 | University Of Florida | Computer controlled guidance of a biopsy needle |
WO2003032837A1 (en) | 2001-10-12 | 2003-04-24 | University Of Florida | Computer controlled guidance of a biopsy needle |
US20100045783A1 (en) | 2001-10-19 | 2010-02-25 | Andrei State | Methods and systems for dynamic virtual convergence and head mountable display using same |
US20040238732A1 (en) | 2001-10-19 | 2004-12-02 | Andrei State | Methods and systems for dynamic virtual convergence and head mountable display |
WO2003034705A2 (en) | 2001-10-19 | 2003-04-24 | University Of North Carolina At Chapel Hill | Methods and systems for dynamic virtual convergence and head mountable display |
US7008373B2 (en) | 2001-11-08 | 2006-03-07 | The Johns Hopkins University | System and method for robot targeting under fluoroscopy based on image servoing |
US6689067B2 (en) | 2001-11-28 | 2004-02-10 | Siemens Corporate Research, Inc. | Method and apparatus for ultrasound guidance of needle biopsies |
US20030135119A1 (en) | 2001-12-31 | 2003-07-17 | Medison Co., Ltd. | Method and apparatus for enabling a biopsy needle to be observed |
US6764449B2 (en) | 2001-12-31 | 2004-07-20 | Medison Co., Ltd. | Method and apparatus for enabling a biopsy needle to be observed |
WO2003105289A2 (en) | 2002-06-07 | 2003-12-18 | University Of North Carolina At Chapel Hill | Methods and systems for laser based real-time structured light depth extraction |
US8221322B2 (en) | 2002-06-07 | 2012-07-17 | Verathon Inc. | Systems and methods to improve clarity in ultrasound images |
US7385708B2 (en) | 2002-06-07 | 2008-06-10 | The University Of North Carolina At Chapel Hill | Methods and systems for laser based real-time structured light depth extraction |
US20080146939A1 (en) | 2002-06-07 | 2008-06-19 | Mcmorrow Gerald | Apparatus and method for image guided insertion and removal of a cannula or needle |
US20050219552A1 (en) | 2002-06-07 | 2005-10-06 | Ackerman Jermy D | Methods and systems for laser based real-time structured light depth extraction |
US6875179B2 (en) | 2002-06-17 | 2005-04-05 | Board Of Trustees Of The University Of Arkansas | Ultrasonic guided catheter deployment system |
US20030231789A1 (en) | 2002-06-18 | 2003-12-18 | Scimed Life Systems, Inc. | Computer generated representation of the imaging pattern of an imaging device |
US6978167B2 (en) | 2002-07-01 | 2005-12-20 | Claron Technology Inc. | Video pose tracking system and method |
US20070032906A1 (en) | 2002-08-13 | 2007-02-08 | Sutherland Garnette R | Microsurgical robot system |
US20050182316A1 (en) | 2002-08-29 | 2005-08-18 | Burdette Everette C. | Method and system for localizing a medical tool |
US20050222574A1 (en) | 2002-09-27 | 2005-10-06 | Aesculap Ag & Co. Kg | Method and apparatus for determining the position of the tibial exit point of the anterior cruciate ligament |
US20040078036A1 (en) | 2002-10-21 | 2004-04-22 | Yaron Keidar | Real-time monitoring and mapping of ablation lesion formation in the heart |
US20040147920A1 (en) | 2002-10-21 | 2004-07-29 | Yaron Keidar | Prediction and assessment of ablation of cardiac tissue |
US20050251148A1 (en) | 2002-11-05 | 2005-11-10 | Aesculap Ag & Co. Kg | Method and device for determining the position of a knee-joint endoprosthesis |
US20060122495A1 (en) | 2002-11-14 | 2006-06-08 | Kienzle Thomas C Iii | Interchangeable localizing devices for use with tracking systems |
US20040243146A1 (en) | 2002-11-18 | 2004-12-02 | Chesbrough Richard M | Method and apparatus for supporting a medical device |
US20040095507A1 (en) | 2002-11-18 | 2004-05-20 | Medicapture, Inc. | Apparatus and method for capturing, processing and storing still images captured inline from an analog video stream and storing in a digital format on removable non-volatile memory |
US20060282023A1 (en) | 2002-12-03 | 2006-12-14 | Aesculap Ag & Co. Kg | Method of determining the position of the articular point of a joint |
US7209776B2 (en) | 2002-12-03 | 2007-04-24 | Aesculap Ag & Co. Kg | Method of determining the position of the articular point of a joint |
US20070167771A1 (en) | 2002-12-17 | 2007-07-19 | G.E. Medical Systems Global Technology Co., Llc | Ultrasound location of anatomical landmarks |
US20040116810A1 (en) | 2002-12-17 | 2004-06-17 | Bjorn Olstad | Ultrasound location of anatomical landmarks |
US7662128B2 (en) | 2002-12-23 | 2010-02-16 | Salcudean Septimiu E | Steerable needle |
US7351205B2 (en) | 2003-01-03 | 2008-04-01 | Civco Medical Instruments Co., Inc. | Shallow angle needle guide apparatus and method |
US7505809B2 (en) | 2003-01-13 | 2009-03-17 | Mediguide Ltd. | Method and system for registering a first image with a second image relative to the body of a patient |
US6936048B2 (en) | 2003-01-16 | 2005-08-30 | Charlotte-Mecklenburg Hospital Authority | Echogenic needle for transvaginal ultrasound directed reduction of uterine fibroids and an associated method |
US20040152970A1 (en) | 2003-01-30 | 2004-08-05 | Mark Hunter | Six degree of freedom alignment display for medical procedures |
US20060052792A1 (en) | 2003-02-26 | 2006-03-09 | Aesculap Ag & Co. Kg | Patella reference device |
US7596267B2 (en) | 2003-02-28 | 2009-09-29 | Cedara Software Corp. | Image region segmentation system and method |
US20120035473A1 (en) | 2003-03-10 | 2012-02-09 | Focus Surgery, Inc. | Laparoscopic hifu probe |
US20070002582A1 (en) | 2003-03-14 | 2007-01-04 | Light Sciences Corporation | Medical apparatus employing flexible light structures and methods for manufacturing same |
US20060241450A1 (en) | 2003-03-17 | 2006-10-26 | Biotelligent Inc. | Ultrasound guided tissue measurement system |
US20060193504A1 (en) | 2003-03-27 | 2006-08-31 | Koninklijke Philips Electronics N.V. | Guidance of invasive medical devices by three dimensional ultrasonic imaging |
US20040243148A1 (en) | 2003-04-08 | 2004-12-02 | Wasielewski Ray C. | Use of micro- and miniature position sensing devices for use in TKA and THA |
US20050010098A1 (en) | 2003-04-11 | 2005-01-13 | Sigmund Frigstad | Method and apparatus for knowledge based diagnostic imaging |
US20040215071A1 (en) | 2003-04-25 | 2004-10-28 | Frank Kevin J. | Method and apparatus for performing 2D to 3D registration |
US20040249282A1 (en) | 2003-06-09 | 2004-12-09 | Bjorn Olstad | System and method for extracting information based on ultrasound-located landmarks |
US20040249281A1 (en) | 2003-06-09 | 2004-12-09 | Bjorn Olstad | Method and apparatus for extracting wall function information relative to ultrasound-located landmarks |
US7747305B2 (en) | 2003-06-11 | 2010-06-29 | Case Western Reserve University | Computer-aided-design of skeletal implants |
US20050107679A1 (en) | 2003-07-11 | 2005-05-19 | Bernhard Geiger | System and method for endoscopic path planning |
US20110237947A1 (en) | 2003-07-21 | 2011-09-29 | The Johns Hopkins University | Robotic 5-dimensional ultrasound |
WO2005010711A2 (en) | 2003-07-21 | 2005-02-03 | Johns Hopkins University | Robotic 5-dimensional ultrasound |
US7398116B2 (en) | 2003-08-11 | 2008-07-08 | Veran Medical Technologies, Inc. | Methods, apparatuses, and systems useful in conducting image guided interventions |
US7833168B2 (en) | 2003-08-13 | 2010-11-16 | Envisioneering Medical Technologies, Llc | Targeted biopsy delivery system |
US20050090742A1 (en) | 2003-08-19 | 2005-04-28 | Yoshitaka Mine | Ultrasonic diagnostic apparatus |
JP2005058584A (en) | 2003-08-19 | 2005-03-10 | Toshiba Corp | Ultrasonic diagnostic equipment |
US7313430B2 (en) | 2003-08-28 | 2007-12-25 | Medtronic Navigation, Inc. | Method and apparatus for performing stereotactic surgery |
US7379769B2 (en) | 2003-09-30 | 2008-05-27 | Sunnybrook Health Sciences Center | Hybrid imaging method to monitor medical device delivery and patient support for use in the method |
US8050736B2 (en) | 2003-09-30 | 2011-11-01 | Hologic, Inc. | Hybrid imaging method to monitor medical device delivery and patient support for use in the method |
US20070276234A1 (en) | 2003-10-21 | 2007-11-29 | The Board Of Trustees Of The Leland Stanford Junior University | Systems and Methods for Intraoperative Targeting |
US20050085717A1 (en) | 2003-10-21 | 2005-04-21 | Ramin Shahidi | Systems and methods for intraoperative targetting |
US20050085718A1 (en) | 2003-10-21 | 2005-04-21 | Ramin Shahidi | Systems and methods for intraoperative targetting |
US20070225553A1 (en) | 2003-10-21 | 2007-09-27 | The Board Of Trustees Of The Leland Stanford Junio | Systems and Methods for Intraoperative Targeting |
US7392076B2 (en) | 2003-11-04 | 2008-06-24 | Stryker Leibinger Gmbh & Co. Kg | System and method of registering image data to intra-operatively digitized landmarks |
US8152724B2 (en) | 2003-11-11 | 2012-04-10 | Soma Access Systems, Llc | Ultrasound guided probe device and method of using same |
US20050111733A1 (en) | 2003-11-26 | 2005-05-26 | Fors Steven L. | Automated digitized film slicing and registration tool |
US7588541B2 (en) | 2003-12-10 | 2009-09-15 | Sonosite, Inc. | Method and system for positioning a medical device at one or more angles relative to an imaging probe |
US20050159641A1 (en) | 2004-01-15 | 2005-07-21 | Pentax Corporation | Optical system for stereoscopic rigid endoscope |
US20060036162A1 (en) | 2004-02-02 | 2006-02-16 | Ramin Shahidi | Method and apparatus for guiding a medical instrument to a subsurface target site in a patient |
US7466303B2 (en) | 2004-02-10 | 2008-12-16 | Sunnybrook Health Sciences Center | Device and process for manipulating real and virtual objects in three-dimensional space |
US8052636B2 (en) | 2004-03-05 | 2011-11-08 | Hansen Medical, Inc. | Robotic catheter system and methods |
US20060293643A1 (en) | 2004-03-05 | 2006-12-28 | Wallace Daniel T | Robotic catheter system |
US7678052B2 (en) | 2004-04-13 | 2010-03-16 | General Electric Company | Method and apparatus for detecting anatomic structures |
JP2005323669A (en) | 2004-05-12 | 2005-11-24 | Toshiba Corp | Ultrasonic diagnosing apparatus for puncture treatment |
US20080287837A1 (en) | 2004-05-21 | 2008-11-20 | Ethicon Endo-Surgery, Inc. | Ultrasound medical system and method |
US20060004275A1 (en) | 2004-06-30 | 2006-01-05 | Vija A H | Systems and methods for localized image registration and fusion |
US20060020204A1 (en) | 2004-07-01 | 2006-01-26 | Bracco Imaging, S.P.A. | System and method for three-dimensional space management and visualization of ultrasound data ("SonoDEX") |
US20060058674A1 (en) | 2004-08-31 | 2006-03-16 | General Electric Company | Optimizing ultrasound acquisition based on ultrasound-located landmarks |
US20060058675A1 (en) | 2004-08-31 | 2006-03-16 | General Electric Company | Three dimensional atrium-ventricle plane detection |
US20060058610A1 (en) | 2004-08-31 | 2006-03-16 | General Electric Company | Increasing the efficiency of quantitation in stress echo |
US20060058609A1 (en) | 2004-08-31 | 2006-03-16 | General Electric Company | Extracting ultrasound summary information useful for inexperienced users of ultrasound |
US7846103B2 (en) | 2004-09-17 | 2010-12-07 | Medical Equipment Diversified Services, Inc. | Probe guide for use with medical imaging systems |
US7833221B2 (en) | 2004-10-22 | 2010-11-16 | Ethicon Endo-Surgery, Inc. | System and method for treatment of tissue using the tissue as a fiducial |
US20060100505A1 (en) | 2004-10-26 | 2006-05-11 | Viswanathan Raju R | Surgical navigation using a three-dimensional user interface |
US20060184040A1 (en) | 2004-12-09 | 2006-08-17 | Keller Kurtis P | Apparatus, system and method for optically analyzing a substrate |
US20060235290A1 (en) | 2005-04-04 | 2006-10-19 | Aesculap Ag & Co. Kg | Method and apparatus for positioning a cutting tool for orthopedic surgery using a localization system |
US20060235538A1 (en) | 2005-04-13 | 2006-10-19 | Tornier | Surgical apparatus for implantation of a partial of total knee prosthesis |
US20070270718A1 (en) | 2005-04-13 | 2007-11-22 | Tornier | Surgical apparatus for implantation of a partial or total knee prosthesis |
US20060253030A1 (en) | 2005-04-26 | 2006-11-09 | Altmann Andres C | Registration of electro-anatomical map with pre-acquired image using ultrasound |
US20060253032A1 (en) | 2005-04-26 | 2006-11-09 | Altmann Andres C | Display of catheter tip with beam direction for ultrasound system |
US20080208081A1 (en) | 2005-05-02 | 2008-08-28 | Smith & Nephew, Inc. | System and Method For Determining Tibial Rotation |
US7912849B2 (en) | 2005-05-06 | 2011-03-22 | Microsoft Corporation | Method for determining contextual summary information across documents |
US20060271056A1 (en) | 2005-05-10 | 2006-11-30 | Smith & Nephew, Inc. | System and method for modular navigated osteotome |
US20070016035A1 (en) | 2005-05-17 | 2007-01-18 | Ge Medical Systems Global Technology Company, Llc | Ultrasonic diagnostic apparatus and ultrasonic image generating method |
US7889905B2 (en) | 2005-05-23 | 2011-02-15 | The Penn State Research Foundation | Fast 3D-2D image registration method with application to continuously guided endoscopy |
US20110201976A1 (en) | 2005-06-01 | 2011-08-18 | Focus Surgery, Inc. | Laparoscopic hifu probe |
US8038631B1 (en) | 2005-06-01 | 2011-10-18 | Sanghvi Narendra T | Laparoscopic HIFU probe |
US20080214932A1 (en) | 2005-06-15 | 2008-09-04 | Aesculap Ag & Co. Kg | Method and surgical navigation system for creating a recess to receive an acetabulum |
WO2007019216A1 (en) | 2005-08-04 | 2007-02-15 | Teratech Corporation | Integrated ultrasound and magnetic resonance imaging system |
US20070167705A1 (en) | 2005-08-04 | 2007-07-19 | Chiang Alice M | Integrated ultrasound imaging system |
US8216149B2 (en) | 2005-08-11 | 2012-07-10 | Kabushiki Kaisha Toshiba | Puncture adaptor, ultrasonic probe for puncture, ultrasonic diagnostic apparatus for puncture, method for detecting angle of puncture needle |
US20080232679A1 (en) | 2005-08-17 | 2008-09-25 | Hahn Daniel V | Apparatus and Method for 3-Dimensional Scanning of an Object |
US8147408B2 (en) | 2005-08-31 | 2012-04-03 | Sonosite, Inc. | Medical device guide locator |
US20070073155A1 (en) | 2005-09-02 | 2007-03-29 | Ultrasound Ventures, Llc | Ultrasound guidance system |
US8257264B2 (en) | 2005-09-02 | 2012-09-04 | Ultrasound Ventures, Llc | Ultrasonic probe with a needle clip and method of using same |
US20070078346A1 (en) | 2005-09-02 | 2007-04-05 | Ultrasound Ventures, Llc | Sterile shell for an ultrasonic probe and method of using same |
US7962193B2 (en) | 2005-09-13 | 2011-06-14 | Veran Medical Technologies, Inc. | Apparatus and method for image guided accuracy verification |
US7920909B2 (en) | 2005-09-13 | 2011-04-05 | Veran Medical Technologies, Inc. | Apparatus and method for automatic image guided accuracy verification |
US20070073455A1 (en) | 2005-09-29 | 2007-03-29 | Toyota Jidosha Kabushiki Kaisha | Vehicle preferential treatment system, electrically powered vehicle, server used for vehicle preferential treatment system, and vehicle preferential treatment method |
US20090118724A1 (en) | 2005-10-05 | 2009-05-07 | Roni Zvuloni | Method and Apparatus for Positioning a Medical Instrument |
US8135669B2 (en) | 2005-10-13 | 2012-03-13 | Microsoft Corporation | Information access with usage-driven metadata feedback |
US20130132374A1 (en) | 2005-10-19 | 2013-05-23 | Microsoft Holdings International B.V. | Intelligent video summaries in information access |
US8296797B2 (en) | 2005-10-19 | 2012-10-23 | Microsoft International Holdings B.V. | Intelligent video summaries in information access |
US8167805B2 (en) | 2005-10-20 | 2012-05-01 | Kona Medical, Inc. | Systems and methods for ultrasound applicator station keeping |
JP2009517177A (en) | 2005-12-02 | 2009-04-30 | アボット・カーディオヴァスキュラー・システムズ・インコーポレーテッド | Method and apparatus for image guided medical procedures |
US20070167801A1 (en) | 2005-12-02 | 2007-07-19 | Webler William E | Methods and apparatuses for image guided medical procedures |
WO2007067323A2 (en) | 2005-12-02 | 2007-06-14 | Abbott Cardiovascular Systems Inc. | Image-guidance in medical image processing |
US20070167699A1 (en) | 2005-12-20 | 2007-07-19 | Fabienne Lathuiliere | Methods and systems for segmentation and surface matching |
US8137281B2 (en) | 2005-12-20 | 2012-03-20 | Shenzhen Mindray Bio-Medical Electronics Co., Ltd. | Structure for attaching needle guide to ultrasound probe |
US20070167701A1 (en) | 2005-12-26 | 2007-07-19 | Depuy Products, Inc. | Computer assisted orthopaedic surgery system with light source and associated method |
US20070239281A1 (en) | 2006-01-10 | 2007-10-11 | Brainlab Ab | Femur head center localization |
US20070244488A1 (en) | 2006-03-03 | 2007-10-18 | Robert Metzger | Tensor for use in surgical navigation |
US7876942B2 (en) | 2006-03-30 | 2011-01-25 | Activiews Ltd. | System and method for optical position measurement and guidance of a rigid or semi-flexible tool to a target |
US20100152570A1 (en) | 2006-04-12 | 2010-06-17 | Nassir Navab | Virtual Penetrating Mirror Device for Visualizing Virtual Objects in Angiographic Applications |
US20110288412A1 (en) | 2006-04-20 | 2011-11-24 | Gynesonics, Inc. | Devices and methods for treatment of tissue |
US20070255136A1 (en) | 2006-04-27 | 2007-11-01 | General Electric Company | Method and system for measuring flow through a heart valve |
US20070291000A1 (en) | 2006-05-16 | 2007-12-20 | Cheng-Chung Liang | Joy-stick like graphical user interface to adjust 3D cross sectional plane in 3D volume |
US20080039723A1 (en) | 2006-05-18 | 2008-02-14 | Suri Jasjit S | System and method for 3-d biopsy |
US20090036902A1 (en) | 2006-06-06 | 2009-02-05 | Intuitive Surgical, Inc. | Interactive user interfaces for robotic minimally invasive surgical systems |
US20090234369A1 (en) | 2006-06-19 | 2009-09-17 | Robarts Research Institute | Apparatus for guiding a medical tool |
US20080004481A1 (en) | 2006-06-28 | 2008-01-03 | Jeffrey Bax | Apparatus and method for guiding insertion of a medical tool |
US20080004516A1 (en) | 2006-06-30 | 2008-01-03 | Disilvestro Mark R | Registration pointer and method for registering a bone of a patient to a computer assisted orthopaedic surgery system |
US20110057930A1 (en) | 2006-07-26 | 2011-03-10 | Inneroptic Technology Inc. | System and method of using high-speed, high-resolution depth extraction to provide three-dimensional imagery for endoscopy |
US20120108955A1 (en) | 2006-08-02 | 2012-05-03 | Inneroptic Technology Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US7728868B2 (en) | 2006-08-02 | 2010-06-01 | Inneroptic Technology, Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US20140016848A1 (en) | 2006-08-02 | 2014-01-16 | Inneroptic Technology, Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US20100198045A1 (en) | 2006-08-02 | 2010-08-05 | Inneroptic Technology Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US8350902B2 (en) | 2006-08-02 | 2013-01-08 | Inneroptic Technology, Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US9659345B2 (en) | 2006-08-02 | 2017-05-23 | Inneroptic Technology, Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US20170323424A1 (en) | 2006-08-02 | 2017-11-09 | Inneroptic Technology, Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US8482606B2 (en) | 2006-08-02 | 2013-07-09 | Inneroptic Technology, Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
WO2008017051A2 (en) | 2006-08-02 | 2008-02-07 | Inneroptic Technology Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US20080030578A1 (en) | 2006-08-02 | 2008-02-07 | Inneroptic Technology Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US20100185087A1 (en) | 2006-08-04 | 2010-07-22 | Nields Morgan W | Methods and apparatuses for performing and monitoring thermal ablation |
US20080051910A1 (en) | 2006-08-08 | 2008-02-28 | Aesculap Ag & Co. Kg | Method and apparatus for positioning a bone prosthesis using a localization system |
US8041413B2 (en) | 2006-10-02 | 2011-10-18 | Hansen Medical, Inc. | Systems and methods for three-dimensional ultrasound mapping |
US20090105597A1 (en) | 2006-10-12 | 2009-04-23 | Innoscion, Llc | Image guided catheter having remotely controlled surfaces-mounted and internal ultrasound transducers |
US20080183080A1 (en) | 2006-10-12 | 2008-07-31 | Innoscion, Llc | Image guided catheter having deployable balloons and pericardial access procedure |
US20080091106A1 (en) | 2006-10-17 | 2008-04-17 | Medison Co., Ltd. | Ultrasound system for fusing an ultrasound image and an external medical image |
US20080114235A1 (en) | 2006-11-15 | 2008-05-15 | Wisconsin Alumni Research Foundation | System and method for simultaneous 3DPR device tracking and imaging under MR-guidance for therapeutic endovascular interventions |
US20100208963A1 (en) | 2006-11-27 | 2010-08-19 | Koninklijke Philips Electronics N. V. | System and method for fusing real-time ultrasound images with pre-acquired medical images |
US20080161824A1 (en) | 2006-12-27 | 2008-07-03 | Howmedica Osteonics Corp. | System and method for performing femoral sizing through navigation |
US20080200794A1 (en) | 2007-02-19 | 2008-08-21 | Robert Teichman | Multi-configuration tracknig array and related method |
US20080208031A1 (en) | 2007-02-28 | 2008-08-28 | Wisconsin Alumni Research Foundation | Voltage standing wave suppression for MR-guided therapeutic interventions |
US20080287794A1 (en) | 2007-05-16 | 2008-11-20 | General Electric Company | Method for implementing an imaging nd navigation system |
US20080287805A1 (en) | 2007-05-16 | 2008-11-20 | General Electric Company | System and method to guide an instrument through an imaged subject |
US8066644B2 (en) | 2007-05-17 | 2011-11-29 | Vanderbilt University | System, method and device for positioning a target located within soft tissue in a path of an instrument |
US7976469B2 (en) | 2007-06-04 | 2011-07-12 | Medtronic, Inc. | Percutaneous needle guide |
US8228028B2 (en) | 2007-06-05 | 2012-07-24 | Ascension Technology Corporation | Systems and methods for compensating for large moving objects in magnetic-tracking environments |
US20090024030A1 (en) | 2007-07-20 | 2009-01-22 | Martin Lachaine | Methods and systems for guiding the acquisition of ultrasound images |
US8073528B2 (en) | 2007-09-30 | 2011-12-06 | Intuitive Surgical Operations, Inc. | Tool tracking systems, methods and computer products for image guided surgery |
US20090118613A1 (en) | 2007-11-01 | 2009-05-07 | Tyco Healthcare Group Lp | Method for Volume Determination and Geometric Reconstruction |
US20100268072A1 (en) | 2007-11-15 | 2010-10-21 | Koninklijke Philips Electronics N.V. | Method and apparatus for positional tracking of therapeutic ultrasound transducer |
US20100268085A1 (en) | 2007-11-16 | 2010-10-21 | Koninklijke Philips Electronics N.V. | Interventional navigation using 3d contrast-enhanced ultrasound |
WO2009063423A1 (en) | 2007-11-16 | 2009-05-22 | Koninklijke Philips Electronics, N.V. | Interventional navigation using 3d contrast-enhanced ultrasound |
US20090131783A1 (en) | 2007-11-21 | 2009-05-21 | Surgi-Vision, Inc. | Methods, systems and computer program products for positioning a guidance apparatus relative to a patient |
US20090137907A1 (en) | 2007-11-22 | 2009-05-28 | Kabushiki Kaisha Toshiba | Imaging diagnosis apparatus having needling navigation control system and a needling navigation controlling method |
US20110282188A1 (en) | 2007-11-26 | 2011-11-17 | C.R. Bard, Inc. | Insertion guidance system for needles and medical components |
US20120143029A1 (en) | 2007-11-26 | 2012-06-07 | Bard Access Systems, Inc. | Systems and methods for guiding a medical instrument |
US20110295108A1 (en) | 2007-11-26 | 2011-12-01 | C.R. Bard, Inc. | Apparatus for use with needle insertion guidance system |
US8023712B2 (en) | 2007-12-27 | 2011-09-20 | Olympus Medical Systems Corp. | Medical system and method for generating medical guide image |
US20110046483A1 (en) | 2008-01-24 | 2011-02-24 | Henry Fuchs | Methods, systems, and computer readable media for image guided ablation |
US20160270862A1 (en) | 2008-01-24 | 2016-09-22 | The University Of North Carolina At Chapel Hill | Methods, systems, and computer readable media for image guided ablation |
WO2009094646A2 (en) | 2008-01-24 | 2009-07-30 | The University Of North Carolina At Chapel Hill | Methods, systems, and computer readable media for image guided ablation |
US20090196480A1 (en) | 2008-02-04 | 2009-08-06 | Alba-Tx, Inc. | Methods And Apparatuses For Planning, Performing, Monitoring And Assessing Thermal Ablation |
US8340379B2 (en) | 2008-03-07 | 2012-12-25 | Inneroptic Technology, Inc. | Systems and methods for displaying guidance data based on updated deformable imaging data |
US8831310B2 (en) | 2008-03-07 | 2014-09-09 | Inneroptic Technology, Inc. | Systems and methods for displaying guidance data based on updated deformable imaging data |
US20090312629A1 (en) | 2008-06-13 | 2009-12-17 | Inneroptic Technology Inc. | Correction of relative tracking errors based on a fiducial |
US20120078094A1 (en) | 2008-09-08 | 2012-03-29 | Olympus Medical Systems Corp. | Ultrasound-Guided Ablation Method and Ultrasound-Guided Ablation System |
US8086298B2 (en) | 2008-09-29 | 2011-12-27 | Civco Medical Instruments Co., Inc. | EM tracking systems for use with ultrasound and other imaging modalities |
US20110201915A1 (en) | 2008-10-23 | 2011-08-18 | Koninklijke Philips Electronics N.V. | Cardiac and or respiratory gated image acquisition system and method for virtual anatomy enriched real time 2d imaging in interventional radiofrequency ablation or pace maker replacement procecure |
WO2010057315A1 (en) | 2008-11-24 | 2010-05-27 | The University Of British Columbia | Apparatus and method for imaging a medical instrument |
US20110301451A1 (en) | 2008-11-24 | 2011-12-08 | The University Of British Columbia | Apparatus And Method For Imaging A Medical Instrument |
US8690776B2 (en) | 2009-02-17 | 2014-04-08 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image guided surgery |
US9364294B2 (en) | 2009-02-17 | 2016-06-14 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image management in image-guided medical procedures |
US20170065352A1 (en) | 2009-02-17 | 2017-03-09 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image management in image-guided medical procedures |
US8641621B2 (en) | 2009-02-17 | 2014-02-04 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image management in image-guided medical procedures |
US9398936B2 (en) | 2009-02-17 | 2016-07-26 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image guided surgery |
US8585598B2 (en) | 2009-02-17 | 2013-11-19 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image guided surgery |
US20170128139A1 (en) | 2009-02-17 | 2017-05-11 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image guided surgery |
WO2010096419A2 (en) | 2009-02-17 | 2010-08-26 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image guided surgery |
US20120059260A1 (en) | 2009-04-28 | 2012-03-08 | Koninklijke Philips Electronics N.V. | Biopsy guide system with an ultrasound transducer and method of using same |
US20100296718A1 (en) | 2009-05-20 | 2010-11-25 | Algotec Systems Ltd. | Soft tissue segmentation using a bone atlas |
US20100298705A1 (en) | 2009-05-20 | 2010-11-25 | Laurent Pelissier | Freehand ultrasound imaging systems and methods for guiding fine elongate instruments |
US20100305448A1 (en) | 2009-05-26 | 2010-12-02 | Anne Cecile Dagonneau | Apparatus and method for indicating ultrasound probe orientation and activation status |
US20100312121A1 (en) | 2009-06-09 | 2010-12-09 | Zhonghui Guan | Apparatus for a needle director for an ultrasound transducer probe |
US20100331252A1 (en) | 2009-06-25 | 2010-12-30 | Hamrick Mark W | Myostatin inhibitor enhancement of musculoskeletal repair |
WO2011014687A2 (en) | 2009-07-31 | 2011-02-03 | Inneroptic Technology, Inc. | Dual-tube stereoscope |
US20110043612A1 (en) | 2009-07-31 | 2011-02-24 | Inneroptic Technology Inc. | Dual-tube stereoscope |
US20110046486A1 (en) | 2009-08-18 | 2011-02-24 | Medison Co., Ltd. | Ultrasound image enhancement in an ultrasound system |
US20110082351A1 (en) | 2009-10-07 | 2011-04-07 | Inneroptic Technology, Inc. | Representing measurement information during a medical procedure |
US9282947B2 (en) | 2009-12-01 | 2016-03-15 | Inneroptic Technology, Inc. | Imager focusing based on intraoperative data |
US20120237105A1 (en) | 2009-12-08 | 2012-09-20 | Koninklijke Philips Electronics N.V. | Ablation treatment planning and device |
US20110230351A1 (en) | 2010-02-10 | 2011-09-22 | Bayer Cropscience Ag | Spiroheterocyclically Substituted Tetramic Acid Derivatives |
US20110238043A1 (en) | 2010-03-25 | 2011-09-29 | Sonowand As | Attachment device for attaching a rigid body to an arbitrary medical instrument |
US9107698B2 (en) | 2010-04-12 | 2015-08-18 | Inneroptic Technology, Inc. | Image annotation in image-guided medical procedures |
US20160166334A1 (en) | 2010-04-12 | 2016-06-16 | Inneroptic Technology, Inc. | Image annotation in image-guided medical procedures |
US20110251483A1 (en) | 2010-04-12 | 2011-10-13 | Inneroptic Technology, Inc. | Image annotation in image-guided medical procedures |
US8554307B2 (en) | 2010-04-12 | 2013-10-08 | Inneroptic Technology, Inc. | Image annotation in image-guided medical procedures |
US20110274324A1 (en) | 2010-05-04 | 2011-11-10 | Logan Clements | System and method for abdominal surface matching using pseudo-features |
WO2012169990A2 (en) | 2010-05-04 | 2012-12-13 | Pathfinder Therapeutics, Inc. | System and method for abdominal surface matching using pseudo-features |
US20120071759A1 (en) | 2010-09-20 | 2012-03-22 | Soma Access Systems, Llc | Virtual image formation method for an ultrasound device |
US20120143055A1 (en) | 2010-12-01 | 2012-06-07 | General Electric Company | Method and system for ultrasound imaging |
US20120138658A1 (en) * | 2010-12-07 | 2012-06-07 | Immersion Corporation | Surgical stapler having haptic feedback |
US20120165679A1 (en) | 2010-12-22 | 2012-06-28 | C. R. Bard. Inc. | Selectable Angle Needle Guide |
US20120259210A1 (en) | 2011-04-11 | 2012-10-11 | Harhen Edward P | Ultrasound guided positioning of cardiac replacement valves with 3d visualization |
US20130044930A1 (en) | 2011-04-18 | 2013-02-21 | Senhu Li | Image segmentation of organs and anatomical structures |
US20130030286A1 (en) | 2011-07-28 | 2013-01-31 | Alouani Ali T | Image guided surgery trackers using multiple asynchronous sensors |
US20140201669A1 (en) | 2011-07-28 | 2014-07-17 | Koninklijke Philips N.V. | Ablation planning system |
US20130079770A1 (en) | 2011-09-22 | 2013-03-28 | Pathfinder Therapeutics, Inc. | Ire guidance collar |
US20130178745A1 (en) | 2011-09-22 | 2013-07-11 | Pathfinder Therapeutics, Inc. | Ultrasound tracking adapter |
US20130218024A1 (en) | 2011-10-09 | 2013-08-22 | Clear Guide Medical, Llc | Interventional In-Situ Image-Guidance by Fusing Ultrasound and Video |
US20130096497A1 (en) | 2011-10-14 | 2013-04-18 | Intuitive Surgical Operations, Inc. | Catheters with control modes for interchangeable probes |
US20130151533A1 (en) | 2011-12-07 | 2013-06-13 | Microsoft Corporation | Provision of query suggestions independent of query logs |
US20140350390A1 (en) | 2012-01-18 | 2014-11-27 | Koninklijke Philips N.V. | Ultrasonic guidance of a needle path during biopsy |
US8670816B2 (en) | 2012-01-30 | 2014-03-11 | Inneroptic Technology, Inc. | Multiple medical device guidance |
US20180289344A1 (en) | 2012-01-30 | 2018-10-11 | Inneroptic Technology, Inc. | Multiple medical device guidance |
US20140180074A1 (en) | 2012-01-30 | 2014-06-26 | Inneroptic Technology, Inc. | Multiple medical device guidance |
WO2013116240A1 (en) | 2012-01-30 | 2013-08-08 | Inneroptic Technology, Inc. | Multiple medical device guidance |
US20140078138A1 (en) | 2012-03-06 | 2014-03-20 | Pathfinder Therapeutics, Inc. | Apparatus and methods for localization and relative positioning of a surgical instrument |
US20130249787A1 (en) | 2012-03-23 | 2013-09-26 | Sony Corporation | Head-mounted display |
US20140051987A1 (en) | 2012-08-14 | 2014-02-20 | Intuitive Surgical Operations, Inc. | Systems and Methods for Configuring Components in a Minimally Invasive Instrument |
US20140275760A1 (en) | 2013-03-13 | 2014-09-18 | Samsung Electronics Co., Ltd. | Augmented reality image display system and surgical robot system comprising the same |
US20140343404A1 (en) | 2013-03-14 | 2014-11-20 | Inneroptic Technology, Inc. | Medical device guidance |
US20140275810A1 (en) | 2013-03-15 | 2014-09-18 | Inneroptic Technology, Inc. | Sensor mount |
US20140275997A1 (en) | 2013-03-15 | 2014-09-18 | Intuitive Surgical Operations, Inc. | Shape sensor systems for tracking interventional instruments and mehods of use |
US20170099479A1 (en) | 2014-05-20 | 2017-04-06 | University Of Washington Through Its Center For Commercialization | Systems and methods for mediated-reality surgical visualization |
US9901406B2 (en) | 2014-10-02 | 2018-02-27 | Inneroptic Technology, Inc. | Affected region display associated with a medical device |
US20160117857A1 (en) | 2014-10-02 | 2016-04-28 | Inneroptic Technology, Inc. | Affected region display |
US20180263713A1 (en) | 2014-10-02 | 2018-09-20 | Inneroptic Technology, Inc. | Affected region display associated with a medical device |
US20160166336A1 (en) | 2014-12-12 | 2016-06-16 | Inneroptic Technology, Inc. | Surgical guidance intersection display |
US20160196694A1 (en) | 2015-01-05 | 2016-07-07 | Worcester Polytechnic Institute | System and method for controlling immersiveness of head-worn displays |
US20170024903A1 (en) | 2015-07-22 | 2017-01-26 | Inneroptic Technology, Inc. | Medical device approaches |
US9675319B1 (en) | 2016-02-17 | 2017-06-13 | Inneroptic Technology, Inc. | Loupe display |
US20170360395A1 (en) | 2016-02-17 | 2017-12-21 | Inneroptic Technology, Inc. | Loupe display |
US20170348067A1 (en) * | 2016-06-02 | 2017-12-07 | Covidien Lp | Assessment of suture or staple line integrity and localization of potential tissue defects along the suture or staple line |
WO2018080844A1 (en) | 2016-10-27 | 2018-05-03 | Inneroptic Technology, Inc. | Medical instrument navigation |
Non-Patent Citations (164)
Title |
---|
"3D Laparoscope Technology," http://www.inneroptic.com/tech_3DL.htm, copyright 2007 InnerOptic Technology, Inc. printed Sep. 19, 2007, 2 pages. |
"AIM 3D Needle Placement Software from InnerOptic", Medgadget, Sep. 21, 2012. |
"Cancer Facts & Figures 2004," www.cancer.org/downloads/STT/CAFF_finalPWSecured.pdf, copyright 2004 American Cancer Society, Inc., printed Sep. 19, 2007, 60 pages. |
"David Laserscanner <-Latest News <- Institute for Robotics and Process Control <- Te . . . ," http://www/rob.cs.tu-bs.de/en/news/david, printed Sep. 19, 2007, 1 page. |
"InnerOptic's AIM System Receives DA 510(K) Clearance", InnerOptic Technology, Inc., Sep. 18, 2012. |
"Laser scanned 3d model Final" video, still image of video attached, http://www.youtube.com/watch?v+DaLgIgmoUf8, copyright 2007 YouTube, LLC, printed Sep. 19, 2007, 2 pages. |
"Olympus Endoscopic Ultrasound System," www.olympusamerica.com/msg_section/download_brochures/135_b_gfum130.pdf, printed Sep. 20, 2007, 20 pages. |
"Point Grey Research Inc.-Imaging Products-Triclops SDK Samples," http://www.ptgrey.com/products/triclopsSDK/samples.asp, copyright 2007 Point Grey Research Inc., printed Sep. 19, 2007, 1 page. |
"Robbins, Mike-Computer Vision Research-Stereo Depth Perception," http://www.compumike.com/vision/stereodepth. php, copyright 2007 Michael F. Robbins, printed Sep. 19, 2007, 3 pages. |
"RUE, Registered Ultrasound-Endoscope," copyright 2007 InnerOptic Technology, Inc., 2 pages. |
"Sony Introduces 'head-mount image processing unit' for endoscopic image display" (Jul. 23, 2013), retrieved Sep. 27, 2016, 14 pages, available at http://www.sony.net/SonyInfo/News/Press/201307/13-085E/index.html. |
"Sony Introduces Head-Mounted Display for Endoscopic Surgery" (Jul. 23, 2013), retrieved Sep. 27, 2016, 5 pages, available at http://www.medgaget.com/2013/07/sony-introduces-head-mounted-display-for-endoscopic-surgery.html. |
"Point Grey Research Inc.—Imaging Products—Triclops SDK Samples," http://www.ptgrey.com/products/triclopsSDK/samples.asp, copyright 2007 Point Grey Research Inc., printed Sep. 19, 2007, 1 page. |
"Robbins, Mike—Computer Vision Research—Stereo Depth Perception," http://www.compumike.com/vision/stereodepth. php, copyright 2007 Michael F. Robbins, printed Sep. 19, 2007, 3 pages. |
"Sony Introduces ‘head-mount image processing unit’ for endoscopic image display" (Jul. 23, 2013), retrieved Sep. 27, 2016, 14 pages, available at http://www.sony.net/SonyInfo/News/Press/201307/13-085E/index.html. |
2010/0045783 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2011/0046486 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2014/0016848 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2014/0180074 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2014/0275810 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2014/0343404 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2016/0117857 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2016/0166334 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2016/0166336 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2016/0270862 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2017/0024903 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2017/0065352 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2017/0128139 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2017/0323424 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2017/0360395 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2018/0263713 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
2018/0289344 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
Advertisement, "Inspeck 3DC 3D Capturor," Inspeck 3DC 3D Capturor (www.inspeck.com), 1998. |
Advertisement, "Virtual 3D High Speed Non-Contact Surface Perception," Virtual 3-D Technologies Corporation (www.virtual3dtech.com)., Dec. 21, 1998. |
Advertisements, "Virtuoso," Visual Interface, Inc. (www.visint.com), Dec. 21, 1998. |
AIM Section 5: 510k Summary, submitted by InnerOptic Technology, Inc., in 5 pages, submission date May 17, 2012. |
Akka, "Automatic Software Control of Display Parameters for Stereoscopic Graphics Images," SPIE vol. 1669: Stereoscopic Displays and Applications III, pp. 31-38 (1992). |
Ali et al., "Near Infrared Spectroscopy and Imaging to Probe Differences in Water Content in Normal and Cancer Human Prostate Tissues," Technology in Cancer Research & Treatment; Oct. 2004; 3(5):491-497; Adenine Press. |
Aylward et al., Analysis of the Parameter Space of a Metric for Registering 3D Vascular Images, in W. Niessen and M. Viergever (Eds.): MICCAI 2001, LNCS 2208, pp. 932-939, 2001. |
Aylward et al., Registration and Analysis of Vascular Images, International Journal of Computer Vision 55(2/3), 123-138, 2003. |
Aylward, et al., Intra-Operative 3D Ultrasound Augmentation, Proceedings of the IEEE International Symposium on Biomedical Imaging, Washington, Jul. 2002. |
Azuma et al., "Improving Static and Dynamic Registration in an Optical See-Through HMD," Paper Presented at SIGGRAPH '94 Annual Conference in Orlando, FL, 17 pages. (1994). |
Azuma, "A Survey of Augmented Reality," Presence: Teleoperators and Virtual Environments 6, 4:1-48 (Aug. 1997). |
Badler et al., "Simulating Humans: Computer Graphics, Animation, and Control," Oxford University Press (1993). |
Bajura, Michael et al., "Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient," Computer Graphics, Proceedings of SIGGRAPH 1992, vol. 26(2), pp. 203-210, available from www.cs.unc.edu/˜fuchs/publications/MergVirtObjs92.pdf, printed Sep. 20, 2007, 8 pages. |
Benavides et al., "Multispectral digital colposcopy for in vivo detection of cervical cancer," Optics Express; May 19, 2003; 11(1 0) Optical Society of America; USA. |
Beraldin, J.A. et al., "Optimized Position Sensors for Flying-Spot Active Triangulation Systems," Proceedings of the Fourth International Conference on a 3-D Digital Imaging and Modeling (3DIM), Banff, Alberta, Canada, Oct. 6-10, 2003, pp. 334-341, NRC 47083, copyright 2003 National Research Council of Canada, http:/iit-iti.nrc-cnrc.gc.ca/iit-publications-iti/docs/NRC-47083.pdf, printed Sep. 19, 2007, 9 pages. |
Billinghurst, M. et al., Research Directions in Handheld AR; Int. J. of Virtual Reality 5(2),51-58 (2006). |
Blais, F., "Review of 20 Years of Range Sensor Development," Journal of Electronic Imaging, 13(1): 231-240, Jan. 2004, NRC 46531, copyright 2004 National Research Council of Canada, http://iit-iti.nrc-cnrc.gc.ca/iit-publications-iti/docs/NRC-46531.pdf, printed Sep. 19, 2007, 14 pages. |
Bouguet, Jean-Yves, "Camera Calibration Toolbox for Matlab," www.vision.caltech.edu/bouguetj/calib_doc, printed Sep. 20, 2007, 5 pages. |
Buxton et al.; "Colposcopically directed punch biopsy: a potentially misleading investigation," British Journal of Obstetrics and Gynecology; Dec. 1991; 98:1273-1276. |
Caines, Judy S. et al. Stereotaxic Needle Core Biopsy of Breast Lesions Using a Regular Mammographic Table with an Adaptable Stereotaxic Device, American Journal of Roentgenology, vol. 163, No. 2, Aug. 1994, pp. 317-321. Downloaded from www.ajrorline.org on Jul. 10, 2013. |
Cancer Prevention & Early Detection Facts & Figures 2004; National Center for Tobacco-Free Kids; 2004; American Cancer Society; USA. |
Cantor et al., "Cost-Effectiveness Analysis of Diagnosis and Management of Cervical Squamous Intraepithelial Lesions," Diagnostic Strategies for SILs; Feb. 1998; 91(2):270-277. |
Catalano et al. "Multiphase helical CT findings after percutaneous ablation procedures for hepatocellular carcinoma." Abdom. Imaging, 25(6),2000, pp. 607-614. |
Chiriboga et al., "Infrared Spectroscopy of Human Tissue. IV. Detection of Dysplastic and Neoplastic Changes of Human Cervical Tissue Via Infrared Microscopy," Cellular and Molecular Biology; 1998; 44(1): 219-229. |
Crawford, David E. et al., "Computer Modeling of Prostate Biopsy: Tumor Size and Location-Not Clinical Significance-Determine Cancer Detection," Journal of Urology, Apr. 1998, vol. 159(4), pp. 1260-1264, 5 pages. |
Crawford, David E. et al., "Computer Modeling of Prostate Biopsy: Tumor Size and Location—Not Clinical Significance—Determine Cancer Detection," Journal of Urology, Apr. 1998, vol. 159(4), pp. 1260-1264, 5 pages. |
Deering, Michael "High Resolution Virtual Reality." Proceedings of SIGGRAPH '92, Computer Graphics, 26(2), 1992, pp. 195-202. |
Depiero et al., "3-D Computer Vision Using Structured Light: Design, Calibration and Implementation Issues," The University of Tennessee, pp. 1-46, (1996). |
Dodd, G.D. et al. "Minimally invasive treatment of malignant hepatic tumors: at the threshold of a major breakthrough." Radiographies 20(1),2000, pp. 9-27. |
Drascic et al., "Perceptual Issues in Augmented Reality," SPIE vol. 2653: Stereoscopic Displays and Virtual Reality Systems III, pp. 123-134 (Feb. 1996). |
Dumoulin, C.L. et al, Real-Time Position Monitoring of Invasive Devices Using Magnetic Resonance, Magnetic Resonance in Medicine, vol. 29, Issue 3, Mar. 1993, pp. 411-415. |
Fahey et al., "Meta-analysis of Pap Test Accuracy; American Journal of Epidemiology," 1995 141(7):680-689; The John Hopkins University School of Hygiene and Public Health; USA. |
Foxlin et al., "An Inertial Head-Orientation Tracker with Automatic Drift Compensation for Use with HMD's," Proceedings of the 1994 Virtual Reality Software and Technology Conference, Aug. 23-26, 1994, Singapore, pp. 159-173 (1994). |
Fronheiser et al., Real-Time 3D Color Doppler for Guidance of Vibrating Interventional Devices, IEEE Ultrasonics Symposium, pp. 149-152 (2004). |
Fuchs, et al.: "Optimizing a Head-Tracked Stereo Display System to Guide Hepatic Tumor Ablation," Departments of Computer Sciences and Radiology, and School of Medicine, University of North Carolina at Chapel Hill; InnerOptic Technology, Inc. 2008. |
Fuchs, et al.: "Virtual Environments Technology to Aid Needle Biopsies of the Breast," Health Care in the Information Age, Ch. 6, pp. 60-61, Presented in San Diego, Jan. 1720, 1996, published by IOS Press and Ohmsha Feb. 1996. |
Fuchs, Henry et al. "Augmented Reality Visualization for Laparoscopic Surgery," Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI) 1998, pp. 934-943, available from www.cs.unc.edu/˜fuchs/publications/AugRealVis_LaparoSurg98.pdf, printed Sep. 20, 2007, 10 pages. |
Fuhrmann A. et al., Comprehensive calibration and registration procedures for augmented reality; Proc. Eurographics Workshop on Virtual Environments 2001,219--228 (2001). |
Garrett, William F. et al., "Real-Time Incremental Visualization of Dynamic Ultrasound Volumes Using Parallel BSP Trees," Proceedings of IEEE Visualization 1996, pp. 235-240, available from www.cs.unc.edu/˜andrei/pubs/1996_VIS_dualBSP_Mac.pdf, printed Sep. 20, 2007, 7 pages. |
Georgakoudi et al., "Trimodal spectroscopy for the detection and characterization of cervical precancers in vivo," American Journal of Obstetrics and Gynecology; Mar. 2002; 186(3):374-382; USA. |
Herline et al., Surface Registration for Use in Interactive, Image-Guided Liver Surgery, Computer Aided Surgery 5:11-17 (2000). |
Holloway, R.; Registration Error Analysis for Augmented Reality; Presence: Teleoperators and Virtual Environments 6(4), 413--432 (1997). |
Hornung et al., "Quantitative near-infrared spectroscopy of cervical dysplasia in vivo," Human Reproduction; 1999; 14(11):2908-2916; European Society of Human Reproduction and Embryology. |
Howard, M.D., et al.: "An Electronic Device for Needle Placement during Sonographically Guided Percutaneous Intervention", Radiology 2001; 218:905-911. |
InnerAim Brochure; 3D Visualization Software for Simpler, Safer, more Precise Aiming, Published no earlier than Apr. 1, 2010. |
International Search Report and Written Opinion, re PCT Application No. PCT/US2017/057042, dated Feb. 28, 2018. |
InVision System Brochure; A "GPS" for Real-Time 3D Needle Visualization & Guidance, Published no earlier than Mar. 1, 2008. |
InVision User Manual; Professional Instructions for Use, Published no earlier than Dec. 1, 2008. |
Jacobs, Marco C. et al., "Managing Latency in Complex Augmented Reality Systems," ACM SIGGRAPH Proceedings of the Symposium of Interactive 3D Graphics 1997, pp. 49-54, available from www.cs.unc.edu/˜us/Latency//ManagingRelativeLatency.html, printed Sep. 20, 2007, 12 pages. |
Jolesz, Ferenc A, M.D., et al. MRI-Guided Laser-Induced Interstitial Thermotherapy: Basic Principles, SPIE Institute on Laser-Induced Interstitial Thermotherapy (L1TT), Jun. 22-23, 1995, Berlin, Germany. |
Kadi, A Majeed, et al., Design and Simulation of an Articulated Surgical Arm for Guiding Sterotactic Neurosurgery, SPIE vol. 1708 Applications of Artificial Intelligence X: Machine Vision and Robotics (1992). Downloaded from: http://proceedings.spiedigitallibrary.org/ on Jul. 11, 2013. |
Kanbara et al., "A Stereoscopic Video See-through Augmented Reality System Based on Real-time Vision-Based Registration," Nara Institute of Science and Technology, pp. 1-8 (2000). |
Kato, Amami, et al., A frameless, armless navigational system for computer-assisted neurosurgery, Journal of Neurosurgery, vol. 74, No. 5, May 1991, pp. 845-849. |
Keller et al., "What is it in Head Mounted Displays (MDs) that really make them all so terrible?," pp. 1-8 (1998). |
Lass, Amir, "Assessment of Ovarian Reserve," Human Reproduction, 2004, vol. 19(3), pp. 467-469, available from http://humrep.oxfordjournals.orgcgi/reprint/19/3/467, printed Sep. 20, 2007, 3 pages. |
Lee et al., "Modeling Real Objects Using Video See-Through Augmented Reality," Presence, 11(2):144-157 (Apr. 2002). |
Lee, et al., "Modeling Real Objects Using Video See-Through Augmented Reality," Proceedings of the Second International Symposium on Mixed Reality, ISMR 2001, pp. 19-26 (Mar. 14-15, 2001). |
Leven et al., DaVinci Canvas: A Telerobotic Surgical System with Integrated, Robot-Assisted, Laparoscopic Ultrasound Capability, in J. Duncan and G. Gerig (Eds.): MICCAI 2005, LNCS 3749, pp. 811-818, 2005. |
Levy, et al., An Internet-Connected, Patient Specific, Deformable Brain Atlas Integrated into a Surgical Navigation System, Journal of Digital Imaging, vol. 10, No. 3. Suppl. 1 Aug. 1997: pp. 231-237. |
Lindeman, A Low-Cost, Low-latency Approach to Dynamic Immersion in Occlusive Head-Mounted Displays, University of Canterbury, WPI,-Poster from IEEE VR 2016, Mar. 19-23, 2016. |
Lindeman, A Low-Cost, Low-latency Approach to Dynamic Immersion in Occlusive Head-Mounted Displays, University of Canterbury, WPI,—Poster from IEEE VR 2016, Mar. 19-23, 2016. |
Livingston, Mark A. et al., "Magnetic Tracker Calibration for Improved Augmented Reality Registration," Presence: Teleoperators and Virtual Environments, 1997, vol. 6(5), pp. 532-546, available from www.cs.unc.edu/˜andrei/pubs/1997_Presence_calibr.pdf, printed Sep. 20, 2007, 14 pages. |
Matsunaga et al., "The Effect of the Ratio Difference of Overlapped Areas of Stereoscopic Images on each Eye in a Teleoperalion," Stereoscopic Displays and Virtual Reality Systems VII, Proceedings of SPIE, 3957:236-243 (2000). |
Meehan, Michael et al., "Effect of Latency on Presence in Stressful Virtual Environment," Proceedings of IEEE Virtual Reality 2003, pp. 141-148, available from http://www.cs.unc.edu/˜eve/pubs.html, printed Sep. 20, 2007, 8 pages. |
Milgram et al., "Adaptation Effects in Stereo due to Online Changes in Camera Configuration," SPIE vol. 1669-13, Stereoscopic Displays and Applications III, 17 pages (1992). |
Mitchell et al., "Colposcopy for the Diagnosis of Squamous Intraepithelial lesions: A metaanalysis," Obstetrics and Gynecology; Apr. 1998; 91(4):626-631. |
Nakamoto et al., 3D Ultrasound System Using a Magneto-optic Hybrid Tracker for Augmented Reality Visualization in Laparoscopic Liver Surgery, in T. Dohi and R. Kikinis (Eds.): MICCAI 2002, LNCS 2489, pp. 148-155, 2002. |
Nordstrom et al., "Identification of Cervical Intraepithelial Neoplasia (CIN) Using UV-Excited Fluorescence and Diffuse-Reflectance Tissue Spectroscopy," Lasers in Surgery and Medicine; 2001; 29; pp. 118-127; Wiley-Liss, Inc. |
Ohbuchi et al. "An Incremental Volume Rendering Algorithm for Interactive 3D Ultrasound Imaging", UNC-CH Computer Science Technical Report TR91-003, (1991). |
Ohbuchi et al., "Incremental Volume Reconstruction and Rendering for 3D Ultrasound Imaging," Visualization in Biomedical Computing, SPIE Proceedings, pp. 312-323, (Oct. 13, 1992). |
Ohbuchi, "Incremental Acquisition and Visualization of 3D Ultrasound Images," Ph.D. Dissertation, UNC-CH Computer Science Technical Report TR95--023, (1993). |
Ohnesorge, Lauren K., "InnerOptic technology wins FDA approval", Triangle Business Journal, Sep. 19, 2012. |
Pogue, Brian W. et al., "Analysis of acetic acid-induced whitening of high-grade squamous intraepitheliallesions," Journal of Biomedical Optics; Oct. 2001; 6(4):397-403. |
Press Release: Pathfinder and InnerOptic Announce Technology Integration to Enhance Visualization and Outcomes in Liver Surgery, Published Mar. 6, 2013. |
Raij, A.B., et al., Comparing Interpersonal Interactions with a Virtual Human to Those with a Real Human; IEEE Transactions on Visualization and Computer Graphics 13(3), 443-457 (2007). |
Raz et al, Real-Time Magnetic Resonance Imaging-Guided Focal Laser Therapy in Patients with Low-Risk Prostate Cancer, European Urology 58, pp. 173-177. Mar. 12, 2010. |
Raz et al, Real-Time Magnetic Resonance Imaging—Guided Focal Laser Therapy in Patients with Low-Risk Prostate Cancer, European Urology 58, pp. 173-177. Mar. 12, 2010. |
Robinett et al., "A Computational Model for the Stereoscopic Optics of a Head-Mounted Display," SPIE vol. 1457, Stereoscopic Displays and Applications II, pp. 140-160 (1991). |
Rolland et al., Towards Quantifying Depth and Size Perception in Virtual Environments, Presence: Teleoperators and Virtual Environments, Winter 1995, vol. 4, Issue 1, pp. 1-21 and 24-49. |
Rosenthal, Michael et al., "Augmented Reality Guidance for Needle Biopsies: A Randomized, Controlled Trial in Phantoms," Proceedings of MICCAI 2001, eds. W. Niessen and M. Viergever, Lecture Notes in Computer Science, 2001, vol. 2208, pp. 240-248, available from www.cs.unc.edu/-us/AugmentedRealityAssistance.pdf, printed Sep. 20, 2007, 9 pages. |
Rosenthal, Michael et al., "Augmented Reality Guidance for Needle Biopsies: An Initial Randomized, Controlled Trial in Phantoms," Proceedings of Medical Image Analysis, Sep. 2002, vol. 6(3), pp. 313-320, available from www.cs.unc.edu/˜fuchs/publications/AugRealGuida_NeedleBiop02.pdf, printed Sep. 20, 2007, 8 pages. |
Rosenthal, Michael et al., "Augmented Reality Guidance for Needle Biopsies: A Randomized, Controlled Trial in Phantoms," Proceedings of MICCAI 2001, eds. W. Niessen and M. Viergever, Lecture Notes in Computer Science, 2001, vol. 2208, pp. 240-248, available from www.cs.unc.edu/—us/AugmentedRealityAssistance.pdf, printed Sep. 20, 2007, 9 pages. |
Screenshots from video produced by the University of North Carolina, produced circa 1992. |
State et al., "Case Study: Observing a Volume Rendered Fetus within a Pregnant Patient," Proceedings of IEEE Visualization 1994, pp. 364-368, available from www.cs.unc.edu/˜fuchs/publications/cs-ObservVolRendFetus94.pdf, printed Sep. 20, 2007, 5 pages. |
State et al., "Interactive Volume Visualization on a Heterogeneous Message-Passing Multicomputer," Proceedings of 1995 Symposium on Interactive 3D Graphics, 1995, pp. 69-74, 208, available from www.cs.unc.edu/˜andrei/pubs/1995_I3D_vol2_Mac.pdf, printed Sep. 20, 2007. |
State et al., "Simulation-Based Design and Rapid Prototyping of a Parallax-Free, Orthoscopic Video See-Through Head-Mounted Display," Proceedings of International Symposium on Mixed and Augmented Reality (ISMAR) 2005, available from www.cs.unc.edu/˜andrei/pubs/2005_ISMAR_VSTHMD_design.pdf, printed Sep. 20, 2007, 4 pages. |
State et al., "Stereo Imagery from the UNC Augmented Reality System for Breast Biopsy Guidance" Proc. Medicine Meets Virtual Reality (MMVR) 2003 (Newport Beach, CA, Jan. 22-25, 2003). |
State et al., "Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic Tracking," ACM SIGGRAPH Computer Graphics, Proceedings of SIGGRAPH 1996, 10 pages (Aug. 1996). |
State et al., "Technologies for Augmented Reality Systems: Realizing Ultrasound-Guided Needle Biopsies," Proc. SIGGRAPH 96 (New Orleans, LA, Aug. 4-9, 1996). In Computer Graphics Proceedings, Annual Conference Series, 1996, ACM SIGGRAPH, pp. 439-446. |
State, Andrei "Exact Eye Contact with Virtual Humans." Proc. IEEE International Workshop on Human Computer Interaction 2007 (Rio de Janeiro, Brazil, Oct. 20, 2007), pp. 138-145. |
State, et al.: Contextually Enhanced 3D Visualization for Multi-Born Tumor Ablation Guidance, Departments of Computer Science and Radiology, and School of Medicine, University of North Carolina at Chapel Hill; InnerOptic Technology, Inc. 2008, Chapel Hill, NC, pp. 70-77. |
StereoMirror Technology Webpage, http://www.planar.com/products/flatpanel_monitors/stereoscopic/ (Printed Dec. 29, 2011). |
Symons et al., "What are You Looking at? Acuity for Triadic Eye Gaze," J. Gen. Psychology 131(4), pp. 451-469 (2004). |
Takacs et al., "The Virtual Human Interface: A Photorealistic Digital Human," IEEE Computer Graphics and Applications 23(5), pp. 38-45 (2003). |
Takagi et al., "Development of a Stereo Video See-through HMD for AR Systems," IEEE, pp. 68-77 (2000). |
Takayama et al., "Virtual Human with Regard to Physical Contact and Eye Contact," Entertaining Computing 2005, LNCS, vol. 3711, pp. 268-278 (2005). |
U.S. Appl. No. 11/828,826 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Jul. 26, 2007, Keller et al. |
U.S. Appl. No. 15/041,868 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Feb. 11, 2016, Fuchs et al. |
U.S. Appl. No. 15/068,323 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Mar. 11, 2016, Razzaque et al. |
U.S. Appl. No. 15/182,346 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Jun. 14, 2016, Razzaque et al. |
U.S. Appl. No. 15/199,630 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Jun. 30, 2016, Razzaque et al. |
U.S. Appl. No. 15/799,639 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Oct. 31, 2017, Green et al. |
U.S. Appl. No. 15/882,709 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Jan. 29, 2018, State et al. |
U.S. Appl. No. 15/995,059 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Apr. 17, 2018, Kohli et al. |
U.S. Appl. No. 16/052,289 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Aug. 1, 2018, Kohli et al. |
U.S. Appl. No. 16/177,894 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Nov. 1, 2018, Keller et al. |
U.S. Appl. No. 16/178,002 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents, filed Nov. 1, 2018, Heaney et al. |
U.S. Pat. No. 8,340,379 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 8,350,902 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 8,482,606 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 8,554,307 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 8,585,598 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 8,641,621 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 8,670,816 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 8,690,776 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 8,831,310 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 9,107,698 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 9,282,947 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents,. |
U.S. Pat. No. 9,364,294 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 9,398,936 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 9,659,345 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 9,675,319 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
U.S. Pat. No. 9,901,406 including its ongoing prosecution history, including without limitation Office Actions, Amendments, Remarks, and any other potentially relevant documents. |
Ultraguide 1000 System, Ultraguide, www.ultraguideinc.com, 1998. |
Van Staveren et al., "Light Scattering in Intralipid-10% in the wavelength range of 400-1100 nm," Applied Optics; Nov. 1991; 30(31):4507-4514. |
Viola et al., "Alignment by Maximization of Mutual Information," International Journal of Computer Vision, vol. 24, No. 2, pp. 137-154 (1997). |
Viola, Paul A., Alignment by Maximization of Mutual Information, Ph.D. Dissertation, MIT-Artificial Intelligence Laboratory Technical Report No. 1548 (Jun. 1995), 156 pages. |
Ware et al., "Dynamic Adjustment of Stereo Display Parameters," IEEE Transactions on Systems, Many and Cybernetics, 28(1):1-19 (1998). |
Watson et al., "Using Texture Maps to Correct for Optical Distortion in Head-Mounted Displays," Proceedings of the Virtual Reality Annual Symposium '95, IEEE, pp. 1-7 (1995). |
Welch, Hybrid Self-Tracker: An Inertial/Optical Hybrid Three-Dimensional Tracking System, University of North Carolina Chapel Hill Department of Computer Science, TR 95-048 (1995). |
Yinghui et al., Real-Time Deformation Using Modal Analysis on Graphics Hardware, GRAPHITE 2006, Kuala Lumpur, Malaysia, Nov. 29-Dec. 2, 2006. |
Zitnick et al., "Multi-Base Stereo Using Surface Extraction," Visual Interface Inc., (Nov. 24, 1996). |
Cited By (259)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10733700B2 (en) | 2006-08-02 | 2020-08-04 | Inneroptic Technology, Inc. | System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities |
US11481868B2 (en) | 2006-08-02 | 2022-10-25 | Inneroptic Technology, Inc. | System and method of providing real-time dynamic imagery of a medical procedure she using multiple modalities |
US11464575B2 (en) | 2009-02-17 | 2022-10-11 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image guided surgery |
US11464578B2 (en) | 2009-02-17 | 2022-10-11 | Inneroptic Technology, Inc. | Systems, methods, apparatuses, and computer-readable media for image management in image-guided medical procedures |
US11051681B2 (en) | 2010-06-24 | 2021-07-06 | Auris Health, Inc. | Methods and devices for controlling a shapeable medical device |
US11857156B2 (en) | 2010-06-24 | 2024-01-02 | Auris Health, Inc. | Methods and devices for controlling a shapeable medical device |
US11871901B2 (en) | 2012-05-20 | 2024-01-16 | Cilag Gmbh International | Method for situational awareness for surgical network or surgical network connected device capable of adjusting function based on a sensed situation or usage |
US11241203B2 (en) | 2013-03-13 | 2022-02-08 | Auris Health, Inc. | Reducing measurement sensor error |
US11969157B2 (en) | 2013-03-15 | 2024-04-30 | Auris Health, Inc. | Systems and methods for tracking robotically controlled medical instruments |
US11426095B2 (en) | 2013-03-15 | 2022-08-30 | Auris Health, Inc. | Flexible instrument localization from both remote and elongation sensors |
US11504187B2 (en) | 2013-03-15 | 2022-11-22 | Auris Health, Inc. | Systems and methods for localizing, tracking and/or controlling medical instruments |
US11129602B2 (en) | 2013-03-15 | 2021-09-28 | Auris Health, Inc. | Systems and methods for tracking robotically controlled medical instruments |
US11020016B2 (en) | 2013-05-30 | 2021-06-01 | Auris Health, Inc. | System and method for displaying anatomy and devices on a movable display |
US11684429B2 (en) | 2014-10-02 | 2023-06-27 | Inneroptic Technology, Inc. | Affected region display associated with a medical device |
US11504192B2 (en) | 2014-10-30 | 2022-11-22 | Cilag Gmbh International | Method of hub communication with surgical instrument systems |
US10820946B2 (en) | 2014-12-12 | 2020-11-03 | Inneroptic Technology, Inc. | Surgical guidance intersection display |
US11931117B2 (en) | 2014-12-12 | 2024-03-19 | Inneroptic Technology, Inc. | Surgical guidance intersection display |
US11534245B2 (en) | 2014-12-12 | 2022-12-27 | Inneroptic Technology, Inc. | Surgical guidance intersection display |
US11103200B2 (en) | 2015-07-22 | 2021-08-31 | Inneroptic Technology, Inc. | Medical device approaches |
US10796432B2 (en) | 2015-09-18 | 2020-10-06 | Auris Health, Inc. | Navigation of tubular networks |
US11403759B2 (en) | 2015-09-18 | 2022-08-02 | Auris Health, Inc. | Navigation of tubular networks |
US10813711B2 (en) | 2015-11-30 | 2020-10-27 | Auris Health, Inc. | Robot-assisted driving systems and methods |
US10806535B2 (en) | 2015-11-30 | 2020-10-20 | Auris Health, Inc. | Robot-assisted driving systems and methods |
US11464591B2 (en) | 2015-11-30 | 2022-10-11 | Auris Health, Inc. | Robot-assisted driving systems and methods |
US11179136B2 (en) | 2016-02-17 | 2021-11-23 | Inneroptic Technology, Inc. | Loupe display |
US10772686B2 (en) | 2016-10-27 | 2020-09-15 | Inneroptic Technology, Inc. | Medical device navigation using a virtual 3D space |
US11369439B2 (en) | 2016-10-27 | 2022-06-28 | Inneroptic Technology, Inc. | Medical device navigation using a virtual 3D space |
US11771309B2 (en) | 2016-12-28 | 2023-10-03 | Auris Health, Inc. | Detecting endolumenal buckling of flexible instruments |
US11707330B2 (en) * | 2017-01-03 | 2023-07-25 | Mako Surgical Corp. | Systems and methods for surgical navigation |
US11490782B2 (en) | 2017-03-31 | 2022-11-08 | Auris Health, Inc. | Robotic systems for navigation of luminal networks that compensate for physiological noise |
US11278357B2 (en) * | 2017-06-23 | 2022-03-22 | Auris Health, Inc. | Robotic systems for determining an angular degree of freedom of a medical device in luminal networks |
US20190216548A1 (en) * | 2017-06-23 | 2019-07-18 | Auris Health, Inc. | Robotic systems for determining a roll of a medical device in luminal networks |
US11759266B2 (en) | 2017-06-23 | 2023-09-19 | Auris Health, Inc. | Robotic systems for determining a roll of a medical device in luminal networks |
US11259879B2 (en) | 2017-08-01 | 2022-03-01 | Inneroptic Technology, Inc. | Selective transparency to assist medical device navigation |
US11850008B2 (en) | 2017-10-13 | 2023-12-26 | Auris Health, Inc. | Image-based branch detection and mapping for navigation |
US11058493B2 (en) | 2017-10-13 | 2021-07-13 | Auris Health, Inc. | Robotic system configured for navigation path tracing |
US11969217B2 (en) | 2017-10-13 | 2024-04-30 | Auris Health, Inc. | Robotic system configured for navigation path tracing |
US11026687B2 (en) | 2017-10-30 | 2021-06-08 | Cilag Gmbh International | Clip applier comprising clip advancing systems |
US11406390B2 (en) | 2017-10-30 | 2022-08-09 | Cilag Gmbh International | Clip applier comprising interchangeable clip reloads |
US11051836B2 (en) | 2017-10-30 | 2021-07-06 | Cilag Gmbh International | Surgical clip applier comprising an empty clip cartridge lockout |
US11229436B2 (en) | 2017-10-30 | 2022-01-25 | Cilag Gmbh International | Surgical system comprising a surgical tool and a surgical hub |
US11510741B2 (en) | 2017-10-30 | 2022-11-29 | Cilag Gmbh International | Method for producing a surgical instrument comprising a smart electrical system |
US11207090B2 (en) | 2017-10-30 | 2021-12-28 | Cilag Gmbh International | Surgical instruments comprising a biased shifting mechanism |
US11026713B2 (en) | 2017-10-30 | 2021-06-08 | Cilag Gmbh International | Surgical clip applier configured to store clips in a stored state |
US11026712B2 (en) | 2017-10-30 | 2021-06-08 | Cilag Gmbh International | Surgical instruments comprising a shifting mechanism |
US11564756B2 (en) | 2017-10-30 | 2023-01-31 | Cilag Gmbh International | Method of hub communication with surgical instrument systems |
US11071560B2 (en) | 2017-10-30 | 2021-07-27 | Cilag Gmbh International | Surgical clip applier comprising adaptive control in response to a strain gauge circuit |
US11564703B2 (en) | 2017-10-30 | 2023-01-31 | Cilag Gmbh International | Surgical suturing instrument comprising a capture width which is larger than trocar diameter |
US10980560B2 (en) | 2017-10-30 | 2021-04-20 | Ethicon Llc | Surgical instrument systems comprising feedback mechanisms |
US11602366B2 (en) | 2017-10-30 | 2023-03-14 | Cilag Gmbh International | Surgical suturing instrument configured to manipulate tissue using mechanical and electrical power |
US11648022B2 (en) | 2017-10-30 | 2023-05-16 | Cilag Gmbh International | Surgical instrument systems comprising battery arrangements |
US10959744B2 (en) | 2017-10-30 | 2021-03-30 | Ethicon Llc | Surgical dissectors and manufacturing techniques |
US11103268B2 (en) | 2017-10-30 | 2021-08-31 | Cilag Gmbh International | Surgical clip applier comprising adaptive firing control |
US11696778B2 (en) | 2017-10-30 | 2023-07-11 | Cilag Gmbh International | Surgical dissectors configured to apply mechanical and electrical energy |
US11109878B2 (en) | 2017-10-30 | 2021-09-07 | Cilag Gmbh International | Surgical clip applier comprising an automatic clip feeding system |
US11413042B2 (en) | 2017-10-30 | 2022-08-16 | Cilag Gmbh International | Clip applier comprising a reciprocating clip advancing member |
US11045197B2 (en) | 2017-10-30 | 2021-06-29 | Cilag Gmbh International | Clip applier comprising a movable clip magazine |
US11123070B2 (en) | 2017-10-30 | 2021-09-21 | Cilag Gmbh International | Clip applier comprising a rotatable clip magazine |
US11129636B2 (en) | 2017-10-30 | 2021-09-28 | Cilag Gmbh International | Surgical instruments comprising an articulation drive that provides for high articulation angles |
US10932806B2 (en) | 2017-10-30 | 2021-03-02 | Ethicon Llc | Reactive algorithm for surgical system |
US11759224B2 (en) | 2017-10-30 | 2023-09-19 | Cilag Gmbh International | Surgical instrument systems comprising handle arrangements |
US11793537B2 (en) | 2017-10-30 | 2023-10-24 | Cilag Gmbh International | Surgical instrument comprising an adaptive electrical system |
US11141160B2 (en) | 2017-10-30 | 2021-10-12 | Cilag Gmbh International | Clip applier comprising a motor controller |
US11801098B2 (en) | 2017-10-30 | 2023-10-31 | Cilag Gmbh International | Method of hub communication with surgical instrument systems |
US11317919B2 (en) | 2017-10-30 | 2022-05-03 | Cilag Gmbh International | Clip applier comprising a clip crimping system |
US11925373B2 (en) | 2017-10-30 | 2024-03-12 | Cilag Gmbh International | Surgical suturing instrument comprising a non-circular needle |
US11311342B2 (en) | 2017-10-30 | 2022-04-26 | Cilag Gmbh International | Method for communicating with surgical instrument systems |
US11291510B2 (en) | 2017-10-30 | 2022-04-05 | Cilag Gmbh International | Method of hub communication with surgical instrument systems |
US11291465B2 (en) | 2017-10-30 | 2022-04-05 | Cilag Gmbh International | Surgical instruments comprising a lockable end effector socket |
US11819231B2 (en) | 2017-10-30 | 2023-11-21 | Cilag Gmbh International | Adaptive control programs for a surgical system comprising more than one type of cartridge |
US10772651B2 (en) | 2017-10-30 | 2020-09-15 | Ethicon Llc | Surgical instruments comprising a system for articulation and rotation compensation |
US11911045B2 (en) | 2017-10-30 | 2024-02-27 | Cllag GmbH International | Method for operating a powered articulating multi-clip applier |
US11510736B2 (en) | 2017-12-14 | 2022-11-29 | Auris Health, Inc. | System and method for estimating instrument location |
US11160615B2 (en) | 2017-12-18 | 2021-11-02 | Auris Health, Inc. | Methods and systems for instrument tracking and navigation within luminal networks |
US11382697B2 (en) | 2017-12-28 | 2022-07-12 | Cilag Gmbh International | Surgical instruments comprising button circuits |
US11771487B2 (en) | 2017-12-28 | 2023-10-03 | Cilag Gmbh International | Mechanisms for controlling different electromechanical systems of an electrosurgical instrument |
US11969142B2 (en) | 2017-12-28 | 2024-04-30 | Cilag Gmbh International | Method of compressing tissue within a stapling device and simultaneously displaying the location of the tissue within the jaws |
US11202570B2 (en) | 2017-12-28 | 2021-12-21 | Cilag Gmbh International | Communication hub and storage device for storing parameters and status of a surgical device to be shared with cloud based analytics systems |
US11213359B2 (en) | 2017-12-28 | 2022-01-04 | Cilag Gmbh International | Controllers for robot-assisted surgical platforms |
US10595887B2 (en) | 2017-12-28 | 2020-03-24 | Ethicon Llc | Systems for adjusting end effector parameters based on perioperative information |
US11969216B2 (en) | 2017-12-28 | 2024-04-30 | Cilag Gmbh International | Surgical network recommendations from real time analysis of procedure variables against a baseline highlighting differences from the optimal solution |
US10695081B2 (en) | 2017-12-28 | 2020-06-30 | Ethicon Llc | Controlling a surgical instrument according to sensed closure parameters |
US11234756B2 (en) | 2017-12-28 | 2022-02-01 | Cilag Gmbh International | Powered surgical tool with predefined adjustable control algorithm for controlling end effector parameter |
US11179204B2 (en) | 2017-12-28 | 2021-11-23 | Cilag Gmbh International | Wireless pairing of a surgical device with another device within a sterile surgical field based on the usage and situational awareness of devices |
US11253315B2 (en) | 2017-12-28 | 2022-02-22 | Cilag Gmbh International | Increasing radio frequency to create pad-less monopolar loop |
US11257589B2 (en) | 2017-12-28 | 2022-02-22 | Cilag Gmbh International | Real-time analysis of comprehensive cost of all instrumentation used in surgery utilizing data fluidity to track instruments through stocking and in-house processes |
US11937769B2 (en) | 2017-12-28 | 2024-03-26 | Cilag Gmbh International | Method of hub communication, processing, storage and display |
US11931110B2 (en) | 2017-12-28 | 2024-03-19 | Cilag Gmbh International | Surgical instrument comprising a control system that uses input from a strain gage circuit |
US11179175B2 (en) | 2017-12-28 | 2021-11-23 | Cilag Gmbh International | Controlling an ultrasonic surgical instrument according to tissue location |
US10755813B2 (en) | 2017-12-28 | 2020-08-25 | Ethicon Llc | Communication of smoke evacuation system parameters to hub or cloud in smoke evacuation module for interactive surgical platform |
US11266468B2 (en) | 2017-12-28 | 2022-03-08 | Cilag Gmbh International | Cooperative utilization of data derived from secondary sources by intelligent surgical hubs |
US11918302B2 (en) | 2017-12-28 | 2024-03-05 | Cilag Gmbh International | Sterile field interactive control displays |
US11273001B2 (en) | 2017-12-28 | 2022-03-15 | Cilag Gmbh International | Surgical hub and modular device response adjustment based on situational awareness |
US11179208B2 (en) | 2017-12-28 | 2021-11-23 | Cilag Gmbh International | Cloud-based medical analytics for security and authentication trends and reactive measures |
US11278281B2 (en) | 2017-12-28 | 2022-03-22 | Cilag Gmbh International | Interactive surgical system |
US10758310B2 (en) | 2017-12-28 | 2020-09-01 | Ethicon Llc | Wireless pairing of a surgical device with another device within a sterile surgical field based on the usage and situational awareness of devices |
US11284936B2 (en) | 2017-12-28 | 2022-03-29 | Cilag Gmbh International | Surgical instrument having a flexible electrode |
US11903587B2 (en) | 2017-12-28 | 2024-02-20 | Cilag Gmbh International | Adjustment to the surgical stapling control based on situational awareness |
US11166772B2 (en) | 2017-12-28 | 2021-11-09 | Cilag Gmbh International | Surgical hub coordination of control and communication of operating room devices |
US11903601B2 (en) | 2017-12-28 | 2024-02-20 | Cilag Gmbh International | Surgical instrument comprising a plurality of drive systems |
US11291495B2 (en) | 2017-12-28 | 2022-04-05 | Cilag Gmbh International | Interruption of energy due to inadvertent capacitive coupling |
US11896443B2 (en) | 2017-12-28 | 2024-02-13 | Cilag Gmbh International | Control of a surgical system through a surgical barrier |
US11896322B2 (en) | 2017-12-28 | 2024-02-13 | Cilag Gmbh International | Sensing the patient position and contact utilizing the mono-polar return pad electrode to provide situational awareness to the hub |
US11890065B2 (en) | 2017-12-28 | 2024-02-06 | Cilag Gmbh International | Surgical system to limit displacement |
US11864845B2 (en) | 2017-12-28 | 2024-01-09 | Cilag Gmbh International | Sterile field interactive control displays |
US11864728B2 (en) | 2017-12-28 | 2024-01-09 | Cilag Gmbh International | Characterization of tissue irregularities through the use of mono-chromatic light refractivity |
US11304763B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Image capturing of the areas outside the abdomen to improve placement and control of a surgical device in use |
US11304720B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Activation of energy devices |
US11308075B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Surgical network, instrument, and cloud responses based on validation of received dataset and authentication of its source and integrity |
US11304745B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Surgical evacuation sensing and display |
US11304699B2 (en) | 2017-12-28 | 2022-04-19 | Cilag Gmbh International | Method for adaptive control schemes for surgical network control and interaction |
US11160605B2 (en) | 2017-12-28 | 2021-11-02 | Cilag Gmbh International | Surgical evacuation sensing and motor control |
US11311306B2 (en) | 2017-12-28 | 2022-04-26 | Cilag Gmbh International | Surgical systems for detecting end effector tissue distribution irregularities |
US10849697B2 (en) | 2017-12-28 | 2020-12-01 | Ethicon Llc | Cloud interface for coupled surgical devices |
US11147607B2 (en) | 2017-12-28 | 2021-10-19 | Cilag Gmbh International | Bipolar combination device that automatically adjusts pressure based on energy modality |
US11857152B2 (en) | 2017-12-28 | 2024-01-02 | Cilag Gmbh International | Surgical hub spatial awareness to determine devices in operating theater |
US10892995B2 (en) | 2017-12-28 | 2021-01-12 | Ethicon Llc | Surgical network determination of prioritization of communication, interaction, or processing based on system or device needs |
US11324557B2 (en) | 2017-12-28 | 2022-05-10 | Cilag Gmbh International | Surgical instrument with a sensing array |
US11844579B2 (en) | 2017-12-28 | 2023-12-19 | Cilag Gmbh International | Adjustments based on airborne particle properties |
US11832840B2 (en) | 2017-12-28 | 2023-12-05 | Cilag Gmbh International | Surgical instrument having a flexible circuit |
US11832899B2 (en) | 2017-12-28 | 2023-12-05 | Cilag Gmbh International | Surgical systems with autonomously adjustable control programs |
US10892899B2 (en) | 2017-12-28 | 2021-01-12 | Ethicon Llc | Self describing data packets generated at an issuing instrument |
US11818052B2 (en) | 2017-12-28 | 2023-11-14 | Cilag Gmbh International | Surgical network determination of prioritization of communication, interaction, or processing based on system or device needs |
US10898622B2 (en) | 2017-12-28 | 2021-01-26 | Ethicon Llc | Surgical evacuation system with a communication circuit for communication between a filter and a smoke evacuation device |
US11364075B2 (en) | 2017-12-28 | 2022-06-21 | Cilag Gmbh International | Radio frequency energy device for delivering combined electrical signals |
US11786251B2 (en) | 2017-12-28 | 2023-10-17 | Cilag Gmbh International | Method for adaptive control schemes for surgical network control and interaction |
US11786245B2 (en) | 2017-12-28 | 2023-10-17 | Cilag Gmbh International | Surgical systems with prioritized data transmission capabilities |
US11376002B2 (en) | 2017-12-28 | 2022-07-05 | Cilag Gmbh International | Surgical instrument cartridge sensor assemblies |
US11779337B2 (en) | 2017-12-28 | 2023-10-10 | Cilag Gmbh International | Method of using reinforced flexible circuits with multiple sensors to optimize performance of radio frequency devices |
US11389164B2 (en) | 2017-12-28 | 2022-07-19 | Cilag Gmbh International | Method of using reinforced flexible circuits with multiple sensors to optimize performance of radio frequency devices |
US11775682B2 (en) | 2017-12-28 | 2023-10-03 | Cilag Gmbh International | Data stripping method to interrogate patient records and create anonymized record |
US11132462B2 (en) | 2017-12-28 | 2021-09-28 | Cilag Gmbh International | Data stripping method to interrogate patient records and create anonymized record |
US11751958B2 (en) | 2017-12-28 | 2023-09-12 | Cilag Gmbh International | Surgical hub coordination of control and communication of operating room devices |
US11114195B2 (en) | 2017-12-28 | 2021-09-07 | Cilag Gmbh International | Surgical instrument with a tissue marking assembly |
US11410259B2 (en) | 2017-12-28 | 2022-08-09 | Cilag Gmbh International | Adaptive control program updates for surgical devices |
US11744604B2 (en) | 2017-12-28 | 2023-09-05 | Cilag Gmbh International | Surgical instrument with a hardware-only control circuit |
US11109866B2 (en) | 2017-12-28 | 2021-09-07 | Cilag Gmbh International | Method for circular stapler control algorithm adjustment based on situational awareness |
US11419667B2 (en) | 2017-12-28 | 2022-08-23 | Cilag Gmbh International | Ultrasonic energy device which varies pressure applied by clamp arm to provide threshold control pressure at a cut progression location |
US11419630B2 (en) | 2017-12-28 | 2022-08-23 | Cilag Gmbh International | Surgical system distributed processing |
US11423007B2 (en) | 2017-12-28 | 2022-08-23 | Cilag Gmbh International | Adjustment of device control programs based on stratified contextual data in addition to the data |
US11424027B2 (en) | 2017-12-28 | 2022-08-23 | Cilag Gmbh International | Method for operating surgical instrument systems |
US11100631B2 (en) | 2017-12-28 | 2021-08-24 | Cilag Gmbh International | Use of laser light and red-green-blue coloration to determine properties of back scattered light |
US11432885B2 (en) | 2017-12-28 | 2022-09-06 | Cilag Gmbh International | Sensing arrangements for robot-assisted surgical platforms |
US11446052B2 (en) | 2017-12-28 | 2022-09-20 | Cilag Gmbh International | Variation of radio frequency and ultrasonic power level in cooperation with varying clamp arm pressure to achieve predefined heat flux or power applied to tissue |
US11737668B2 (en) | 2017-12-28 | 2023-08-29 | Cilag Gmbh International | Communication hub and storage device for storing parameters and status of a surgical device to be shared with cloud based analytics systems |
US11712303B2 (en) | 2017-12-28 | 2023-08-01 | Cilag Gmbh International | Surgical instrument comprising a control circuit |
US10932872B2 (en) | 2017-12-28 | 2021-03-02 | Ethicon Llc | Cloud-based medical analytics for linking of local usage trends with the resource acquisition behaviors of larger data set |
US11096693B2 (en) | 2017-12-28 | 2021-08-24 | Cilag Gmbh International | Adjustment of staple height of at least one row of staples based on the sensed tissue thickness or force in closing |
US11701185B2 (en) | 2017-12-28 | 2023-07-18 | Cilag Gmbh International | Wireless pairing of a surgical device with another device within a sterile surgical field based on the usage and situational awareness of devices |
US11696760B2 (en) | 2017-12-28 | 2023-07-11 | Cilag Gmbh International | Safety systems for smart powered surgical stapling |
US11464535B2 (en) | 2017-12-28 | 2022-10-11 | Cilag Gmbh International | Detection of end effector emersion in liquid |
US11464559B2 (en) | 2017-12-28 | 2022-10-11 | Cilag Gmbh International | Estimating state of ultrasonic end effector and control system therefor |
US10944728B2 (en) | 2017-12-28 | 2021-03-09 | Ethicon Llc | Interactive surgical systems with encrypted communication capabilities |
US11076921B2 (en) | 2017-12-28 | 2021-08-03 | Cilag Gmbh International | Adaptive control program updates for surgical hubs |
US11069012B2 (en) | 2017-12-28 | 2021-07-20 | Cilag Gmbh International | Interactive surgical systems with condition handling of devices and data capabilities |
US11058498B2 (en) | 2017-12-28 | 2021-07-13 | Cilag Gmbh International | Cooperative surgical actions for robot-assisted surgical platforms |
US10943454B2 (en) | 2017-12-28 | 2021-03-09 | Ethicon Llc | Detection and escalation of security responses of surgical instruments to increasing severity threats |
US11051876B2 (en) | 2017-12-28 | 2021-07-06 | Cilag Gmbh International | Surgical evacuation flow paths |
US11056244B2 (en) | 2017-12-28 | 2021-07-06 | Cilag Gmbh International | Automated data scaling, alignment, and organizing based on predefined parameters within surgical networks |
US11045591B2 (en) | 2017-12-28 | 2021-06-29 | Cilag Gmbh International | Dual in-series large and small droplet filters |
US11678881B2 (en) | 2017-12-28 | 2023-06-20 | Cilag Gmbh International | Spatial awareness of surgical hubs in operating rooms |
US11529187B2 (en) | 2017-12-28 | 2022-12-20 | Cilag Gmbh International | Surgical evacuation sensor arrangements |
US11672605B2 (en) | 2017-12-28 | 2023-06-13 | Cilag Gmbh International | Sterile field interactive control displays |
US11666331B2 (en) | 2017-12-28 | 2023-06-06 | Cilag Gmbh International | Systems for detecting proximity of surgical end effector to cancerous tissue |
US11540855B2 (en) | 2017-12-28 | 2023-01-03 | Cilag Gmbh International | Controlling activation of an ultrasonic surgical instrument according to the presence of tissue |
US11559308B2 (en) | 2017-12-28 | 2023-01-24 | Cilag Gmbh International | Method for smart energy device infrastructure |
US11559307B2 (en) | 2017-12-28 | 2023-01-24 | Cilag Gmbh International | Method of robotic hub communication, detection, and control |
US11013563B2 (en) | 2017-12-28 | 2021-05-25 | Ethicon Llc | Drive arrangements for robot-assisted surgical platforms |
US10987178B2 (en) | 2017-12-28 | 2021-04-27 | Ethicon Llc | Surgical hub control arrangements |
US11571234B2 (en) | 2017-12-28 | 2023-02-07 | Cilag Gmbh International | Temperature control of ultrasonic end effector and control system therefor |
US11576677B2 (en) | 2017-12-28 | 2023-02-14 | Cilag Gmbh International | Method of hub communication, processing, display, and cloud analytics |
US11659023B2 (en) | 2017-12-28 | 2023-05-23 | Cilag Gmbh International | Method of hub communication |
US10966791B2 (en) | 2017-12-28 | 2021-04-06 | Ethicon Llc | Cloud-based medical analytics for medical facility segmented individualization of instrument function |
US11589932B2 (en) | 2017-12-28 | 2023-02-28 | Cilag Gmbh International | Usage and technique analysis of surgeon / staff performance against a baseline to optimize device utilization and performance for both current and future procedures |
US11633237B2 (en) | 2017-12-28 | 2023-04-25 | Cilag Gmbh International | Usage and technique analysis of surgeon / staff performance against a baseline to optimize device utilization and performance for both current and future procedures |
US11589888B2 (en) | 2017-12-28 | 2023-02-28 | Cilag Gmbh International | Method for controlling smart energy devices |
US11612444B2 (en) | 2017-12-28 | 2023-03-28 | Cilag Gmbh International | Adjustment of a surgical device function based on situational awareness |
US11601371B2 (en) | 2017-12-28 | 2023-03-07 | Cilag Gmbh International | Surgical network determination of prioritization of communication, interaction, or processing based on system or device needs |
US11596291B2 (en) | 2017-12-28 | 2023-03-07 | Cilag Gmbh International | Method of compressing tissue within a stapling device and simultaneously displaying of the location of the tissue within the jaws |
US11612408B2 (en) | 2017-12-28 | 2023-03-28 | Cilag Gmbh International | Determining tissue composition via an ultrasonic system |
US11602393B2 (en) | 2017-12-28 | 2023-03-14 | Cilag Gmbh International | Surgical evacuation sensing and generator control |
US11707293B2 (en) | 2018-03-08 | 2023-07-25 | Cilag Gmbh International | Ultrasonic sealing algorithm with temperature control |
US11844545B2 (en) | 2018-03-08 | 2023-12-19 | Cilag Gmbh International | Calcified vessel identification |
US11317937B2 (en) | 2018-03-08 | 2022-05-03 | Cilag Gmbh International | Determining the state of an ultrasonic end effector |
US11589915B2 (en) | 2018-03-08 | 2023-02-28 | Cilag Gmbh International | In-the-jaw classifier based on a model |
US11617597B2 (en) | 2018-03-08 | 2023-04-04 | Cilag Gmbh International | Application of smart ultrasonic blade technology |
US11337746B2 (en) | 2018-03-08 | 2022-05-24 | Cilag Gmbh International | Smart blade and power pulsing |
US11986233B2 (en) | 2018-03-08 | 2024-05-21 | Cilag Gmbh International | Adjustment of complex impedance to compensate for lost power in an articulating ultrasonic device |
US11344326B2 (en) | 2018-03-08 | 2022-05-31 | Cilag Gmbh International | Smart blade technology to control blade instability |
US11298148B2 (en) | 2018-03-08 | 2022-04-12 | Cilag Gmbh International | Live time tissue classification using electrical parameters |
US11389188B2 (en) | 2018-03-08 | 2022-07-19 | Cilag Gmbh International | Start temperature of blade |
US11534196B2 (en) | 2018-03-08 | 2022-12-27 | Cilag Gmbh International | Using spectroscopy to determine device use state in combo instrument |
US11839396B2 (en) | 2018-03-08 | 2023-12-12 | Cilag Gmbh International | Fine dissection mode for tissue classification |
US11678901B2 (en) | 2018-03-08 | 2023-06-20 | Cilag Gmbh International | Vessel sensing for adaptive advanced hemostasis |
US11678927B2 (en) | 2018-03-08 | 2023-06-20 | Cilag Gmbh International | Detection of large vessels during parenchymal dissection using a smart blade |
US11259830B2 (en) | 2018-03-08 | 2022-03-01 | Cilag Gmbh International | Methods for controlling temperature in ultrasonic device |
US11399858B2 (en) | 2018-03-08 | 2022-08-02 | Cilag Gmbh International | Application of smart blade technology |
US11457944B2 (en) | 2018-03-08 | 2022-10-04 | Cilag Gmbh International | Adaptive advanced tissue treatment pad saver mode |
US11701162B2 (en) | 2018-03-08 | 2023-07-18 | Cilag Gmbh International | Smart blade application for reusable and disposable devices |
US11701139B2 (en) | 2018-03-08 | 2023-07-18 | Cilag Gmbh International | Methods for controlling temperature in ultrasonic device |
US11464532B2 (en) | 2018-03-08 | 2022-10-11 | Cilag Gmbh International | Methods for estimating and controlling state of ultrasonic end effector |
US11219453B2 (en) | 2018-03-28 | 2022-01-11 | Cilag Gmbh International | Surgical stapling devices with cartridge compatible closure and firing lockout arrangements |
US10898277B2 (en) | 2018-03-28 | 2021-01-26 | Auris Health, Inc. | Systems and methods for registration of location sensors |
US11090047B2 (en) | 2018-03-28 | 2021-08-17 | Cilag Gmbh International | Surgical instrument comprising an adaptive control system |
US11712173B2 (en) | 2018-03-28 | 2023-08-01 | Auris Health, Inc. | Systems and methods for displaying estimated location of instrument |
US11950898B2 (en) | 2018-03-28 | 2024-04-09 | Auris Health, Inc. | Systems and methods for displaying estimated location of instrument |
US11406382B2 (en) | 2018-03-28 | 2022-08-09 | Cilag Gmbh International | Staple cartridge comprising a lockout key configured to lift a firing member |
US11471156B2 (en) | 2018-03-28 | 2022-10-18 | Cilag Gmbh International | Surgical stapling devices with improved rotary driven closure systems |
US11937817B2 (en) | 2018-03-28 | 2024-03-26 | Cilag Gmbh International | Surgical instruments with asymmetric jaw arrangements and separate closure and firing systems |
US11986185B2 (en) | 2018-03-28 | 2024-05-21 | Cilag Gmbh International | Methods for controlling a surgical stapler |
US11197668B2 (en) | 2018-03-28 | 2021-12-14 | Cilag Gmbh International | Surgical stapling assembly comprising a lockout and an exterior access orifice to permit artificial unlocking of the lockout |
US11931027B2 (en) | 2018-03-28 | 2024-03-19 | Cilag Gmbh Interntional | Surgical instrument comprising an adaptive control system |
US11096688B2 (en) | 2018-03-28 | 2021-08-24 | Cilag Gmbh International | Rotary driven firing members with different anvil and channel engagement features |
US11213294B2 (en) | 2018-03-28 | 2022-01-04 | Cilag Gmbh International | Surgical instrument comprising co-operating lockout features |
US11259806B2 (en) | 2018-03-28 | 2022-03-01 | Cilag Gmbh International | Surgical stapling devices with features for blocking advancement of a camming assembly of an incompatible cartridge installed therein |
US11129611B2 (en) | 2018-03-28 | 2021-09-28 | Cilag Gmbh International | Surgical staplers with arrangements for maintaining a firing member thereof in a locked configuration unless a compatible cartridge has been installed therein |
US11278280B2 (en) | 2018-03-28 | 2022-03-22 | Cilag Gmbh International | Surgical instrument comprising a jaw closure lockout |
US11166716B2 (en) | 2018-03-28 | 2021-11-09 | Cilag Gmbh International | Stapling instrument comprising a deactivatable lockout |
US10973520B2 (en) | 2018-03-28 | 2021-04-13 | Ethicon Llc | Surgical staple cartridge with firing member driven camming assembly that has an onboard tissue cutting feature |
US10827913B2 (en) | 2018-03-28 | 2020-11-10 | Auris Health, Inc. | Systems and methods for displaying estimated location of instrument |
US11207067B2 (en) | 2018-03-28 | 2021-12-28 | Cilag Gmbh International | Surgical stapling device with separate rotary driven closure and firing systems and firing member that engages both jaws while firing |
US11576730B2 (en) | 2018-03-28 | 2023-02-14 | Auris Health, Inc. | Systems and methods for registration of location sensors |
US11589865B2 (en) | 2018-03-28 | 2023-02-28 | Cilag Gmbh International | Methods for controlling a powered surgical stapler that has separate rotary closure and firing systems |
US11793580B2 (en) | 2018-05-30 | 2023-10-24 | Auris Health, Inc. | Systems and methods for location sensor-based branch prediction |
US10905499B2 (en) | 2018-05-30 | 2021-02-02 | Auris Health, Inc. | Systems and methods for location sensor-based branch prediction |
US10898286B2 (en) | 2018-05-31 | 2021-01-26 | Auris Health, Inc. | Path-based navigation of tubular networks |
US11864850B2 (en) | 2018-05-31 | 2024-01-09 | Auris Health, Inc. | Path-based navigation of tubular networks |
US10898275B2 (en) | 2018-05-31 | 2021-01-26 | Auris Health, Inc. | Image-based airway analysis and mapping |
US11759090B2 (en) | 2018-05-31 | 2023-09-19 | Auris Health, Inc. | Image-based airway analysis and mapping |
US11503986B2 (en) | 2018-05-31 | 2022-11-22 | Auris Health, Inc. | Robotic systems and methods for navigation of luminal network that detect physiological noise |
US11998193B2 (en) | 2018-12-19 | 2024-06-04 | Cilag Gmbh International | Method for usage of the shroud as an aspect of sensing or controlling a powered surgical device, and a control algorithm to adjust its default operation |
US20200214682A1 (en) * | 2019-01-07 | 2020-07-09 | Butterfly Network, Inc. | Methods and apparatuses for tele-medicine |
US11298130B2 (en) | 2019-02-19 | 2022-04-12 | Cilag Gmbh International | Staple cartridge retainer with frangible authentication key |
US11272931B2 (en) | 2019-02-19 | 2022-03-15 | Cilag Gmbh International | Dual cam cartridge based feature for unlocking a surgical stapler lockout |
US11357503B2 (en) | 2019-02-19 | 2022-06-14 | Cilag Gmbh International | Staple cartridge retainers with frangible retention features and methods of using same |
US11517309B2 (en) | 2019-02-19 | 2022-12-06 | Cilag Gmbh International | Staple cartridge retainer with retractable authentication key |
US11751872B2 (en) | 2019-02-19 | 2023-09-12 | Cilag Gmbh International | Insertable deactivator element for surgical stapler lockouts |
US11291445B2 (en) | 2019-02-19 | 2022-04-05 | Cilag Gmbh International | Surgical staple cartridges with integral authentication keys |
US11369377B2 (en) | 2019-02-19 | 2022-06-28 | Cilag Gmbh International | Surgical stapling assembly with cartridge based retainer configured to unlock a firing lockout |
US11291444B2 (en) | 2019-02-19 | 2022-04-05 | Cilag Gmbh International | Surgical stapling assembly with cartridge based retainer configured to unlock a closure lockout |
US11317915B2 (en) | 2019-02-19 | 2022-05-03 | Cilag Gmbh International | Universal cartridge based key feature that unlocks multiple lockout arrangements in different surgical staplers |
US11298129B2 (en) | 2019-02-19 | 2022-04-12 | Cilag Gmbh International | Method for providing an authentication lockout in a surgical stapler with a replaceable cartridge |
US11331101B2 (en) | 2019-02-19 | 2022-05-17 | Cilag Gmbh International | Deactivator element for defeating surgical stapling device lockouts |
US11259807B2 (en) | 2019-02-19 | 2022-03-01 | Cilag Gmbh International | Staple cartridges with cam surfaces configured to engage primary and secondary portions of a lockout of a surgical stapling device |
US11925350B2 (en) | 2019-02-19 | 2024-03-12 | Cilag Gmbh International | Method for providing an authentication lockout in a surgical stapler with a replaceable cartridge |
US11331100B2 (en) | 2019-02-19 | 2022-05-17 | Cilag Gmbh International | Staple cartridge retainer system with authentication keys |
USD952144S1 (en) | 2019-06-25 | 2022-05-17 | Cilag Gmbh International | Surgical staple cartridge retainer with firing system authentication key |
USD950728S1 (en) | 2019-06-25 | 2022-05-03 | Cilag Gmbh International | Surgical staple cartridge |
USD964564S1 (en) | 2019-06-25 | 2022-09-20 | Cilag Gmbh International | Surgical staple cartridge retainer with a closure system authentication key |
US11944422B2 (en) | 2019-08-30 | 2024-04-02 | Auris Health, Inc. | Image reliability determination for instrument localization |
US11207141B2 (en) | 2019-08-30 | 2021-12-28 | Auris Health, Inc. | Systems and methods for weight-based registration of location sensors |
US11147633B2 (en) | 2019-08-30 | 2021-10-19 | Auris Health, Inc. | Instrument image reliability systems and methods |
US11925347B2 (en) | 2019-12-13 | 2024-03-12 | Dinesh Vyas | Stapler apparatus and methods for use |
US20210177402A1 (en) * | 2019-12-13 | 2021-06-17 | Dinesh Vyas | Stapler apparatus and methods for use |
US20230056943A1 (en) * | 2019-12-13 | 2023-02-23 | Dinesh Vyas | Stapler apparatus and methods for use |
US11298195B2 (en) | 2019-12-31 | 2022-04-12 | Auris Health, Inc. | Anatomical feature identification and targeting |
US11660147B2 (en) | 2019-12-31 | 2023-05-30 | Auris Health, Inc. | Alignment techniques for percutaneous access |
US11602372B2 (en) | 2019-12-31 | 2023-03-14 | Auris Health, Inc. | Alignment interfaces for percutaneous access |
US20230087202A1 (en) * | 2021-09-17 | 2023-03-23 | Ford Global Technologies, Llc | Augmented Reality And Touch-Based User Engagement Parking Assist |
Also Published As
Publication number | Publication date |
---|---|
US11369439B2 (en) | 2022-06-28 |
US20180116731A1 (en) | 2018-05-03 |
US20190321107A1 (en) | 2019-10-24 |
US20210113273A1 (en) | 2021-04-22 |
WO2018080844A1 (en) | 2018-05-03 |
US10772686B2 (en) | 2020-09-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11369439B2 (en) | Medical device navigation using a virtual 3D space | |
US11684429B2 (en) | Affected region display associated with a medical device | |
US11931117B2 (en) | Surgical guidance intersection display | |
US11179136B2 (en) | Loupe display | |
US11484365B2 (en) | Medical image guidance | |
US11259879B2 (en) | Selective transparency to assist medical device navigation | |
US20200046315A1 (en) | Medical device guidance | |
US11464575B2 (en) | Systems, methods, apparatuses, and computer-readable media for image guided surgery | |
US8670816B2 (en) | Multiple medical device guidance | |
US20190247130A1 (en) | Systems, methods, apparatuses, and computer-readable media for image management in image-guided medical procedures |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INNEROPTIC TECHNOLOGY, INC., NORTH CAROLINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:STATE, ANDREI;HEANEY, BRIAN;KOHLI, LUV;AND OTHERS;SIGNING DATES FROM 20170329 TO 20170403;REEL/FRAME:041859/0177 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 4 |