Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
  • A61B8/46—Ultrasonic, sonic or infrasonic diagnostic devices with special arrangements for interfacing with the operator or the patient
  • A61B8/467—Ultrasonic, sonic or infrasonic diagnostic devices with special arrangements for interfacing with the operator or the patient characterised by special input means
  • A61B8/468—Ultrasonic, sonic or infrasonic diagnostic devices with special arrangements for interfacing with the operator or the patient characterised by special input means allowing annotation or message recording
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/44—Constructional features of apparatus for radiation diagnosis
  • A61B6/4417—Constructional features of apparatus for radiation diagnosis related to combined acquisition of different diagnostic modalities
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/46—Arrangements for interfacing with the operator or the patient
  • A61B6/467—Arrangements for interfacing with the operator or the patient characterised by special input means
  • A61B6/468—Arrangements for interfacing with the operator or the patient characterised by special input means allowing annotation or message recording
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
  • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
  • A61B6/5229—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image
  • A61B6/5235—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image combining images from the same or different ionising radiation imaging techniques, e.g. PET and CT
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
  • A61B8/08—Clinical applications
  • A61B8/0883—Clinical applications for diagnosis of the heart
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
  • A61B8/12—Diagnosis using ultrasonic, sonic or infrasonic waves in body cavities or body tracts, e.g. by using catheters
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
  • A61B8/42—Details of probe positioning or probe attachment to the patient
  • A61B8/4209—Details of probe positioning or probe attachment to the patient by using holders, e.g. positioning frames
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
  • A61B8/42—Details of probe positioning or probe attachment to the patient
  • A61B8/4245—Details of probe positioning or probe attachment to the patient involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
  • A61B8/44—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
  • A61B8/4416—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device related to combined acquisition of different diagnostic modalities, e.g. combination of ultrasound and X-ray acquisitions
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
  • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
  • A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
  • A61B8/5238—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image
  • A61B8/5261—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image combining images from different diagnostic modalities, e.g. ultrasound and X-ray
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
  • G05B15/00—Systems controlled by a computer
  • G05B15/02—Systems controlled by a computer electric
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
  • A61B6/501—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of the head, e.g. neuroimaging or craniography

Definitions

• This disclosure relates to medical instruments and more particularly to systems and methods for automated steering and maintaining of clinical or standard views using image annotations.
• TEE transesophageal echocardiography
• This task may include a positioning and deployment of a device within a target, a quantitative measurement of the target or an evaluation of the function of the target at a specific point in time.
• TEE views of targets needed in structural heart disease interventions include, but are not limited to the 'en face' view of the mitral valve, mid-esophageal four-chamber view, long-axis view, transgastric view, tri-leaflet aortic valve view, x-plane view, etc. These views can help tremendously to enable the cardiologist and echocardiographer carry out a specific task within the patient, but they are often challenging to obtain and require continuous manual manipulation and adjustment of the TEE probe head within the patient by the
• an annotation device configured to generate annotations in images of a first imaging modality or a second imaging modality.
• a registration module is configured to fuse images of the first and second imaging modalities including the annotations.
• a robot guidance control system is configured to guide a robot in accordance with the annotations and a measured position of the robot to position and maintain an assigned view position in a fused image of the first and second imaging modalities.
• Another imaging system includes a first imaging modality including an x-ray system and a second imaging modality including an ultrasound system with a transesophageal echocardiography (TEE) probe.
• An annotation device is configured to generate annotations in images of at least one of the first imaging modality or the second imaging modality.
• a registration module is configured to fuse images of the first and second imaging modalities including the annotations.
• a robot guidance control system is configured to guide a robot in accordance with the annotations and a measured position of the robot to position and maintain the probe to permit assigned views to be maintained.
• At least one display device has a screen to display a fused image of the first and second imaging modalities such that the fused image maintains an assigned view.
• a method for maintaining an imaging perspective includes generating annotations in images of at least one of a first imaging modality or a second imaging modality; fusing images of the first and second imaging modalities including the annotations; and guiding a robot in accordance with the annotations and a measured position of the robot to position and maintain an assigned view position in a fused image of the first and second imaging modalities.
• FIG. 1 is a block/flow diagram showing a system for directing or maintaining image view in multiple imaging modalities with annotated markings in accordance with one embodiment
• FIG. 2 is an image showing a mitral valve, aortic valve and left atrial appendage ostium and including automatically generated annotations for TEE views in accordance with one illustrative embodiment
• FIG. 3 is another annotated image including an elliptical annotation of an atrial septal defect in accordance with another illustrative embodiment
• FIG. 4 is another annotated image including a point marker annotation on an atrial septal defect in accordance with another illustrative embodiment
• FIG. 5 is a block/flow diagram showing a robotically controlled TEE positioning system in accordance with one embodiment.
• FIG. 6 is a flow diagram showing a method for maintaining an imaging perspective in accordance with illustrative embodiments.
• TEE transesophageal echocardiography
• TEE is an imaging modality (using ultrasound) that is often used to find viewing planes of the anatomy to facilitate execution of specific tasks during the intervention. It can be challenging for the echocardiographer to find a correct viewing plane, since substantial manual manipulation and adjustment (or correction) of the TEE probe position and orientation are needed.
• the present principles provide automated methods that enable the echocardiographer (or interventional cardiologist) to select a specific or pre-defined view for a particular target (e.g., 'en face' view of the mitral valve, mid-esophageal four-chamber view, long-axis view, transgastric view, tri-leaflet aortic valve view, etc.).
• These methods combine robotic manipulation of the TEE probe head position and orientation based on both the selected view of interest and feedback from annotations (or markers) placed on the target using an imaging system, e.g., a fused x- ray/TEE imaging system.
• annotations can be placed or generated on a target in the x- ray and TEE modalities.
• Geometric descriptors of an annotation of a target e.g., geometric coordinates, distances, orientation angles, etc.
• x-ray/TEE image guidance system it is feasible to register x-ray and TEE images (ultrasound images) together into geometric correspondence with each other.
• a fused x-ray/TEE image guidance system e.g., Philips ® EchoNavigatorTM
• These annotations may include geometric information about the target.
• geometric information from the annotation(s) can be generated and delivered as continuous feedback to a robotic TEE probe positioning system.
• the robotic TEE probe positioning system adjusts probe position and orientation automatically to obtain the desired TEE view for the specific task in question.
• the present principles generate geometric data from annotations that have been placed on clinical targets in images on the fused x-ray/TEE system. These can be used as continuous feedback to automatically adjust TEE probe position after a desired TEE view has been selected.
• the present invention will be described in terms of medical instruments; however, the teachings of the present invention are much broader and are applicable to any imaging instruments and imaging modalities.
• the present principles are employed in tracking or analyzing complex biological or mechanical systems.
• the present principles are applicable to internal tracking procedures of biological systems and procedures in all areas of the body such as the lungs, gastro-intestinal tract, excretory organs, blood vessels, etc.
• the elements depicted in the FIGS may be implemented in various combinations of hardware and software and provide functions which may be combined in a single element or multiple elements.
• the functions of the various elements shown in the FIGS can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software.
• processors When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term "processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory
• DSP digital signal processor
• ROM read-only memory
• RAM random access memory
• non-volatile storage etc.
• embodiments of the present invention can take the form of a computer program product accessible from a computer-usable or computer-readable storage medium providing program code for use by or in connection with a computer or any instruction execution system.
• a computer-usable or computer readable storage medium can be any apparatus that may include, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
• the medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium.
• Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk.
• Current examples of optical disks include compact disk - read only memory (CD-ROM), compact disk - read/write (CD-R/W), Blu-RayTM and DVD.
• such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C).
• This may be extended, as readily apparent by one of ordinary skill in this and related arts, for as many items listed.
• System 100 may include a workstation or console 112 from which a procedure is supervised and/or managed.
• Workstation 112 preferably includes one or more processors 114 and memory 116 for storing programs and applications.
• Memory 116 may store a registration module 115 configured to align coordinate systems from multiple imaging systems and/or devices.
• a medical device or instrument 102 may include a catheter, a guidewire, a probe, an endoscope, an electrode, a filter device, a balloon device, or other medical component, etc.
• the medical device 102 is passed into the subject through a natural orifice or through a port surgically installed through the subject to enter an internal lumen, such as the esophagus or the like.
• the medical device 102 may be guided and controlled using a robot 144.
• the robot 144 is controlled by a robot guidance/control device 156.
• the guidance/control device 156 uses markings or other criteria set up in the system 100 by an annotation device 154 to guide the robot 144.
• the annotation device 154 includes markings or annotations input by a user or automatically generated using image processing to assist in achieving a plurality of different views.
• the views may be a predetermined/pre-defined set of views such that once the markings are in place the views can be achieved repeatably and maintained throughout a session or procedure.
• annotation device 154 is configured to receive feedback from the user or from another part of the system 100 and place markings within a preoperative image or real-time image from a first imaging system 136 (e.g., x-ray). The image from the first imaging system 136 is then registered to or fused with to an image from a second imaging system 138 (e.g., a TEE image) (or vice versa) using the registration module 1 15. Any suitable registration method may be employed.
• Workstation 1 12 includes one or more displays 1 18 for viewing internal images of a subject (patient) or volume 134 and may include rendering images 152 as fused or as overlays of one other with the annotated markings. Displays 1 18 may also permit a user or users to interact with other users, or with the workstation 1 12 and its components and functions, or any other element within the system 100. This is further facilitated by an interface 130, which may include a keyboard, mouse, a joystick, a haptic device, speakers, microphone or any other peripheral or control to permit user feedback from and interaction with the workstation 1 12. The interface 130 may also be employed to permit the user or users to input annotations or markings to function as guideposts for the robot positioning and imaging.
• system 100 is configured to perform transesophageal echocardiography (TEE) imaging for a procedure.
• the imaging system 136 includes an x-ray system and the imaging system 138 includes a TEE imaging system.
• TEE transesophageal echocardiography
• an echocardiographer and an interventionalist may be present.
• a desired clinical TEE view is agreed upon by the echocardiographer and the interventionalist.
• the view may be a standard or commonly used view, (e.g., 'en face' view of the mitral or aortic valve, mid-esophageal four-chamber view, long-axis view, transgastric view, tri-leaflet aortic valve view, etc.).
• the selected view may be chosen on either the robot 144 (e.g., a robotic TEE probe positioning system) using TEE imaging 138 by the echocardiographer or on a fused x-ray/TEE image (registered TEE imaging 138 and x-ray imaging 136) by the interventionalist.
• system 100 may be referred to as a fused x-ray/TEE image guidance system.
• Annotations relevant to the anatomy and view of interest may automatically be generated by the annotation device 154 on fused x-ray/TEE images 152 and displayed to both the interventionalist and echocardiographer on displays 118.
• a curved annotation may be drawn (either automatically or manually) to outline commissures of the mitral valve.
• the 3D geometric information generated by this annotation could then be sent to the robot 144 to orient and position the TEE probe 102 for imaging 138 such that a clear 'en-face' view of the mitral valve can be displayed in 3D TEE images 152.
• These images 152 can then be used to align and position a mitral clip optimally between the mitral valve commissures before deploying the clip in a mitral clipping procedure.
• an elliptical or curved annotation can be automatically or manually placed on the ostium of the left atrial appendage.
• the geometric information from this annotation can be sent as feedback to the robot 144, which can then automatically position the TEE probe 102 such that an optimal TEE view can be generated to help guide and deploy a closure device or plug in the left atrial appendage.
• Geometric descriptors of an annotation of a target can be used as feedback to the robotic guidance/control system 156 and can orient and position the TEE probe head (102) such that the desired view can be achieved.
• the necessary geometric data associated with the annotation(s) (e.g., 3D-coordinate positions, distances, normal vectors, angulation and curvature, centroids, area, major and minor axes, eccentricity, etc.) is sent as continuous feedback to the robot 144 (e.g., robotic TEE probe positioning system) to triangulate relative positions and orientations between the annotation(s) and the probe/instrument 102.
• This information can then be employed to position and deploy the probe 102 (e.g., TEE probe) appropriately in a continuous feedback loop to provide and maintain the desired TEE view.
• the repositioning of the probe 102 is not carried out automatically by the robot 144.
• the guidance control 156 may provide cues (e.g., sensory feedback, such as visual indicators on display 1 18, audio through interface 130, haptic feedback through interface, etc.) to enable an operator to actuate the necessary degrees of freedom of the robot 144 via the user interface 130, such as a joystick, to obtain an optimum visualization.
• the operator is 'in the loop' and the control software of the guidance control device 156 guides their actions using annotations as guide posts or references. For example, a sound (or visual signal) may indicate that the probe is getting closer or further from a desired position (annotation).
• a warning system 160 may be connected to the annotation device 154.
• the warning system 160 monitors any deviation of a current view from the intended (e.g., standard view).
• the warning system 160 may generate messages or provide color coding of annotations to indicate the amount and/or direction of the deviation.
• a color coded system may be generated on the fused x-ray/TEE image (or just the TEE image) to show how much the current TEE view deviates from the desired chosen view, (in terms of orientation, position, angle, etc.). This can assist whether the TEE probe position is manually or automatically adjusted.
• the warning system 160 could also be employed as a safety feature to tell the operator if a given view is not possible.
• an automatically generated annotation from a 3D-zoom TEE view of a mitral valve 202, aortic valve 204 and left atrial appendage ostium 206 are shown in accordance with one illustrative embodiment.
• the annotations are generated by the annotation device 154 or other software configured to make measurements based on features deciphered in the images.
• the annotations may include ellipses, points, lines, etc. between points of interest, etc. These annotations may be generated based on previous selections by the user to highlight different known features in a region.
• another annotated image includes an elliptical annotation 208 of an atrial septal defect.
• a point marker annotation 210 is shown on an atrial septal defect.
• FIG. 2 the examples of the automatically generated annotation of the mitral valve 202, aortic valve 204 and left atrial appendage ostium 206 are shown using a fused x-ray/TEE image.
• the system 100 is able to generate these annotations for 3D-zoom and full volume TEE views containing all three structures at once.
• the level of geometric detail in these annotations is currently limited to 3D position coordinates, but in accordance with the present principles, additional geometric data is employed for these annotations and can be generated and used as feedback to a robotic TEE positioning system to move the probe to a desired view.
• 3 and 4 show less detailed annotations such as point markers 210 and ellipses 208 from which detailed geometric information can be generated to steer the TEE probe head to less conventional or nonstandard clinical TEE views.
• An example of this could be steering to a color Doppler TEE view that shows a paravalvular leak in detail in an en face or long axis orientation, around a previously implanted prosthetic aortic valve.
• shapes and contours of the annotations (204-210) may be automatically and dynamically altered in the fused x-ray/TEE image as the anatomy of interest moves and changes.
• the corresponding geometric feedback to be delivered to the robot 144 may also be updated accordingly. For example, as the esophagus flexes so to do the shapes of the annotations (204-210).
• the robot 144 may be updated to adjust for these movements.
• interventional tools can be automatically annotated by the annotation device 154.
• an interventional catheter can be detected and annotated in an x-ray image and that information can be used to automatically steer the TEE probe head 102 to less conventional or non-standard clinical TEE views showing the catheter.
• This device view can be saved alongside the standard clinical views. This is particularly useful if an interventional device is visible in the first imaging modality and not visible in the second imaging modality.
• the annotation can improve the device steering by depicting a virtual representation of the device in the second imaging modality using a geometric representation.
• an automatic generation of more detailed geometric data from less detailed geometric data may be provided. For example, if several points on an ellipse annotation (e.g., ellipse 208) are generated and known after placing the annotation, major and minor axes, area, eccentricity and the plane occupied may also be computed for the ellipse 208. From this plane, the normal to the ellipse 208 and the position of its center could be determined. This information may be important for the robot 144 (FIG. 1) to calculate how the probe 102 (FIG. 1) should be moved to visualize a target in the desired view.
• an ellipse annotation e.g., ellipse 208
• major and minor axes, area, eccentricity and the plane occupied may also be computed for the ellipse 208. From this plane, the normal to the ellipse 208 and the position of its center could be determined. This information may be important for the robot 144 (FIG. 1) to calculate how the probe 102 (FI
• Another example of this includes computing the centroid and normal of a 3D curve annotation, e.g., curve 206, which will provide information about its position and orientation relative to the TEE probe 102. Initially, the position of the center of a transducer of the probe head would be compared with the geometry of the annotations of interest for a particular view, but this could be extended to include additional details about the probe head transducer such as its area, dimensions and normal, to further assist with automatic probe positioning.
• a 3D curve annotation e.g., curve 206
• a robotically controlled TEE positioning system 300 (e.g., robot 144, FIG. 1) is illustratively shown.
• the system 300 may be retrofitted with an existing TEE probe 302 and employed to remotely control a position and orientation of a distal tip of the probe 302.
• the system 300 includes an integrated motor or motors 304 and encoders 306 to provide position feedback to control software, e.g., robot guidance/control 156.
• the robot guidance/control 156 continuously monitors currents of the motor 304 and/or the encoders 306 to ensure the system 300 does not apply excessive force (or over extends) when in-vivo.
• the system 300 can accept geometric data from the annotations in the image views and combine the annotations with its own position and orientation data to compute instructions, e.g., how to actuate the probe 302 until a desired view is obtained.
• the geometric data can be sent to the system 300 from the workstation 112 (FIG. 1) in a continuous feedback loop such that the view can be maintained if the patient or probe is manually moved. In other words, the system 300 will continuously hold a view despite any movement.
• the system 300 may include a display 310 such that a user (e.g., echocardiographer) can select the desired TEE view right on the system 300.
• the annotation device 154 may be included on the TEE imaging system 138.
• the annotation generation capability along with the position and orientation data generation for a control system of the robotic TEE probe positioning system 300 is directly on the TEE imaging system 138 (FIG. 1). This can be useful, in situations where fused x-ray/TEE imaging is not employed.
• the annotation capability may be extended to automatically (robotically) position less invasive transthoracic echocardiography (TTE) ultrasound imaging probes as well for specific views typically acquired with this modality.
• TTE transthoracic echocardiography
• the present principles improve image guidance and workflow for both the echocardiographer and interventionalist when using TEE and x-ray image guidance, e.g., in interventional cardiology and structural heart disease interventions.
• the present principles may be extended to other imaging modalities and procedures.
• the present principles may be employed as enhancements to systems such as, e.g., the Philips®
• EchoNavigatorTM fused x-ray/TEE image guidance system and CX50, iE33 and EPIQ ultrasound imaging systems.
• annotations are generated in images of at least one imaging modality.
• the annotations may be generated automatically in accordance with physical features in a view or manually by adding annotations using a user interface.
• images of two or more imaging modalities are fused including the annotations.
• the imaging modalities may include an x-ray system and an ultrasound system where the probe may include a transesophageal echocardiography (TEE) probe.
• TEE transesophageal echocardiography
• a robot is guided in accordance with the annotations and a measured position of the robot. This permits a position to be achieved and maintained for a probe (e.g., an ultrasound imaging probe or TEE probe) in an assigned view position in a fused image of the first and second imaging modalities.
• the assigned view may include a standard view or a view created by the user.
• the robot guidance may include tracking movement of the annotations to adjust the position of the probe.
• the annotations may be dynamically changed in accordance with patient or device movement, the robot tracking may continue using the updated annotations.
• differences between annotations may be monitored between a current view with annotations and a previous view.
• the changes may be indicated or employ a color coded system to show deviations.
• a warning system may be provided to alert a user of the deviations or to provide guidance to the user for manually achieving or maintaining an assigned view.
• the robot may be manually guided using sensory feedback and/or the annotations as a reference to achieve and maintain a position.

Landscapes

• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Engineering & Computer Science (AREA)
• Medical Informatics (AREA)
• Physics & Mathematics (AREA)
• Public Health (AREA)
• General Health & Medical Sciences (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• Radiology & Medical Imaging (AREA)
• Animal Behavior & Ethology (AREA)
• Veterinary Medicine (AREA)
• Biomedical Technology (AREA)
• Molecular Biology (AREA)
• Surgery (AREA)
• Pathology (AREA)
• Biophysics (AREA)
• Heart & Thoracic Surgery (AREA)
• High Energy & Nuclear Physics (AREA)
• Optics & Photonics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Cardiology (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Automation & Control Theory (AREA)
• Human Computer Interaction (AREA)
• Ultra Sonic Daignosis Equipment (AREA)
• Apparatus For Radiation Diagnosis (AREA)
• Epidemiology (AREA)
• Primary Health Care (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=57868299

Family Applications (1)

Country Status (5)

Cited By (12)

Families Citing this family (7)

Citations (3)

Family Cites Families (35)

• 2017
  • 2017-01-09 WO PCT/IB2017/050075 patent/WO2017122109A1/en not_active Ceased
  • 2017-01-09 EP EP17701188.9A patent/EP3402408B1/en active Active
  • 2017-01-09 CN CN201780006566.7A patent/CN108471998B/zh active Active
  • 2017-01-09 JP JP2018536234A patent/JP6902547B2/ja active Active
  • 2017-01-09 US US16/069,082 patent/US20190021699A1/en not_active Abandoned

Patent Citations (3)

Also Published As

Similar Documents

Legal Events