US20230096715A1

US20230096715A1 - Surgical Systems and Methods for Selectively Pressurizing a Natural Body Lumen

Info

Publication number: US20230096715A1
Application number: US17/449,772
Authority: US
Inventors: Frederick E. Shelton, IV; Jason L. Harris
Original assignee: Cilag GmbH International
Current assignee: Cilag GmbH International
Priority date: 2021-09-29
Filing date: 2021-10-01
Publication date: 2023-03-30
Also published as: EP4281001A1; US20230107857A1; US20230095278A1; US20230100415A1; US20230098538A1; EP4251077A1; US20230097151A1; US11992200B2; US20230101376A1; US20230101757A1; US11937798B2; US20230094881A1; US20230404562A1; US20230107005A1; US11937799B2; US11723642B2; US20230096880A1

Abstract

Surgical systems for use with a surgical instrument for endoluminal access are provided. In one exemplary embodiment, the surgical instrument includes at least one deployable sealing element and fluid channel. The at least one deployable sealing element is operatively coupled to the surgical instrument and configured to move between unexpanded and expanded states. When the sealing element is in the expanded state, the deployable sealing element is configured to form a first seal at a portion of a natural body lumen. The fluid channel extends through the surgical instrument and has an opening distal to the first seal. The fluid channel is configured to allow fluid ingress and egress distal to the portion of the natural body lumen while the at least one deployable sealing member is in the expanded state, thereby selectively pressurizing the natural body lumen distal to the portion. Methods for using the same are also provided.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent Application No. 63/249,980 filed on Sep. 29, 2021, and entitled “Cooperative Access,” the disclosure of which is incorporated herein by reference in its entirety.

FIELD

Surgical systems for selectively pressurizing a natural body lumen and methods of using the same are provided.

BACKGROUND

Surgical systems often incorporate an imaging system, which can allow medical practitioners to view a surgical site and/or one or more portions thereof on one or more displays, (e.g., a monitor, a computer tablet screen, etc.). The display(s) can be local and/or remote to a surgical theater. The imaging system can include a scope with a camera that views the surgical site and transmits the view to the one or more displays viewable by medical practitioner(s).
Imaging systems can be limited by the information that they are able to recognize and/or convey to the medical practitioner(s). For example, certain concealed structures, physical contours, and/or dimensions within a three-dimensional space may be unrecognizable intraoperatively by certain imaging systems. For another example, certain imaging systems may be incapable of communicating and/or conveying certain information to the medical practitioner(s) intraoperatively.
Accordingly, there remains a need for improved surgical imaging.

SUMMARY

Surgical systems for use with a surgical instrument for endoluminal access are provided. In one exemplary embodiment, the surgical instrument includes at least one deployable sealing element and fluid channel. The at least one deployable sealing element is operatively coupled to the surgical instrument and configured to move between unexpanded and expanded states. When the sealing element is in the expanded state, the deployable sealing element is configured to form a first seal at a portion of a natural body lumen or an organ. The fluid channel extends through the surgical instrument and has an opening distal to the first seal. The fluid channel is configured to allow fluid ingress and egress distal to the portion of the natural body lumen or the organ while the at least one deployable sealing member is in the expanded state, thereby selectively pressurizing the natural body lumen or the organ distal to the portion.
The at least one deployable sealing element can have a variety of configurations. In some embodiments, the at least one deployable sealing element can be configured to expand to contact an internal surface of the natural body lumen or the organ. In certain embodiments, at least one deployable sealing element can be an inflatable balloon configured to be filled with a fluid to move from the unexpanded state to the expanded state.
In some embodiments, the surgical instrument can include an optical sensor arranged at a distal end thereof.
The surgical instrument can have a variety of configurations. In some embodiments, the surgical instrument can include a second deployable sealing element coupled to the surgical instrument and distal to the opening of the fluid channel. The second deployable sealing element can be configured to move between unexpanded and expanded states. In certain embodiments, when in the expanded state, the second deployable sealing element can be configured to form a second seal at a second portion of the natural body lumen or the organ. In some embodiments, the at least one deployable sealing element and the second deployable sealing element can be expanded separately. In other embodiments, the at least one deployable sealing element and the second deployable sealing element can be expanded simultaneously.
In some embodiments, when pressurized, the portion of the natural body lumen or organ distal to the at least one deployable sealing element has a first pressure, and a portion outside of the natural body lumen or organ has a second pressure, different than the first pressure.
Methods of operating the surgical systems are also provided. In one exemplary embodiment, a method of operating a surgical system can include inserting a surgical instrument into a natural body lumen or an organ. The surgical instrument has a fluid channel extending therethrough and at least one deployable sealing element operatively coupled to the surgical instrument. The method can include expanding a first deployable sealing element of the at least one deployable sealing element from an unexpanded state to an expanded state to form a first seal within the natural body lumen or the organ. The method can further include injecting fluid through the fluid channel and into a portion of the natural body lumen or the organ distal to the first seal to thereby inflate the portion of the natural body lumen or the organ. The method can further include pressurizing the portion of the natural body lumen or the organ.
The at least one deployable sealing element can have a variety of configurations. In some embodiments, the at least one deployable sealing element can be configured to expand to contact an internal surface of the natural body lumen or the organ. In some embodiments, at least one deployable sealing element can be configured to be filled with a fluid to move from the unexpanded state to the expanded state.
The surgical instrument can have a variety of configurations. In some embodiments, the surgical instrument can further include a second deployable sealing element coupled to the surgical instrument and distal to the opening of the fluid channel. The second deployable sealing element can be configured to transition between unexpanded and expanded states. In other embodiments, when in the expanded state, the second deployable sealing element can be configured to form a second seal within the natural body lumen or the organ, wherein the portion of the natural body lumen or the organ is located between the first and second deployable sealing elements. In some embodiments, a pressure differential can be created within the portion of the natural body lumen or the organ relative to an area outside of the natural body lumen or the organ. In other embodiments, the at least one deployable sealing element and the second deployable sealing element can be expanded separately. In certain embodiments, the at least one deployable sealing element and the second deployable sealing element can be expanded simultaneously.

BRIEF DESCRIPTION OF DRAWINGS

The present invention is described by way of reference to the accompanying figures which are as follows:

FIG. 1 is a schematic view of one embodiment of a surgical visualization system;

FIG. 2 is a schematic view of triangularization between a surgical device, an imaging device, and a critical structure of FIG. 1 ;

FIG. 3 is a schematic view of another embodiment of a surgical visualization system;

FIG. 4 is a schematic view of one embodiment of a control system for a surgical visualization system;

FIG. 5 is a schematic view of one embodiment of a control circuit of a control system for a surgical visualization system;

FIG. 6 is a schematic view of one embodiment of a combinational logic circuit of a surgical visualization system;

FIG. 7 is a schematic view of one embodiment of a sequential logic circuit of a surgical visualization system;

FIG. 8 is a schematic view of yet another embodiment of a surgical visualization system;

FIG. 9 is a schematic view of another embodiment of a control system for a surgical visualization system;

FIG. 10 is a graph showing wavelength versus absorption coefficient for various biological materials;

FIG. 11 is a schematic view of one embodiment of a spectral emitter visualizing a surgical site;

FIG. 12 is a graph depicting illustrative hyperspectral identifying signatures to differentiate a ureter from obscurants;

FIG. 13 is a graph depicting illustrative hyperspectral identifying signatures to differentiate an artery from obscurants;

FIG. 14 is a graph depicting illustrative hyperspectral identifying signatures to differentiate a nerve from obscurants;

FIG. 15 is a schematic view of one embodiment of a near infrared (NIR) time-of-flight measurement system being utilized intraoperatively;

FIG. 16 shows a time-of-flight timing diagram for the system of FIG. 15 ;

FIG. 17 is a schematic view of another embodiment of a near infrared (NIR) time-of-flight measurement system being utilized intraoperatively;

FIG. 18 is a schematic view of one embodiment of a computer-implemented interactive surgical system;

FIG. 19 is a schematic view of one embodiment a surgical system being used to perform a surgical procedure in an operating room;

FIG. 20 is a schematic view of one embodiment of a surgical system including a smart surgical instrument and a surgical hub;

FIG. 21 is a flowchart showing a method of controlling the smart surgical instrument of FIG. 20 ;

FIG. 22 is a schematic view of one embodiment of a surgical system;

FIG. 23 is a schematic view of the embodiment of the surgical system of FIG. 22 ;

FIG. 24 is a schematic view of the embodiment of the surgical system of FIG. 22 ;

FIG. 25 is a schematic view of a colon;

FIG. 26 is a schematic view of another embodiment of a surgical system;

FIG. 27 is a schematic view of the embodiment of the surgical system of FIG. 26 ; and

FIG. 28 is a graph depicting the pressurization testing of the surgical system of FIG. 26 .

DETAILED DESCRIPTION

Certain exemplary embodiments will now be described to provide an overall understanding of the principles of the structure, function, manufacture, and use of the devices, systems, and methods disclosed herein. One or more examples of these embodiments are illustrated in the accompanying drawings. A person skilled in the art will understand that the devices, systems, and methods specifically described herein and illustrated in the accompanying drawings are non-limiting exemplary embodiments and that the scope of the present invention is defined solely by the claims. The features illustrated or described in connection with one exemplary embodiment may be combined with the features of other embodiments. Such modifications and variations are intended to be included within the scope of the present invention.
Further, in the present disclosure, like-named components of the embodiments generally have similar features, and thus within a particular embodiment each feature of each like-named component is not necessarily fully elaborated upon. Additionally, to the extent that linear or circular dimensions are used in the description of the disclosed systems, devices, and methods, such dimensions are not intended to limit the types of shapes that can be used in conjunction with such systems, devices, and methods. A person skilled in the art will recognize that an equivalent to such linear and circular dimensions can easily be determined for any geometric shape. A person skilled in the art will appreciate that a dimension may not be a precise value but nevertheless be considered to be at about that value due to any number of factors such as manufacturing tolerances and sensitivity of measurement equipment. Sizes and shapes of the systems and devices, and the components thereof, can depend at least on the size and shape of components with which the systems and devices will be used.
Surgical Visualization
In general, a surgical visualization system is configured to leverage “digital surgery” to obtain additional information about a patient's anatomy and/or a surgical procedure. The surgical visualization system is further configured to convey data to one or more medical practitioners in a helpful manner. Various aspects of the present disclosure provide improved visualization of the patient's anatomy and/or the surgical procedure, and/or use visualization to provide improved control of a surgical tool (also referred to herein as a “surgical device” or a “surgical instrument”).
“Digital surgery” can embrace robotic systems, advanced imaging, advanced instrumentation, artificial intelligence, machine learning, data analytics for performance tracking and benchmarking, connectivity both inside and outside of the operating room (OR), and more. Although various surgical visualization systems described herein can be used in combination with a robotic surgical system, surgical visualization systems are not limited to use with a robotic surgical system. In certain instances, surgical visualization using a surgical visualization system can occur without robotics and/or with limited and/or optional robotic assistance. Similarly, digital surgery can occur without robotics and/or with limited and/or optional robotic assistance.
In certain instances, a surgical system that incorporates a surgical visualization system may enable smart dissection in order to identify and avoid critical structures. Critical structures include anatomical structures such as a ureter, an artery such as a superior mesenteric artery, a vein such as a portal vein, a nerve such as a phrenic nerve, and/or a tumor, among other anatomical structures. In other instances, a critical structure can be a foreign structure in the anatomical field, such as a surgical device, a surgical fastener, a clip, a tack, a bougie, a band, a plate, and other foreign structures. Critical structures can be determined on a patient-by-patient and/or a procedure-by-procedure basis. Smart dissection technology may provide, for example, improved intraoperative guidance for dissection and/or may enable smarter decisions with critical anatomy detection and avoidance technology.
A surgical system incorporating a surgical visualization system may enable smart anastomosis technologies that provide more consistent anastomoses at optimal location(s) with improved workflow. Cancer localization technologies may be improved with a surgical visualization platform. For example, cancer localization technologies can identify and track a cancer location, orientation, and its margins. In certain instances, the cancer localization technologies may compensate for movement of a surgical instrument, a patient, and/or the patient's anatomy during a surgical procedure in order to provide guidance back to the point of interest for medical practitioner(s).
A surgical visualization system may provide improved tissue characterization and/or lymph node diagnostics and mapping. For example, tissue characterization technologies may characterize tissue type and health without the need for physical haptics, especially when dissecting and/or placing stapling devices within the tissue. Certain tissue characterization technologies may be utilized without ionizing radiation and/or contrast agents. With respect to lymph node diagnostics and mapping, a surgical visualization platform may, for example, preoperatively locate, map, and ideally diagnose the lymph system and/or lymph nodes involved in cancerous diagnosis and staging.
During a surgical procedure, information available to a medical practitioner via the “naked eye” and/or an imaging system may provide an incomplete view of the surgical site. For example, certain structures, such as structures embedded or buried within an organ, can be at least partially concealed or hidden from view. Additionally, certain dimensions and/or relative distances can be difficult to ascertain with existing sensor systems and/or difficult for the “naked eye” to perceive. Moreover, certain structures can move pre-operatively (e.g., before a surgical procedure but after a preoperative scan) and/or intraoperatively. In such instances, the medical practitioner can be unable to accurately determine the location of a critical structure intraoperatively.
When the position of a critical structure is uncertain and/or when the proximity between the critical structure and a surgical tool is unknown, a medical practitioner's decision-making process can be inhibited. For example, a medical practitioner may avoid certain areas in order to avoid inadvertent dissection of a critical structure; however, the avoided area may be unnecessarily large and/or at least partially misplaced. Due to uncertainty and/or overly/excessive exercises in caution, the medical practitioner may not access certain desired regions. For example, excess caution may cause a medical practitioner to leave a portion of a tumor and/or other undesirable tissue in an effort to avoid a critical structure even if the critical structure is not in the particular area and/or would not be negatively impacted by the medical practitioner working in that particular area. In certain instances, surgical results can be improved with increased knowledge and/or certainty, which can allow a surgeon to be more accurate and, in certain instances, less conservative/more aggressive with respect to particular anatomical areas.
A surgical visualization system can allow for intraoperative identification and avoidance of critical structures. The surgical visualization system may thus enable enhanced intraoperative decision making and improved surgical outcomes. The surgical visualization system can provide advanced visualization capabilities beyond what a medical practitioner sees with the “naked eye” and/or beyond what an imaging system can recognize and/or convey to the medical practitioner. The surgical visualization system can augment and enhance what a medical practitioner is able to know prior to tissue treatment (e.g., dissection, etc.) and, thus, may improve outcomes in various instances. As a result, the medical practitioner can confidently maintain momentum throughout the surgical procedure knowing that the surgical visualization system is tracking a critical structure, which may be approached during dissection, for example. The surgical visualization system can provide an indication to the medical practitioner in sufficient time for the medical practitioner to pause and/or slow down the surgical procedure and evaluate the proximity to the critical structure to prevent inadvertent damage thereto. The surgical visualization system can provide an ideal, optimized, and/or customizable amount of information to the medical practitioner to allow the medical practitioner to move confidently and/or quickly through tissue while avoiding inadvertent damage to healthy tissue and/or critical structure(s) and, thus, to minimize the risk of harm resulting from the surgical procedure.
Surgical visualization systems are described in detail below. In general, a surgical visualization system can include a first light emitter configured to emit a plurality of spectral waves, a second light emitter configured to emit a light pattern, and a receiver, or sensor, configured to detect visible light, molecular responses to the spectral waves (spectral imaging), and/or the light pattern. The surgical visualization system can also include an imaging system and a control circuit in signal communication with the receiver and the imaging system. Based on output from the receiver, the control circuit can determine a geometric surface map, e.g., three-dimensional surface topography, of the visible surfaces at the surgical site and a distance with respect to the surgical site, such as a distance to an at least partially concealed structure. The imaging system can convey the geometric surface map and the distance to a medical practitioner. In such instances, an augmented view of the surgical site provided to the medical practitioner can provide a representation of the concealed structure within the relevant context of the surgical site. For example, the imaging system can virtually augment the concealed structure on the geometric surface map of the concealing and/or obstructing tissue similar to a line drawn on the ground to indicate a utility line below the surface. Additionally or alternatively, the imaging system can convey the proximity of a surgical tool to the visible and obstructing tissue and/or to the at least partially concealed structure and/or a depth of the concealed structure below the visible surface of the obstructing tissue. For example, the visualization system can determine a distance with respect to the augmented line on the surface of the visible tissue and convey the distance to the imaging system.
Throughout the present disclosure, any reference to “light,” unless specifically in reference to visible light, can include electromagnetic radiation (EMR) or photons in the visible and/or non-visible portions of the EMR wavelength spectrum. The visible spectrum, sometimes referred to as the optical spectrum or luminous spectrum, is that portion of the electromagnetic spectrum that is visible to (e.g., can be detected by) the human eye and may be referred to as “visible light” or simply “light.” A typical human eye will respond to wavelengths in air that are from about 380 nm to about 750 nm. The invisible spectrum (e.g., the non-luminous spectrum) is that portion of the electromagnetic spectrum that lies below and above the visible spectrum. The invisible spectrum is not detectable by the human eye. Wavelengths greater than about 750 nm are longer than the red visible spectrum, and they become invisible infrared (IR), microwave, and radio electromagnetic radiation. Wavelengths less than about 380 nm are shorter than the violet spectrum, and they become invisible ultraviolet, x-ray, and gamma ray electromagnetic radiation.
FIG. 1 illustrates one embodiment of a surgical visualization system 100. The surgical visualization system 100 is configured to create a visual representation of a critical structure 101 within an anatomical field. The critical structure 101 can include a single critical structure or a plurality of critical structures. As discussed herein, the critical structure 101 can be any of a variety of structures, such as an anatomical structure, e.g., a ureter, an artery such as a superior mesenteric artery, a vein such as a portal vein, a nerve such as a phrenic nerve, a vessel, a tumor, or other anatomical structure, or a foreign structure, e.g., a surgical device, a surgical fastener, a surgical clip, a surgical tack, a bougie, a surgical band, a surgical plate, or other foreign structure. As discussed herein, the critical structure 101 can be identified on a patient-by-patient and/or a procedure-by-procedure basis. Embodiments of critical structures and of identifying critical structures using a visualization system are further described in U.S. Pat. No. 10,792,034 entitled “Visualization Of Surgical Devices” issued Oct. 6, 2020, which is hereby incorporated by reference in its entirety.
In some instances, the critical structure 101 can be embedded in tissue 103. The tissue 103 can be any of a variety of tissues, such as fat, connective tissue, adhesions, and/or organs. Stated differently, the critical structure 101 may be positioned below a surface 105 of the tissue 103. In such instances, the tissue 103 conceals the critical structure 101 from the medical practitioner's “naked eye” view. The tissue 103 also obscures the critical structure 101 from the view of an imaging device 120 of the surgical visualization system 100. Instead of being fully obscured, the critical structure 101 can be partially obscured from the view of the medical practitioner and/or the imaging device 120.
The surgical visualization system 100 can be used for clinical analysis and/or medical intervention. In certain instances, the surgical visualization system 100 can be used intraoperatively to provide real-time information to the medical practitioner during a surgical procedure, such as real-time information regarding proximity data, dimensions, and/or distances. A person skilled in the art will appreciate that information may not be precisely real time but nevertheless be considered to be real time for any of a variety of reasons, such as time delay induced by data transmission, time delay induced by data processing, and/or sensitivity of measurement equipment. The surgical visualization system 100 is configured for intraoperative identification of critical structure(s) and/or to facilitate the avoidance of the critical structure(s) 101 by a surgical device. For example, by identifying the critical structure 101, a medical practitioner can avoid maneuvering a surgical device around the critical structure 101 and/or a region in a predefined proximity of the critical structure 101 during a surgical procedure. For another example, by identifying the critical structure 101, a medical practitioner can avoid dissection of and/or near the critical structure 101, thereby helping to prevent damage to the critical structure 101 and/or helping to prevent a surgical device being used by the medical practitioner from being damaged by the critical structure 101.
The surgical visualization system 100 is configured to incorporate tissue identification and geometric surface mapping in combination with the surgical visualization system's distance sensor system 104. In combination, these features of the surgical visualization system 100 can determine a position of a critical structure 101 within the anatomical field and/or the proximity of a surgical device 102 to the surface 105 of visible tissue 103 and/or to the critical structure 101. Moreover, the surgical visualization system 100 includes an imaging system that includes the imaging device 120 configured to provide real-time views of the surgical site. The imaging device 120 can include, for example, a spectral camera (e.g., a hyperspectral camera, multispectral camera, or selective spectral camera), which is configured to detect reflected spectral waveforms and generate a spectral cube of images based on the molecular response to the different wavelengths. Views from the imaging device 120 can be provided in real time to a medical practitioner, such as on a display (e.g., a monitor, a computer tablet screen, etc.). The displayed views can be augmented with additional information based on the tissue identification, landscape mapping, and the distance sensor system 104. In such instances, the surgical visualization system 100 includes a plurality of subsystems—an imaging subsystem, a surface mapping subsystem, a tissue identification subsystem, and/or a distance determining subsystem. These subsystems can cooperate to intra-operatively provide advanced data synthesis and integrated information to the medical practitioner.
The imaging device 120 can be configured to detect visible light, spectral light waves (visible or invisible), and a structured light pattern (visible or invisible). Examples of the imaging device 120 includes scopes, e.g., an endoscope, an arthroscope, an angioscope, a bronchoscope, a choledochoscope, a colonoscope, a cytoscope, a duodenoscope, an enteroscope, an esophagogastro-duodenoscope (gastroscope), a laryngoscope, a nasopharyngo-neproscope, a sigmoidoscope, a thoracoscope, an ureteroscope, or an exoscope. Scopes can be particularly useful in minimally invasive surgical procedures. In open surgery applications, the imaging device 120 may not include a scope.
The tissue identification subsystem can be achieved with a spectral imaging system. The spectral imaging system can rely on imaging such as hyperspectral imaging, multispectral imaging, or selective spectral imaging. Embodiments of hyperspectral imaging of tissue are further described in U.S. Pat. No. 9,274,047 entitled “System And Method For Gross Anatomic Pathology Using Hyperspectral Imaging” issued Mar. 1, 2016, which is hereby incorporated by reference in its entirety.
The surface mapping subsystem can be achieved with a light pattern system. Various surface mapping techniques using a light pattern (or structured light) for surface mapping can be utilized in the surgical visualization systems described herein. Structured light is the process of projecting a known pattern (often a grid or horizontal bars) on to a surface. In certain instances, invisible (or imperceptible) structured light can be utilized, in which the structured light is used without interfering with other computer vision tasks for which the projected pattern may be confusing. For example, infrared light or extremely fast frame rates of visible light that alternate between two exact opposite patterns can be utilized to prevent interference. Embodiments of surface mapping and a surgical system including a light source and a projector for projecting a light pattern are further described in U.S. Pat. Pub. No. 2017/0055819 entitled “Set Comprising A Surgical Instrument” published Mar. 2, 2017, U.S. Pat. Pub. No. 2017/0251900 entitled “Depiction System” published Sep. 7, 2017, and U.S. patent application Ser. No. 16/729,751 entitled “Surgical Systems For Generating Three Dimensional Constructs Of Anatomical Organs And Coupling Identified Anatomical Structures Thereto” filed Dec. 30, 2019, which are hereby incorporated by reference in their entireties.
The distance determining system can be incorporated into the surface mapping system. For example, structured light can be utilized to generate a three-dimensional (3D) virtual model of the visible surface 105 and determine various distances with respect to the visible surface 105. Additionally or alternatively, the distance determining system can rely on time-of-flight measurements to determine one or more distances to the identified tissue (or other structures) at the surgical site.
The surgical visualization system 100 also includes a surgical device 102. The surgical device 102 can be any suitable surgical device. Examples of the surgical device 102 includes a surgical dissector, a surgical stapler, a surgical grasper, a clip applier, a smoke evacuator, a surgical energy device (e.g., mono-polar probes, bi-polar probes, ablation probes, an ultrasound device, an ultrasonic end effector, etc.), etc. In some embodiments, the surgical device 102 includes an end effector having opposing jaws that extend from a distal end of a shaft of the surgical device 102 and that are configured to engage tissue therebetween.
The surgical visualization system 100 can be configured to identify the critical structure 101 and a proximity of the surgical device 102 to the critical structure 101. The imaging device 120 of the surgical visualization system 100 is configured to detect light at various wavelengths, such as visible light, spectral light waves (visible or invisible), and a structured light pattern (visible or invisible). The imaging device 120 can include a plurality of lenses, sensors, and/or receivers for detecting the different signals. For example, the imaging device 120 can be a hyperspectral, multispectral, or selective spectral camera, as described herein. The imaging device 120 can include a waveform sensor 122 (such as a spectral image sensor, detector, and/or three-dimensional camera lens). For example, the imaging device 120 can include a right-side lens and a left-side lens used together to record two two-dimensional images at the same time and, thus, generate a three-dimensional (3D) image of the surgical site, render a three-dimensional image of the surgical site, and/or determine one or more distances at the surgical site. Additionally or alternatively, the imaging device 120 can be configured to receive images indicative of the topography of the visible tissue and the identification and position of hidden critical structures, as further described herein. For example, a field of view of the imaging device 120 can overlap with a pattern of light (structured light) on the surface 105 of the tissue 103, as shown in FIG. 1 .
As in this illustrated embodiment, the surgical visualization system 100 can be incorporated into a robotic surgical system 110. The robotic surgical system 110 can have a variety of configurations, as discussed herein. In this illustrated embodiment, the robotic surgical system 110 includes a first robotic arm 112 and a second robotic arm 114. The robotic arms 112, 114 each include rigid structural members 116 and joints 118, which can include servomotor controls. The first robotic arm 112 is configured to maneuver the surgical device 102, and the second robotic arm 114 is configured to maneuver the imaging device 120. A robotic control unit of the robotic surgical system 110 is configured to issue control motions to the first and second robotic arms 112, 114, which can affect the surgical device 102 and the imaging device 120, respectively.
In some embodiments, one or more of the robotic arms 112, 114 can be separate from the main robotic system 110 used in the surgical procedure. For example, at least one of the robotic arms 112, 114 can be positioned and registered to a particular coordinate system without a servomotor control. For example, a closed-loop control system and/or a plurality of sensors for the robotic arms 112, 114 can control and/or register the position of the robotic arm(s) 112, 114 relative to the particular coordinate system. Similarly, the position of the surgical device 102 and the imaging device 120 can be registered relative to a particular coordinate system.
Examples of robotic surgical systems include the Ottava™ robotic-assisted surgery system (Johnson & Johnson of New Brunswick, N.J.), da Vinci® surgical systems (Intuitive Surgical, Inc. of Sunnyvale, Calif.), the Hugo™ robotic-assisted surgery system (Medtronic PLC of Minneapolis, Minn.), the Versius® surgical robotic system (CMR Surgical Ltd of Cambridge, UK), and the Monarch© platform (Auris Health, Inc. of Redwood City, Calif.). Embodiments of various robotic surgical systems and using robotic surgical systems are further described in U.S. Pat. Pub. No. 2018/0177556 entitled “Flexible Instrument Insertion Using An Adaptive Force Threshold” filed Dec. 28, 2016, U.S. Pat. Pub. No. 2020/0000530 entitled “Systems And Techniques For Providing Multiple Perspectives During Medical Procedures” filed Apr. 16, 2019, U.S. Pat. Pub. No. 2020/0170720 entitled “Image-Based Branch Detection And Mapping For Navigation” filed Feb. 7, 2020, U.S. Pat. Pub. No. 2020/0188043 entitled “Surgical Robotics System” filed Dec. 9, 2019, U.S. Pat. Pub. No. 2020/0085516 entitled “Systems And Methods For Concomitant Medical Procedures” filed Sep. 3, 2019, U.S. Pat. No. 8,831,782 entitled “Patient-Side Surgeon Interface For A Teleoperated Surgical Instrument” filed Jul. 15, 2013, and Intl. Pat. Pub. No. WO 2014151621 entitled “Hyperdexterous Surgical System” filed Mar. 13, 2014, which are hereby incorporated by reference in their entireties.
The surgical visualization system 100 also includes an emitter 106. The emitter 106 is configured to emit a pattern of light, such as stripes, grid lines, and/or dots, to enable the determination of the topography or landscape of the surface 105. For example, projected light arrays 130 can be used for three-dimensional scanning and registration on the surface 105. The projected light arrays 130 can be emitted from the emitter 106 located on the surgical device 102 and/or one of the robotic arms 112, 114 and/or the imaging device 120. In one aspect, the projected light array 130 is employed by the surgical visualization system 100 to determine the shape defined by the surface 105 of the tissue 103 and/or motion of the surface 105 intraoperatively. The imaging device 120 is configured to detect the projected light arrays 130 reflected from the surface 105 to determine the topography of the surface 105 and various distances with respect to the surface 105.
As in this illustrated embodiment, the imaging device 120 can include an optical waveform emitter 123, such as by being mounted on or otherwise attached on the imaging device 120. The optical waveform emitter 123 is configured to emit electromagnetic radiation 124 (near-infrared (NIR) photons) that can penetrate the surface 105 of the tissue 103 and reach the critical structure 101. The imaging device 120 and the optical waveform emitter 123 can be positionable by the robotic arm 114. The optical waveform emitter 123 is mounted on or otherwise on the imaging device 122 but in other embodiments can be positioned on a separate surgical device from the imaging device 120. A corresponding waveform sensor 122 (e.g., an image sensor, spectrometer, or vibrational sensor) of the imaging device 120 is configured to detect the effect of the electromagnetic radiation received by the waveform sensor 122. The wavelengths of the electromagnetic radiation 124 emitted by the optical waveform emitter 123 are configured to enable the identification of the type of anatomical and/or physical structure, such as the critical structure 101. The identification of the critical structure 101 can be accomplished through spectral analysis, photo-acoustics, and/or ultrasound, for example. In one aspect, the wavelengths of the electromagnetic radiation 124 can be variable. The waveform sensor 122 and optical waveform emitter 123 can be inclusive of a multispectral imaging system and/or a selective spectral imaging system, for example. In other instances, the waveform sensor 122 and optical waveform emitter 123 can be inclusive of a photoacoustic imaging system, for example.
The distance sensor system 104 of the surgical visualization system 100 is configured to determine one or more distances at the surgical site. The distance sensor system 104 can be a time-of-flight distance sensor system that includes an emitter, such as the emitter 106 as in this illustrated embodiment, and that includes a receiver 108. In other instances, the time-of-flight emitter can be separate from the structured light emitter. The emitter 106 can include a very tiny laser source, and the receiver 108 can include a matching sensor. The distance sensor system 104 is configured to detect the “time of flight,” or how long the laser light emitted by the emitter 106 has taken to bounce back to the sensor portion of the receiver 108. Use of a very narrow light source in the emitter 106 enables the distance sensor system 104 to determining the distance to the surface 105 of the tissue 103 directly in front of the distance sensor system 104.
The receiver 108 of the distance sensor system 104 is positioned on the surgical device 102 in this illustrated embodiment, but in other embodiments the receiver 108 can be mounted on a separate surgical device instead of the surgical device 102. For example, the receiver 108 can be mounted on a cannula or trocar through which the surgical device 102 extends to reach the surgical site. In still other embodiments, the receiver 108 for the distance sensor system 104 can be mounted on a separate robotically-controlled arm of the robotic system 110 (e.g., on the second robotic arm 114) than the first robotic arm 112 to which the surgical device 102 is coupled, can be mounted on a movable arm that is operated by another robot, or be mounted to an operating room (OR) table or fixture. In some embodiments, the imaging device 120 includes the receiver 108 to allow for determining the distance from the emitter 106 to the surface 105 of the tissue 103 using a line between the emitter 106 on the surgical device 102 and the imaging device 120. For example, the distance d_ecan be triangulated based on known positions of the emitter 106 (on the surgical device 102) and the receiver 108 (on the imaging device 120) of the distance sensor system 104. The three-dimensional position of the receiver 108 can be known and/or registered to the robot coordinate plane intraoperatively.
As in this illustrated embodiment, the position of the emitter 106 of the distance sensor system 104 can be controlled by the first robotic arm 112, and the position of the receiver 108 of the distance sensor system 104 can be controlled by the second robotic arm 114. In other embodiments, the surgical visualization system 100 can be utilized apart from a robotic system. In such instances, the distance sensor system 104 can be independent of the robotic system.
In FIG. 1 , d_eis emitter-to-tissue distance from the emitter 106 to the surface 105 of the tissue 103, and d_tis device-to-tissue distance from a distal end of the surgical device 102 to the surface 105 of the tissue 103. The distance sensor system 104 is configured to determine the emitter-to-tissue distance d_e. The device-to-tissue distance d_tis obtainable from the known position of the emitter 106 on the surgical device 102, e.g., on a shaft thereof proximal to the surgical device's distal end, relative to the distal end of the surgical device 102. In other words, when the distance between the emitter 106 and the distal end of the surgical device 102 is known, the device-to-tissue distance d_tcan be determined from the emitter-to-tissue distance d_e. In some embodiments, the shaft of the surgical device 102 can include one or more articulation joints and can be articulatable with respect to the emitter 106 and jaws at the distal end of the surgical device 102. The articulation configuration can include a multi-joint vertebrae-like structure, for example. In some embodiments, a three-dimensional camera can be utilized to triangulate one or more distances to the surface 105.
In FIG. 1 , d_wis camera-to-critical structure distance from the optical waveform emitter 123 located on the imaging device 120 to the surface of the critical structure 101, and d_Ais a depth of the critical structure 101 below the surface 105 of the tissue 103 (e.g., the distance between the portion of the surface 105 closest to the surgical device 102 and the critical structure 101). The time-of-flight of the optical waveforms emitted from the optical waveform emitter 123 located on the imaging device 120 are configured to determine the camera-to-critical structure distance d_w.
As shown in FIG. 2 , the depth d_Aof the critical structure 101 relative to the surface 105 of the tissue 103 can be determined by triangulating from the distance d_wand known positions of the emitter 106 on the surgical device 102 and the optical waveform emitter 123 on the imaging device 120 (and, thus, the known distance dx therebetween) to determine the distance d_y, which is the sum of the distances d_eand d_A. Additionally or alternatively, time-of-flight from the optical waveform emitter 123 can be configured to determine the distance from the optical waveform emitter 123 to the surface 105 of the tissue 103. For example, a first waveform (or range of waveforms) can be utilized to determine the camera-to-critical structure distance d_wand a second waveform (or range of waveforms) can be utilized to determine the distance to the surface 105 of the tissue 103. In such instances, the different waveforms can be utilized to determine the depth of the critical structure 101 below the surface 105 of the tissue 103.
Additionally or alternatively, the distance d_Acan be determined from an ultrasound, a registered magnetic resonance imaging (MRI), or computerized tomography (CT) scan. In still other instances, the distance d_Acan be determined with spectral imaging because the detection signal received by the imaging device 120 can vary based on the type of material, e.g., type of the tissue 103. For example, fat can decrease the detection signal in a first way, or a first amount, and collagen can decrease the detection signal in a different, second way, or a second amount.
In another embodiment of a surgical visualization system 160 illustrated in FIG. 3 , a surgical device 162, and not the imaging device 120, includes the optical waveform emitter 123 and the waveform sensor 122 that is configured to detect the reflected waveforms. The optical waveform emitter 123 is configured to emit waveforms for determining the distances d_tand d_wfrom a common device, such as the surgical device 162, as described herein. In such instances, the distance d_Afrom the surface 105 of the tissue 103 to the surface of the critical structure 101 can be determined as follows:
d _A =d _w −d _t
The surgical visualization system 100 includes a control system configured to control various aspects of the surgical visualization system 100. FIG. 4 illustrates one embodiment of a control system 133 that can be utilized as the control system of the surgical visualization system 100 (or other surgical visualization system described herein). The control system 133 includes a control circuit 132 configured to be in signal communication with a memory 134. The memory 134 is configured to store instructions executable by the control circuit 132, such as instructions to determine and/or recognize critical structures (e.g., the critical structure 101 of FIG. 1 ), instructions to determine and/or compute one or more distances and/or three-dimensional digital representations, and instructions to communicate information to a medical practitioner. As in this illustrated embodiment, the memory 134 can store surface mapping logic 136, imaging logic 138, tissue identification logic 140, and distance determining logic 141, although the memory 134 can store any combinations of the logics 136, 138, 140, 141 and/or can combine various logics together. The control system 133 also includes an imaging system 142 including a camera 144 (e.g., the imaging system including the imaging device 120 of FIG. 1 ), a display 146 (e.g., a monitor, a computer tablet screen, etc.), and controls 148 of the camera 144 and the display 146. The camera 144 includes an image sensor 135 (e.g., the waveform sensor 122) configured to receive signals from various light sources emitting light at various visible and invisible spectra (e.g., visible light, spectral imagers, three-dimensional lens, etc.). The display 146 is configured to depict real, virtual, and/or virtually-augmented images and/or information to a medical practitioner.
In an exemplary embodiment, the image sensor 135 is a solid-state electronic device containing up to millions of discrete photodetector sites called pixels. The image sensor 135 technology falls into one of two categories: Charge-Coupled Device (CCD) and Complementary Metal Oxide Semiconductor (CMOS) imagers and more recently, short-wave infrared (SWIR) is an emerging technology in imaging. Another type of the image sensor 135 employs a hybrid CCD/CMOS architecture (sold under the name “sCMOS”) and consists of CMOS readout integrated circuits (ROICs) that are bump bonded to a CCD imaging substrate. CCD and CMOS image sensors 135 are sensitive to wavelengths in a range of about 350 nm to about 1050 nm, such as in a range of about 400 nm to about 1000 nm. A person skilled in the art will appreciate that a value may not be precisely at a value but nevertheless considered to be about that value for any of a variety of reasons, such as sensitivity of measurement equipment and manufacturing tolerances. CMOS sensors are, in general, more sensitive to IR wavelengths than CCD sensors. Solid state image sensors 135 are based on the photoelectric effect and, as a result, cannot distinguish between colors. Accordingly, there are two types of color CCD cameras: single chip and three-chip. Single chip color CCD cameras offer a common, low-cost imaging solution and use a mosaic (e.g., Bayer) optical filter to separate incoming light into a series of colors and employ an interpolation algorithm to resolve full color images. Each color is, then, directed to a different set of pixels. Three-chip color CCD cameras provide higher resolution by employing a prism to direct each section of the incident spectrum to a different chip. More accurate color reproduction is possible, as each point in space of the object has separate RGB intensity values, rather than using an algorithm to determine the color. Three-chip cameras offer extremely high resolutions.
The control system 133 also includes an emitter (e.g., the emitter 106) including a spectral light source 150 and a structured light source 152 each operably coupled to the control circuit 133. A single source can be pulsed to emit wavelengths of light in the spectral light source 150 range and wavelengths of light in the structured light source 152 range. Alternatively, a single light source can be pulsed to provide light in the invisible spectrum (e.g., infrared spectral light) and wavelengths of light on the visible spectrum. The spectral light source 150 can be, for example, a hyperspectral light source, a multispectral light source, and/or a selective spectral light source. The tissue identification logic 140 is configured to identify critical structure(s) (e.g., the critical structure 101 of FIG. 1 ) via data from the spectral light source 150 received by the image sensor 135 of the camera 144. The surface mapping logic 136 is configured to determine the surface contours of the visible tissue (e.g., the tissue 103) based on reflected structured light. With time-of-flight measurements, the distance determining logic 141 is configured to determine one or more distance(s) to the visible tissue and/or the critical structure. Output from each of the surface mapping logic 136, the tissue identification logic 140, and the distance determining logic 141 is configured to be provided to the imaging logic 138, and combined, blended, and/or overlaid by the imaging logic 138 to be conveyed to a medical practitioner via the display 146 of the imaging system 142.
The control circuit 132 can have a variety of configurations. FIG. 5 illustrates one embodiment of a control circuit 170 that can be used as the control circuit 132 configured to control aspects of the surgical visualization system 100. The control circuit 170 is configured to implement various processes described herein. The control circuit 170 includes a microcontroller that includes a processor 172 (e.g., a microprocessor or microcontroller) operably coupled to a memory 174. The memory 174 is configured to store machine-executable instructions that, when executed by the processor 172, cause the processor 172 to execute machine instructions to implement various processes described herein. The processor 172 can be any one of a number of single-core or multicore processors known in the art. The memory 174 can include volatile and non-volatile storage media. The processor 172 includes an instruction processing unit 176 and an arithmetic unit 178. The instruction processing unit 176 is configured to receive instructions from the memory 174.
The surface mapping logic 136, the imaging logic 138, the tissue identification logic 140, and the distance determining logic 141 can have a variety of configurations. FIG. 6 illustrates one embodiment of a combinational logic circuit 180 configured to control aspects of the surgical visualization system 100 using logic such as one or more of the surface mapping logic 136, the imaging logic 138, the tissue identification logic 140, and the distance determining logic 141. The combinational logic circuit 180 includes a finite state machine that includes a combinational logic 182 configured to receive data associated with a surgical device (e.g. the surgical device 102 and/or the imaging device 120) at an input 184, process the data by the combinational logic 182, and provide an output 184 to a control circuit (e.g., the control circuit 132).
FIG. 7 illustrates one embodiment of a sequential logic circuit 190 configured to control aspects of the surgical visualization system 100 using logic such as one or more of the surface mapping logic 136, the imaging logic 138, the tissue identification logic 140, and the distance determining logic 141. The sequential logic circuit 190 includes a finite state machine that includes a combinational logic 192, a memory 194, and a clock 196. The memory 194 is configured to store a current state of the finite state machine. The sequential logic circuit 190 can be synchronous or asynchronous. The combinational logic 192 is configured to receive data associated with a surgical device (e.g. the surgical device 102 and/or the imaging device 120) at an input 426, process the data by the combinational logic 192, and provide an output 499 to a control circuit (e.g., the control circuit 132). In some embodiments, the sequential logic circuit 190 can include a combination of a processor (e.g., processor 172 of FIG. 5 ) and a finite state machine to implement various processes herein. In some embodiments, the finite state machine can include a combination of a combinational logic circuit (e.g., the combinational logic circuit 192 of FIG. 7 ) and the sequential logic circuit 190.
FIG. 8 illustrates another embodiment of a surgical visualization system 200. The surgical visualization system 200 is generally configured and used similar to the surgical visualization system 100 of FIG. 1 , e.g., includes a surgical device 202 and an imaging device 220. The imaging device 220 includes a spectral light emitter 223 configured to emit spectral light in a plurality of wavelengths to obtain a spectral image of hidden structures, for example. The imaging device 220 can also include a three-dimensional camera and associated electronic processing circuits. The surgical visualization system 200 is shown being utilized intraoperatively to identify and facilitate avoidance of certain critical structures, such as a ureter 201 a and vessels 201 b, in an organ 203 (a uterus in this embodiment) that are not visible on a surface 205 of the organ 203.
The surgical visualization system 200 is configured to determine an emitter-to-tissue distance d_efrom an emitter 206 on the surgical device 202 to the surface 205 of the uterus 203 via structured light. The surgical visualization system 200 is configured to extrapolate a device-to-tissue distance d_tfrom the surgical device 202 to the surface 205 of the uterus 203 based on the emitter-to-tissue distance d_e. The surgical visualization system 200 is also configured to determine a tissue-to-ureter distance d_Afrom the ureter 201 a to the surface 205 and a camera-to ureter distance d_wfrom the imaging device 220 to the ureter 201 a. As described herein, e.g., with respect to the surgical visualization system 100 of FIG. 1 , the surgical visualization system 200 is configured to determine the distance d_wwith spectral imaging and time-of-flight sensors, for example. In various embodiments, the surgical visualization system 200 can determine (e.g., triangulate) the tissue-to-ureter distance d_A(or depth) based on other distances and/or the surface mapping logic described herein.
As mentioned above, a surgical visualization system includes a control system configured to control various aspects of the surgical visualization system. The control system can have a variety of configurations. FIG. 9 illustrates one embodiment of a control system 600 for a surgical visualization system, such as the surgical visualization system 100 of FIG. 1 , the surgical visualization system 200 of FIG. 8 , or other surgical visualization system described herein. The control system 600 is a conversion system that integrates spectral signature tissue identification and structured light tissue positioning to identify a critical structure, especially when those structure(s) are obscured by tissue, e.g., by fat, connective tissue, blood tissue, and/or organ(s), and/or by blood, and/or to detect tissue variability, such as differentiating tumors and/or non-healthy tissue from healthy tissue within an organ.
The control system 600 is configured for implementing a hyperspectral imaging and visualization system in which a molecular response is utilized to detect and identify anatomy in a surgical field of view. The control system 600 includes a conversion logic circuit 648 configured to convert tissue data to usable information for surgeons and/or other medical practitioners. For example, variable reflectance based on wavelengths with respect to obscuring material can be utilized to identify the critical structure in the anatomy. Moreover, the control system 600 is configured to combine the identified spectral signature and the structural light data in an image. For example, the control system 600 can be employed to create of three-dimensional data set for surgical use in a system with augmentation image overlays. Techniques can be employed both intraoperatively and preoperatively using additional visual information. In various embodiments, the control system 600 is configured to provide warnings to a medical practitioner when in the proximity of one or more critical structures. Various algorithms can be employed to guide robotic automation and semi-automated approaches based on the surgical procedure and proximity to the critical structure(s).
A projected array of lights is employed by the control system 600 to determine tissue shape and motion intraoperatively. Alternatively, flash Lidar may be utilized for surface mapping of the tissue.
The control system 600 is configured to detect the critical structure, which as mentioned above can include one or more critical structures, and provide an image overlay of the critical structure and measure the distance to the surface of the visible tissue and the distance to the embedded/buried critical structure(s). The control system 600 can measure the distance to the surface of the visible tissue or detect the critical structure and provide an image overlay of the critical structure.
The control system 600 includes a spectral control circuit 602. The spectral control circuit 602 can be a field programmable gate array (FPGA) or another suitable circuit configuration, such as the configurations described with respect to FIG. 6 , FIG. 7 , and FIG. 8 . The spectral control circuit 602 includes a processor 604 configured to receive video input signals from a video input processor 606. The processor 604 can be configured for hyperspectral processing and can utilize C/C++ code, for example. The video input processor 606 is configured to receive video-in of control (metadata) data such as shutter time, wave length, and sensor analytics, for example. The processor 604 is configured to process the video input signal from the video input processor 606 and provide a video output signal to a video output processor 608, which includes a hyperspectral video-out of interface control (metadata) data, for example. The video output processor 608 is configured to provides the video output signal to an image overlay controller 610.
The video input processor 606 is operatively coupled to a camera 612 at the patient side via a patient isolation circuit 614. The camera 612 includes a solid state image sensor 634. The patient isolation circuit 614 can include a plurality of transformers so that the patient is isolated from other circuits in the system. The camera 612 is configured to receive intraoperative images through optics 632 and the image sensor 634. The image sensor 634 can include a CMOS image sensor, for example, or can include another image sensor technology, such as those discussed herein in connection with FIG. 4 . The camera 612 is configured to output 613 images in 14 bit/pixel signals. A person skilled in the art will appreciate that higher or lower pixel resolutions can be employed. The isolated camera output signal 613 is provided to a color RGB fusion circuit 616, which in this illustrated embodiment employs a hardware register 618 and a Nios2 co-processor 620 configured to process the camera output signal 613. A color RGB fusion output signal is provided to the video input processor 606 and a laser pulsing control circuit 622.
The laser pulsing control circuit 622 is configured to control a laser light engine 624. The laser light engine 624 is configured to output light in a plurality of wavelengths (λ1, λ2, λ3 . . . λn) including near infrared (NIR). The laser light engine 624 can operate in a plurality of modes. For example, the laser light engine 624 can operate in two modes. In a first mode, e.g., a normal operating mode, the laser light engine 624 is configured to output an illuminating signal. In a second mode, e.g., an identification mode, the laser light engine 624 is configured to output RGBG and NTR light. In various embodiments, the laser light engine 624 can operate in a polarizing mode.
Light output 626 from the laser light engine 624 is configured to illuminate targeted anatomy in an intraoperative surgical site 627. The laser pulsing control circuit 622 is also configured to control a laser pulse controller 628 for a laser pattern projector 630 configured to project a laser light pattern 631, such as a grid or pattern of lines and/or dots, at a predetermined wavelength (λ2) on an operative tissue or organ at the surgical site 627. The camera 612 is configured to receive the patterned light as well as the reflected light output through the camera optics 632. The image sensor 634 is configured to convert the received light into a digital signal.
The color RGB fusion circuit 616 is also configured to output signals to the image overlay controller 610 and a video input module 636 for reading the laser light pattern 631 projected onto the targeted anatomy at the surgical site 627 by the laser pattern projector 630. A processing module 638 is configured to process the laser light pattern 631 and output a first video output signal 640 representative of the distance to the visible tissue at the surgical site 627. The data is provided to the image overlay controller 610. The processing module 638 is also configured to output a second video signal 642 representative of a three-dimensional rendered shape of the tissue or organ of the targeted anatomy at the surgical site.
The first and second video output signals 640, 642 include data representative of the position of the critical structure on a three-dimensional surface model, which is provided to an integration module 643. In combination with data from the video out processor 608 of the spectral control circuit 602, the integration module 643 is configured to determine the distance (e.g., distance d_Aof FIG. 1 ) to a buried critical structure (e.g., via triangularization algorithms 644), and the distance to the buried critical structure can be provided to the image overlay controller 610 via a video out processor 646. The foregoing conversion logic can encompass the conversion logic circuit 648 intermediate video monitors 652 and the camera 624/laser pattern projector 630 positioned at the surgical site 627.
Preoperative data 650, such as from a CT or MRI scan, can be employed to register or align certain three-dimensional deformable tissue in various instances. Such preoperative data 650 can be provided to the integration module 643 and ultimately to the image overlay controller 610 so that such information can be overlaid with the views from the camera 612 and provided to the video monitors 652. Embodiments of registration of preoperative data are further described in U.S. Pat. Pub. No. 2020/0015907 entitled “Integration Of Imaging Data” filed Sep. 11, 2018, which is hereby incorporated by reference herein in its entirety.
The video monitors 652 are configured to output the integrated/augmented views from the image overlay controller 610. A medical practitioner can select and/or toggle between different views on one or more displays. On a first display 652 a, which is a monitor in this illustrated embodiment, the medical practitioner can toggle between (A) a view in which a three-dimensional rendering of the visible tissue is depicted and (B) an augmented view in which one or more hidden critical structures are depicted over the three-dimensional rendering of the visible tissue. On a second display 652 b, which is a monitor in this illustrated embodiment, the medical practitioner can toggle on distance measurements to one or more hidden critical structures and/or the surface of visible tissue, for example.
The various surgical visualization systems described herein can be utilized to visualize various different types of tissues and/or anatomical structures, including tissues and/or anatomical structures that may be obscured from being visualized by EMR in the visible portion of the spectrum. The surgical visualization system can utilize a spectral imaging system, as mentioned above, which can be configured to visualize different types of tissues based upon their varying combinations of constituent materials. In particular, a spectral imaging system can be configured to detect the presence of various constituent materials within a tissue being visualized based on the absorption coefficient of the tissue across various EMR wavelengths. The spectral imaging system can be configured to characterize the tissue type of the tissue being visualized based upon the particular combination of constituent materials.
FIG. 10 shows a graph 300 depicting how the absorption coefficient of various biological materials varies across the EMR wavelength spectrum. In the graph 300, the vertical axis 302 represents absorption coefficient of the biological material in cm⁻¹, and the horizontal axis 304 represents EMR wavelength in μm. A first line 306 in the graph 300 represents the absorption coefficient of water at various EMR wavelengths, a second line 308 represents the absorption coefficient of protein at various EMR wavelengths, a third line 310 represents the absorption coefficient of melanin at various EMR wavelengths, a fourth line 312 represents the absorption coefficient of deoxygenated hemoglobin at various EMR wavelengths, a fifth line 314 represents the absorption coefficient of oxygenated hemoglobin at various EMR wavelengths, and a sixth line 316 represents the absorption coefficient of collagen at various EMR wavelengths. Different tissue types have different combinations of constituent materials and, therefore, the tissue type(s) being visualized by a surgical visualization system can be identified and differentiated between according to the particular combination of detected constituent materials. Accordingly, a spectral imaging system of a surgical visualization system can be configured to emit EMIR at a number of different wavelengths, determine the constituent materials of the tissue based on the detected absorption EMIR absorption response at the different wavelengths, and then characterize the tissue type based on the particular detected combination of constituent materials.
FIG. 11 shows an embodiment of the utilization of spectral imaging techniques to visualize different tissue types and/or anatomical structures. In FIG. 11 , a spectral emitter 320 (e.g., the spectral light source 150 of FIG. 4 ) is being utilized by an imaging system to visualize a surgical site 322. The EMR emitted by the spectral emitter 320 and reflected from the tissues and/or structures at the surgical site 322 is received by an image sensor (e.g., the image sensor 135 of FIG. 4 ) to visualize the tissues and/or structures, which can be either visible (e.g., be located at a surface of the surgical site 322) or obscured (e.g., underlay other tissue and/or structures at the surgical site 322). In this embodiment, an imaging system (e.g., the imaging system 142 of FIG. 4 ) visualizes a tumor 324, an artery 326, and various abnormalities 328 (e.g., tissues not confirming to known or expected spectral signatures) based upon the spectral signatures characterized by the differing absorptive characteristics (e.g., absorption coefficient) of the constituent materials for each of the different tissue/structure types. The visualized tissues and structures can be displayed on a display screen associated with or coupled to the imaging system (e.g., the display 146 of the imaging system 142 of FIG. 4 ), on a primary display (e.g., the primary display 819 of FIG. 19 ), on a non-sterile display (e.g., the non-sterile displays 807, 809 of FIG. 19 ), on a display of a surgical hub (e.g., the display of the surgical hub 806 of FIG. 19 ), on a device/instrument display, and/or on another display.
The imaging system can be configured to tailor or update the displayed surgical site visualization according to the identified tissue and/or structure types. For example, as shown in FIG. 11 , the imaging system can display a margin 330 associated with the tumor 324 being visualized on a display screen associated with or coupled to the imaging system, on a primary display, on a non-sterile display, on a display of a surgical hub, on a device/instrument display, and/or on another display. The margin 330 can indicate the area or amount of tissue that should be excised to ensure complete removal of the tumor 324. The surgical visualization system's control system (e.g., the control system 133 of FIG. 4 ) can be configured to control or update the dimensions of the margin 330 based on the tissues and/or structures identified by the imaging system. In this illustrated embodiment, the imaging system has identified multiple abnormalities 328 within the field of view (FOV). Accordingly, the control system can adjust the displayed margin 330 to a first updated margin 332 having sufficient dimensions to encompass the abnormalities 328. Further, the imaging system has also identified the artery 326 partially overlapping with the initially displayed margin 330 (as indicated by a highlighted region 334 of the artery 326). Accordingly, the control system can adjust the displayed margin to a second updated margin 336 having sufficient dimensions to encompass the relevant portion of the artery 326.
Tissues and/or structures can also be imaged or characterized according to their reflective characteristics, in addition to or in lieu of their absorptive characteristics described above with respect to FIG. 10 and FIG. 11 , across the EMR wavelength spectrum. For example, FIG. 12 , FIG. 13 , and FIG. 14 illustrate various graphs of reflectance of different types of tissues or structures across different EMR wavelengths. FIG. 12 is a graphical representation 340 of an illustrative ureter signature versus obscurants. FIG. 13 is a graphical representation 342 of an illustrative artery signature versus obscurants. FIG. 14 is a graphical representation 344 of an illustrative nerve signature versus obscurants. The plots in FIG. 12 , FIG. 13 , and FIG. 14 represent reflectance as a function of wavelength (nm) for the particular structures (ureter, artery, and nerve) relative to the corresponding reflectances of fat, lung tissue, and blood at the corresponding wavelengths. These graphs are simply for illustrative purposes and it should be understood that other tissues and/or structures could have corresponding detectable reflectance signatures that would allow the tissues and/or structures to be identified and visualized.
Select wavelengths for spectral imaging can be identified and utilized based on the anticipated critical structures and/or obscurants at a surgical site (e.g., “selective spectral” imaging). By utilizing selective spectral imaging, the amount of time required to obtain the spectral image can be minimized such that the information can be obtained in real-time and utilized intraoperatively. The wavelengths can be selected by a medical practitioner or by a control circuit based on input by a user, e.g., a medical practitioner. In certain instances, the wavelengths can be selected based on machine learning and/or big data accessible to the control circuit via, e.g., a cloud or surgical hub.
FIG. 15 illustrates one embodiment of spectral imaging to tissue being utilized intraoperatively to measure a distance between a waveform emitter and a critical structure that is obscured by tissue. FIG. 15 shows an embodiment of a time-of-flight sensor system 404 utilizing waveforms 424, 425. The time-of-flight sensor system 404 can be incorporated into a surgical visualization system, e.g., as the sensor system 104 of the surgical visualization system 100 of FIG. 1 . The time-of-flight sensor system 404 includes a waveform emitter 406 and a waveform receiver 408 on the same surgical device 402 (e.g., the emitter 106 and the receiver 108 on the same surgical device 102 of FIG. 1 ). The emitted wave 400 extends to a critical structure 401 (e.g., the critical structure 101 of FIG. 1 ) from the emitter 406, and the received wave 425 is reflected back to by the receiver 408 from the critical structure 401. The surgical device 402 in this illustrated embodiment is positioned through a trocar 410 that extends into a cavity 407 in a patient. Although the trocar 410 is used in this in this illustrated embodiment, other trocars or other access devices can be used, or no access device may be used.
The waveforms 424, 425 are configured to penetrate obscuring tissue 403, such as by having wavelengths in the NIR or SWIR spectrum of wavelengths. A spectral signal (e.g., hyperspectral, multispectral, or selective spectral) or a photoacoustic signal is emitted from the emitter 406, as shown by a first arrow 407 pointing distally, and can penetrate the tissue 403 in which the critical structure 401 is concealed. The emitted waveform 424 is reflected by the critical structure 401, as shown by a second arrow 409 pointing proximally. The received waveform 425 can be delayed due to a distance d between a distal end of the surgical device 402 and the critical structure 401. The waveforms 424, 425 can be selected to target the critical structure 401 within the tissue 403 based on the spectral signature of the critical structure 401, as described herein. The emitter 406 is configured to provide a binary signal on and off, as shown in FIG. 16 , for example, which can be measured by the receiver 408.
Based on the delay between the emitted wave 424 and the received wave 425, the time-of-flight sensor system 404 is configured to determine the distance d. A time-of-flight timing diagram 430 for the emitter 406 and the receiver 408 of FIG. 15 is shown in FIG. 16 . The delay is a function of the distance d and the distance d is given by:
$d = \frac{ct}{2} \cdot \frac{q_{2}}{q_{1} + q_{2}}$
where c=the speed of light; t=length of pulse; q1=accumulated charge while light is emitted; and q2=accumulated charge while light is not being emitted.
The time-of-flight of the waveforms 424, 425 corresponds to the distance d in FIG. 15 . In various instances, additional emitters/receivers and/or pulsing signals from the emitter 406 can be configured to emit a non-penetrating signal. The non-penetrating signal can be configured to determine the distance from the emitter 406 to the surface 405 of the obscuring tissue 403. In various instances, a depth of the critical structure 401 can be determined by:
d _A =d _w −d _t
where d_A=the depth of the critical structure 401; d_w=the distance from the emitter 406 to the critical structure 401 (d in FIG. 15 ); and d_t,=the distance from the emitter 406 (on the distal end of the surgical device 402) to the surface 405 of the obscuring tissue 403.
FIG. 17 illustrates another embodiment of a time-of-flight sensor system 504 utilizing waves 524 a, 524 b, 524 c, 525 a, 525 b, 525 c is shown. The time-of-flight sensor system 504 can be incorporated into a surgical visualization system, e.g., as the sensor system 104 of the surgical visualization system 100 of FIG. 1 . The time-of-flight sensor system 504 includes a waveform emitter 506 and a waveform receiver 508 (e.g., the emitter 106 and the receiver 108 of FIG. 1 ). The waveform emitter 506 is positioned on a first surgical device 502 a (e.g., the surgical device 102 of FIG. 1 ), and the waveform receiver 508 is positioned on a second surgical device 502 b. The surgical devices 502 a, 502 b are positioned through first and second trocars 510 a, 510 b, respectively, which extend into a cavity 507 in a patient. Although the trocars 510 a, 510 b are used in this in this illustrated embodiment, other trocars or other access devices can be used, or no access device may be used. The emitted waves 524 a, 524 b, 524 c extend toward a surgical site from the emitter 506, and the received waves 525 a, 525 b, 525 c are reflected back to the receiver 508 from various structures and/or surfaces at the surgical site.
The different emitted waves 524 a, 524 b, 524 c are configured to target different types of material at the surgical site. For example, the wave 524 a targets obscuring tissue 503, the wave 524 b targets a first critical structure 501 a (e.g., the critical structure 101 of FIG. 1 ), which is a vessel in this illustrated embodiment, and the wave 524 c targets a second critical structure 501 b (e.g., the critical structure 101 of FIG. 1 ), which is a cancerous tumor in this illustrated embodiment. The wavelengths of the waves 524 a, 524 b, 524 c can be in the visible light, NIR, or SWIR spectrum of wavelengths. For example, visible light can be reflected off a surface 505 of the tissue 503, and NIR and/or SWIR waveforms can penetrate the surface 505 of the tissue 503. In various aspects, as described herein, a spectral signal (e.g., hyperspectral, multispectral, or selective spectral) or a photoacoustic signal can be emitted from the emitter 506. The waves 524 b, 524 c can be selected to target the critical structures 501 a, 501 b within the tissue 503 based on the spectral signature of the critical structure 501 a, 501 b, as described herein. Photoacoustic imaging is further described in various U.S. patent applications, which are incorporated by reference herein in the present disclosure.
The emitted waves 524 a, 524 b, 524 c are reflected off the targeted material, namely the surface 505, the first critical structure 501 a, and the second structure 501 b, respectively. The received waveforms 525 a, 525 b, 525 c can be delayed due to distances d_1a, d_2a, d_3a, d_1b, d_2b, d_2c.
In the time-of-flight sensor system 504, in which the emitter 506 and the receiver 508 are independently positionable (e.g., on separate surgical devices 502 a, 502 b and/or controlled by separate robotic arms), the various distances d_1a, d_2a, d_3a, d_1b, d_2b, d_2ccan be calculated from the known position of the emitter 506 and the receiver 508. For example, the positions can be known when the surgical devices 502 a, 502 b are robotically-controlled. Knowledge of the positions of the emitter 506 and the receiver 508, as well as the time of the photon stream to target a certain tissue and the information received by the receiver 508 of that particular response can allow a determination of the distances d_1a, d_2a, d_3a, d_1b, d_2b, d_2c. In one aspect, the distance to the obscured critical structures 501 a, 501 b can be triangulated using penetrating wavelengths. Because the speed of light is constant for any wavelength of visible or invisible light, the time-of-flight sensor system 504 can determine the various distances.
In a view provided to the medical practitioner, such as on a display, the receiver 508 can be rotated such that a center of mass of the target structure in the resulting images remains constant, e.g., in a plane perpendicular to an axis of a select target structure 503, 501 a, or 501 b. Such an orientation can quickly communicate one or more relevant distances and/or perspectives with respect to the target structure. For example, as shown in FIG. 17 , the surgical site is displayed from a viewpoint in which the critical structure 501 a is perpendicular to the viewing plane (e.g., the vessel is oriented in/out of the page). Such an orientation can be default setting; however, the view can be rotated or otherwise adjusted by a medical practitioner. In certain instances, the medical practitioner can toggle between different surfaces and/or target structures that define the viewpoint of the surgical site provided by the imaging system.
As in this illustrated embodiment, the receiver 508 can be mounted on the trocar 510 b (or other access device) through which the surgical device 502 b is positioned. In other embodiments, the receiver 508 can be mounted on a separate robotic arm for which the three-dimensional position is known. In various instances, the receiver 508 can be mounted on a movable arm that is separate from a robotic surgical system that controls the surgical device 502 a or can be mounted to an operating room (OR) table or fixture that is intraoperatively registerable to the robot coordinate plane. In such instances, the position of the emitter 506 and the receiver 508 can be registerable to the same coordinate plane such that the distances can be triangulated from outputs from the time-of-flight sensor system 504.
Combining time-of-flight sensor systems and near-infrared spectroscopy (NIRS), termed TOF-NIRS, which is capable of measuring the time-resolved profiles of NIR light with nanosecond resolution can be found in “Time-Of-Flight Near-Infrared Spectroscopy For Nondestructive Measurement Of Internal Quality In Grapefruit,” Journal of the American Society for Horticultural Science, May 2013 vol. 138 no. 3 225-228, which is hereby incorporated by reference in its entirety.
Embodiments of visualization systems and aspects and uses thereof are described further in U.S. Pat. Pub. No. 2020/0015923 entitled “Surgical Visualization Platform” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015900 entitled “Controlling An Emitter Assembly Pulse Sequence” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015668 entitled “Singular EMR Source Emitter Assembly” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015925 entitled “Combination Emitter And Camera Assembly” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/00015899 entitled “Surgical Visualization With Proximity Tracking Features” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/00015903 entitled “Surgical Visualization Of Multiple Targets” filed Sep. 11, 2018, U.S. Pat. No. 10,792,034 entitled “Visualization Of Surgical Devices” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015897 entitled “Operative Communication Of Light” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015924 entitled “Robotic Light Projection Tools” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015898 entitled “Surgical Visualization Feedback System” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015906 entitled “Surgical Visualization And Monitoring” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015907 entitled “Integration Of Imaging Data” filed Sep. 11, 2018, U.S. Pat. No. 10,925,598 entitled “Robotically-Assisted Surgical Suturing Systems” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015901 entitled “Safety Logic For Surgical Suturing Systems” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015914 entitled “Robotic Systems With Separate Photoacoustic Receivers” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015902 entitled “Force Sensor Through Structured Light Deflection” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2019/0201136 entitled “Method Of Hub Communication” filed Dec. 4, 2018, U.S. patent application Ser. No. 16/729,772 entitled “Analyzing Surgical Trends By A Surgical System” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,747 entitled “Dynamic Surgical Visualization Systems” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,744 entitled “Visualization Systems Using Structured Light” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,778 entitled “System And Method For Determining, Adjusting, And Managing Resection Margin About A Subject Tissue” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,729 entitled “Surgical Systems For Proposing And Corroborating Organ Portion Removals” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,778 entitled “Surgical System For Overlaying Surgical Instrument Data Onto A Virtual Three Dimensional Construct Of An Organ” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,751 entitled “Surgical Systems For Generating Three Dimensional Constructs Of Anatomical Organs And Coupling Identified Anatomical Structures Thereto” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,740 entitled “Surgical Systems Correlating Visualization Data And Powered Surgical Instrument Data” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,737 entitled “Adaptive Surgical System Control According To Surgical Smoke Cloud Characteristics” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,796 entitled “Adaptive Surgical System Control According To Surgical Smoke Particulate Characteristics” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,803 entitled “Adaptive Visualization By A Surgical System” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,807 entitled “Method Of Using Imaging Devices In Surgery” filed Dec. 30, 2019, U.S. Prov. Pat. App. No. 63/249,652 entitled “Surgical Devices, Systems, and Methods Using Fiducial Identification and Tracking” filed on Sep. 29, 2021, U.S. Prov. Pat. App. No. 63/249,658 entitled “Surgical Devices, Systems, and Methods for Control of One Visualization with Another” filed on Sep. 29, 2021, U.S. Prov. Pat. App. No. 63/249,870 entitled “Methods and Systems for Controlling Cooperative Surgical Instruments” filed on Sep. 29, 2021, U.S. Prov. Pat. App. No. 63/249,881 entitled “Methods and Systems for Controlling Cooperative Surgical Instruments with Variable Surgical Site Access Trajectories” filed on Sep. 29, 2021, U.S. Prov. Pat. App. No. 63/249,877 entitled “Methods and Systems for Controlling Cooperative Surgical Instruments” filed on Sep. 29, 2021, and U.S. Prov. Pat. App. No. 63/249,980 entitled “Cooperative Access” filed on Sep. 29, 2021, which are hereby incorporated by reference in their entireties.
Surgical Hubs
The various visualization or imaging systems described herein can be incorporated into a system that includes a surgical hub. In general, a surgical hub can be a component of a comprehensive digital medical system capable of spanning multiple medical facilities and configured to provide integrated and comprehensive improved medical care to a vast number of patients. The comprehensive digital medical system includes a cloud-based medical analytics system that is configured to interconnect to multiple surgical hubs located across many different medical facilities. The surgical hubs are configured to interconnect with one or more elements, such as one or more surgical instruments that are used to conduct medical procedures on patients and/or one or more visualization systems that are used during performance of medical procedures. The surgical hubs provide a wide array of functionality to improve the outcomes of medical procedures. The data generated by the various surgical devices, visualization systems, and surgical hubs about the patient and the medical procedure may be transmitted to the cloud-based medical analytics system. This data may then be aggregated with similar data gathered from many other surgical hubs, visualization systems, and surgical instruments located at other medical facilities. Various patterns and correlations may be found through the cloud-based analytics system analyzing the collected data. Improvements in the techniques used to generate the data may be generated as a result, and these improvements may then be disseminated to the various surgical hubs, visualization systems, and surgical instruments. Due to the interconnectedness of all of the aforementioned components, improvements in medical procedures and practices may be found that otherwise may not be found if the many components were not so interconnected.
Examples of surgical hubs configured to receive, analyze, and output data, and methods of using such surgical hubs, are further described in U.S. Pat. Pub. No. 2019/0200844 entitled “Method Of Hub Communication, Processing, Storage And Display” filed Dec. 4, 2018, U.S. Pat. Pub. No. 2019/0200981 entitled “Method Of Compressing Tissue Within A Stapling Device And Simultaneously Displaying The Location Of The Tissue Within The Jaws” filed Dec. 4, 2018, U.S. Pat. Pub. No. 2019/0201046 entitled “Method For Controlling Smart Energy Devices” filed Dec. 4, 2018, U.S. Pat. Pub. No. 2019/0201114 entitled “Adaptive Control Program Updates For Surgical Hubs” filed Mar. 29, 2018, U.S. Pat. Pub. No. 2019/0201140 entitled “Surgical Hub Situational Awareness” filed Mar. 29, 2018, U.S. Pat. Pub. No. 2019/0206004 entitled “Interactive Surgical Systems With Condition Handling Of Devices And Data Capabilities” filed Mar. 29, 2018, U.S. Pat. Pub. No. 2019/0206555 entitled “Cloud-based Medical Analytics For Customization And Recommendations To A User” filed Mar. 29, 2018, and U.S. Pat. Pub. No. 2019/0207857 entitled “Surgical Network Determination Of Prioritization Of Communication, Interaction, Or Processing Based On System Or Device Needs” filed Nov. 6, 2018, which are hereby incorporated by reference in their entireties.
FIG. 18 illustrates one embodiment of a computer-implemented interactive surgical system 700 that includes one or more surgical systems 702 and a cloud-based system (e.g., a cloud 704 that can include a remote server 713 coupled to a storage device 705). Each surgical system 702 includes at least one surgical hub 706 in communication with the cloud 704. In one example, as illustrated in FIG. 18 , the surgical system 702 includes a visualization system 708, a robotic system 710, and an intelligent (or “smart”) surgical instrument 712, which are configured to communicate with one another and/or the hub 706. The intelligent surgical instrument 712 can include imaging device(s). The surgical system 702 can include an M number of hubs 706, an N number of visualization systems 708, an O number of robotic systems 710, and a P number of intelligent surgical instruments 712, where M, N, O, and P are integers greater than or equal to one that may or may not be equal to any one or more of each other. Various exemplary intelligent surgical instruments and robotic systems are described herein.
Data received by a surgical hub from a surgical visualization system can be used in any of a variety of ways. In an exemplary embodiment, the surgical hub can receive data from a surgical visualization system in use with a patient in a surgical setting, e.g., in use in an operating room during performance of a surgical procedure. The surgical hub can use the received data in any of one or more ways, as discussed herein.
The surgical hub can be configured to analyze received data in real time with use of the surgical visualization system and adjust control one or more of the surgical visualization system and/or one or more intelligent surgical instruments in use with the patient based on the analysis of the received data. Such adjustment can include, for example, adjusting one or operational control parameters of intelligent surgical instrument(s), causing one or more sensors of one or more intelligent surgical instruments to take a measurement to help gain an understanding of the patient's current physiological condition, and/or current operational status of an intelligent surgical instrument, and other adjustments. Controlling and adjusting operation of intelligent surgical instruments is discussed further below. Examples of operational control parameters of an intelligent surgical instrument include motor speed, cutting element speed, time, duration, level of energy application, and light emission. Examples of surgical hubs and of controlling and adjusting intelligent surgical instrument operation are described further in previously mentioned U.S. patent application Ser. No. 16/729,772 entitled “Analyzing Surgical Trends By A Surgical System” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,747 entitled “Dynamic Surgical Visualization Systems” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,744 entitled “Visualization Systems Using Structured Light” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,778 entitled “System And Method For Determining, Adjusting, And Managing Resection Margin About A Subject Tissue” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,729 entitled “Surgical Systems For Proposing And Corroborating Organ Portion Removals” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,778 entitled “Surgical System For Overlaying Surgical Instrument Data Onto A Virtual Three Dimensional Construct Of An Organ” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,751 entitled “Surgical Systems For Generating Three Dimensional Constructs Of Anatomical Organs And Coupling Identified Anatomical Structures Thereto” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,740 entitled “Surgical Systems Correlating Visualization Data And Powered Surgical Instrument Data” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,737 entitled “Adaptive Surgical System Control According To Surgical Smoke Cloud Characteristics” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,796 entitled “Adaptive Surgical System Control According To Surgical Smoke Particulate Characteristics” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,803 entitled “Adaptive Visualization By A Surgical System” filed Dec. 30, 2019, and U.S. patent application Ser. No. 16/729,807 entitled “Method Of Using Imaging Devices In Surgery” filed Dec. 30, 2019, and in U.S. patent application Ser. No. 17/068,857 entitled “Adaptive Responses From Smart Packaging Of Drug Delivery Absorbable Adjuncts” filed Oct. 13, 2020, U.S. patent application Ser. No. 17/068,858 entitled “Drug Administration Devices That Communicate With Surgical Hubs” filed Oct. 13, 2020, U.S. patent application Ser. No. 17/068,859 entitled “Controlling Operation Of Drug Administration Devices Using Surgical Hubs” filed Oct. 13, 2020, U.S. patent application Ser. No. 17/068,863 entitled “Patient Monitoring Using Drug Administration Devices” filed Oct. 13, 2020, U.S. patent application Ser. No. 17/068,865 entitled “Monitoring And Communicating Information Using Drug Administration Devices” filed Oct. 13, 2020, and U.S. patent application Ser. No. 17/068,867 entitled “Aggregating And Analyzing Drug Administration Data” filed Oct. 13, 2020, which are hereby incorporated by reference in their entireties.
The surgical hub can be configured to cause visualization of the received data to be provided in the surgical setting on a display so that a medical practitioner in the surgical setting can view the data and thereby receive an understanding of the operation of the imaging device(s) in use in the surgical setting. Such information provided via visualization can include text and/or images.
FIG. 19 illustrates one embodiment of a surgical system 802 including a surgical hub 806 (e.g., the surgical hub 706 of FIG. 18 or other surgical hub described herein), a robotic surgical system 810 (e.g., the robotic surgical system 110 of FIG. 1 or other robotic surgical system herein), and a visualization system 808 (e.g., the visualization system 100 of FIG. 1 or other visualization system described herein). The surgical hub 806 can be in communication with a cloud, as discussed herein. FIG. 19 shows the surgical system 802 being used to perform a surgical procedure on a patient who is lying down on an operating table 814 in a surgical operating room 816. The robotic system 810 includes a surgeon's console 818, a patient side cart 820 (surgical robot), and a robotic system surgical hub 822. The robotic system surgical hub 822 is generally configured similar to the surgical hub 822 and can be in communication with a cloud. In some embodiments, the robotic system surgical hub 822 and the surgical hub 806 can be combined. The patient side cart 820 can manipulate an intelligent surgical tool 812 through a minimally invasive incision in the body of the patient while a medical practitioner, e.g., a surgeon, nurse, and/or other medical practitioner, views the surgical site through the surgeon's console 818. An image of the surgical site can be obtained by an imaging device 824 (e.g., the imaging device 120 of FIG. 1 or other imaging device described herein), which can be manipulated by the patient side cart 820 to orient the imaging device 824. The robotic system surgical hub 822 can be used to process the images of the surgical site for subsequent display to the surgeon through the surgeon's console 818.
A primary display 819 is positioned in the sterile field of the operating room 816 and is configured to be visible to an operator at the operating table 814. In addition, as in this illustrated embodiment, a visualization tower 818 can positioned outside the sterile field. The visualization tower 818 includes a first non-sterile display 807 and a second non-sterile display 809, which face away from each other. The visualization system 808, guided by the surgical hub 806, is configured to utilize the displays 807, 809, 819 to coordinate information flow to medical practitioners inside and outside the sterile field. For example, the surgical hub 806 can cause the visualization system 808 to display a snapshot and/or a video of a surgical site, as obtained by the imaging device 824, on one or both of the non-sterile displays 807, 809, while maintaining a live feed of the surgical site on the primary display 819. The snapshot and/or video on the non-sterile display 807 and/or 809 can permit a non-sterile medical practitioner to perform a diagnostic step relevant to the surgical procedure, for example.
The surgical hub 806 is configured to route a diagnostic input or feedback entered by a non-sterile medical practitioner at the visualization tower 818 to the primary display 819 within the sterile field, where it can be viewed by a sterile medical practitioner at the operating table 814. For example, the input can be in the form of a modification to the snapshot and/or video displayed on the non-sterile display 807 and/or 809, which can be routed to the primary display 819 by the surgical hub 806.
The surgical hub 806 is configured to coordinate information flow to a display of the intelligent surgical instrument 812, as is described in various U.S. Patent applications that are incorporated by reference herein in the present disclosure. A diagnostic input or feedback entered by a non-sterile operator at the visualization tower 818 can be routed by the surgical hub 806 to the display 819 within the sterile field, where it can be viewed by the operator of the surgical instrument 812 and/or by other medical practitioner(s) in the sterile field.
The intelligent surgical instrument 812 and the imaging device 824, which is also an intelligent surgical tool, is being used with the patient in the surgical procedure as part of the surgical system 802. Other intelligent surgical instruments 812 a that can be used in the surgical procedure, e.g., that can be removably coupled to the patient side cart 820 and be in communication with the robotic surgical system 810 and the surgical hub 806, are also shown in FIG. 19 as being available. Non-intelligent (or “dumb”) surgical instruments 817, e.g., scissors, trocars, cannulas, scalpels, etc., that cannot be in communication with the robotic surgical system 810 and the surgical hub 806 are also shown in FIG. 19 as being available for use.
Operating Intelligent Surgical Instruments
An intelligent surgical device can have an algorithm stored thereon, e.g., in a memory thereof, configured to be executable on board the intelligent surgical device, e.g., by a processor thereof, to control operation of the intelligent surgical device. In some embodiments, instead of or in addition to being stored on the intelligent surgical device, the algorithm can be stored on a surgical hub, e.g., in a memory thereof, that is configured to communicate with the intelligent surgical device.
The algorithm is stored in the form of one or more sets of pluralities of data points defining and/or representing instructions, notifications, signals, etc. to control functions of the intelligent surgical device. In some embodiments, data gathered by the intelligent surgical device can be used by the intelligent surgical device, e.g., by a processor of the intelligent surgical device, to change at least one variable parameter of the algorithm. As discussed above, a surgical hub can be in communication with an intelligent surgical device, so data gathered by the intelligent surgical device can be communicated to the surgical hub and/or data gathered by another device in communication with the surgical hub can be communicated to the surgical hub, and data can be communicated from the surgical hub to the intelligent surgical device. Thus, instead of or in addition to the intelligent surgical device being configured to change a stored variable parameter, the surgical hub can be configured to communicate the changed at least one variable, alone or as part of the algorithm, to the intelligent surgical device and/or the surgical hub can communicate an instruction to the intelligent surgical device to change the at least one variable as determined by the surgical hub.
The at least one variable parameter is among the algorithm's data points, e.g., are included in instructions for operating the intelligent surgical device, and are thus each able to be changed by changing one or more of the stored pluralities of data points of the algorithm. After the at least one variable parameter has been changed, subsequent execution of the algorithm is according to the changed algorithm. As such, operation of the intelligent surgical device over time can be managed for a patient to increase the beneficial results use of the intelligent surgical device by taking into consideration actual situations of the patient and actual conditions and/or results of the surgical procedure in which the intelligent surgical device is being used. Changing the at least one variable parameter is automated to improve patient outcomes. Thus, the intelligent surgical device can be configured to provide personalized medicine based on the patient and the patient's surrounding conditions to provide a smart system. In a surgical setting in which the intelligent surgical device is being used during performance of a surgical procedure, automated changing of the at least one variable parameter may allow for the intelligent surgical device to be controlled based on data gathered during the performance of the surgical procedure, which may help ensure that the intelligent surgical device is used efficiently and correctly and/or may help reduce chances of patient harm by harming a critical anatomical structure.
The at least one variable parameter can be any of a variety of different operational parameters. Examples of variable parameters include motor speed, motor torque, energy level, energy application duration, tissue compression rate, jaw closure rate, cutting element speed, load threshold, etc.
FIG. 20 illustrates one embodiment of an intelligent surgical instrument 900 including a memory 902 having an algorithm 904 stored therein that includes at least one variable parameter. The algorithm 904 can be a single algorithm or can include a plurality of algorithms, e.g., separate algorithms for different aspects of the surgical instrument's operation, where each algorithm includes at least one variable parameter. The intelligent surgical instrument 900 can be the surgical device 102 of FIG. 1 , the imaging device 120 of FIG. 1 , the surgical device 202 of FIG. 8 , the imaging device 220 of FIG. 8 , the surgical device 402 of FIG. 15 , the surgical device 502 a of FIG. 17 , the surgical device 502 b of FIG. 17 , the surgical device 712 of FIG. 18 , the surgical device 812 of FIG. 19 , the imaging device 824 of FIG. 19 , or other intelligent surgical instrument. The surgical instrument 900 also includes a processor 906 configured to execute the algorithm 904 to control operation of at least one aspect of the surgical instrument 900. To execute the algorithm 904, the processor 906 is configured to run a program stored in the memory 902 to access a plurality of data points of the algorithm 904 in the memory 902.
The surgical instrument 900 also includes a communications interface 908, e.g., a wireless transceiver or other wired or wireless communications interface, configured to communicate with another device, such as a surgical hub 910. The communications interface 908 can be configured to allow one-way communication, such as providing data to a remote server (e.g., a cloud server or other server) and/or to a local, surgical hub server, and/or receiving instructions or commands from a remote server and/or a local, surgical hub server, or two-way communication, such as providing information, messages, data, etc. regarding the surgical instrument 900 and/or data stored thereon and receiving instructions, such as from a doctor; a remote server regarding updates to software; a local, surgical hub server regarding updates to software; etc.
The surgical instrument 900 is simplified in FIG. 20 and can include additional components, e.g., a bus system, a handle, a elongate shaft having an end effector at a distal end thereof, a power source, etc. The processor 906 can also be configured to execute instructions stored in the memory 902 to control the device 900 generally, including other electrical components thereof such as the communications interface 908, an audio speaker, a user interface, etc.
The processor 906 is configured to change at least one variable parameter of the algorithm 904 such that a subsequent execution of the algorithm 904 will be in accordance with the changed at least one variable parameter. To change the at least one variable parameter of the algorithm 904, the processor 906 is configured to modify or update the data point(s) of the at least one variable parameter in the memory 902. The processor 906 can be configured to change the at least one variable parameter of the algorithm 904 in real time with use of the surgical device 900 during performance of a surgical procedure, which may accommodate real time conditions.
Additionally or alternatively to the processor 906 changing the at least one variable parameter, the processor 906 can be configured to change the algorithm 904 and/or at least one variable parameter of the algorithm 904 in response to an instruction received from the surgical hub 910. In some embodiments, the processor 906 is configured to change the at least one variable parameter only after communicating with the surgical hub 910 and receiving an instruction therefrom, which may help ensure coordinated action of the surgical instrument 900 with other aspects of the surgical procedure in which the surgical instrument 900 is being used.
In an exemplary embodiment, the processor 906 executes the algorithm 904 to control operation of the surgical instrument 900, changes the at least one variable parameter of the algorithm 904 based on real time data, and executes the algorithm 904 after changing the at least one variable parameter to control operation of the surgical instrument 900.
FIG. 21 illustrates one embodiment of a method 912 of using of the surgical instrument 900 including a change of at least one variable parameter of the algorithm 904. The processor 906 controls 914 operation of the surgical instrument 900 by executing the algorithm 904 stored in the memory 902. Based on any of this subsequently known data and/or subsequently gathered data, the processor 904 changes 916 the at least one variable parameter of the algorithm 904 as discussed above. After changing the at least one variable parameter, the processor 906 controls 918 operation of the surgical instrument 900 by executing the algorithm 904, now with the changed at least one variable parameter. The processor 904 can change 916 the at least one variable parameter any number of times during performance of a surgical procedure, e.g., zero, one, two, three, etc. During any part of the method 912, the surgical instrument 900 can communicate with one or more computer systems, e.g., the surgical hub 910, a remote server such as a cloud server, etc., using the communications interface 908 to provide data thereto and/or receive instructions therefrom.
Situational Awareness
Operation of an intelligent surgical instrument can be altered based on situational awareness of the patient. The operation of the intelligent surgical instrument can be altered manually, such as by a user of the intelligent surgical instrument handling the instrument differently, providing a different input to the instrument, ceasing use of the instrument, etc. Additionally or alternatively, the operation of an intelligent surgical instrument can be changed automatically by an algorithm of the instrument being changed, e.g., by changing at least one variable parameter of the algorithm. As mentioned above, the algorithm can be adjusted automatically without user input requesting the change. Automating the adjustment during performance of a surgical procedure may help save time, may allow medical practitioners to focus on other aspects of the surgical procedure, and/or may ease the process of using the surgical instrument for a medical practitioner, which each may improve patient outcomes, such as by avoiding a critical structure, controlling the surgical instrument with consideration of a tissue type the instrument is being used on and/or near, etc.
The visualization systems described herein can be utilized as part of a situational awareness system that can be embodied or executed by a surgical hub, e.g., the surgical hub 706, the surgical hub 806, or other surgical hub described herein. In particular, characterizing, identifying, and/or visualizing surgical instruments (including their positions, orientations, and actions), tissues, structures, users, and/or other things located within the surgical field or the operating theater can provide contextual data that can be utilized by a situational awareness system to infer various information, such as a type of surgical procedure or a step thereof being performed, a type of tissue(s) and/or structure(s) being manipulated by a surgeon or other medical practitioner, and other information. The contextual data can then be utilized by the situational awareness system to provide alerts to a user, suggest subsequent steps or actions for the user to undertake, prepare surgical devices in anticipation for their use (e.g., activate an electrosurgical generator in anticipation of an electrosurgical instrument being utilized in a subsequent step of the surgical procedure, etc.), control operation of intelligent surgical instruments (e.g., customize surgical instrument operational parameters of an algorithm as discussed further below), and so on.
Although an intelligent surgical device including an algorithm that responds to sensed data, e.g., by having at least one variable parameter of the algorithm changed, can be an improvement over a “dumb” device that operates without accounting for sensed data, some sensed data can be incomplete or inconclusive when considered in isolation, e.g., without the context of the type of surgical procedure being performed or the type of tissue that is being operated on. Without knowing the procedural context (e.g., knowing the type of tissue being operated on or the type of procedure being performed), the algorithm may control the surgical device incorrectly or sub-optimally given the particular context-free sensed data. For example, the optimal manner for an algorithm to control a surgical instrument in response to a particular sensed parameter can vary according to the particular tissue type being operated on. This is due to the fact that different tissue types have different properties (e.g., resistance to tearing, ease of being cut, etc.) and thus respond differently to actions taken by surgical instruments. Therefore, it may be desirable for a surgical instrument to take different actions even when the same measurement for a particular parameter is sensed. As one example, the optimal manner in which to control a surgical stapler in response to the surgical stapler sensing an unexpectedly high force to close its end effector will vary depending upon whether the tissue type is susceptible or resistant to tearing. For tissues that are susceptible to tearing, such as lung tissue, the surgical instrument's control algorithm would optimally ramp down the motor in response to an unexpectedly high force to close to avoid tearing the tissue, e.g., change a variable parameter controlling motor speed or torque so the motor is slower. For tissues that are resistant to tearing, such as stomach tissue, the instrument's algorithm would optimally ramp up the motor in response to an unexpectedly high force to close to ensure that the end effector is clamped properly on the tissue, e.g., change a variable parameter controlling motor speed or torque so the motor is faster. Without knowing whether lung or stomach tissue has been clamped, the algorithm may be sub-optimally changed or not changed at all.
A surgical hub can be configured to derive information about a surgical procedure being performed based on data received from various data sources and then control modular devices accordingly. In other words, the surgical hub can be configured to infer information about the surgical procedure from received data and then control the modular devices operably coupled to the surgical hub based upon the inferred context of the surgical procedure. Modular devices can include any surgical device that is controllable by a situational awareness system, such as visualization system devices (e.g., a camera, a display screen, etc.), smart surgical instruments (e.g., an ultrasonic surgical instrument, an electrosurgical instrument, a surgical stapler, smoke evacuators, scopes, etc.). A modular device can include sensor(s)s configured to detect parameters associated with a patient with which the device is being used and/or associated with the modular device itself.
The contextual information derived or inferred from the received data can include, for example, a type of surgical procedure being performed, a particular step of the surgical procedure that the surgeon (or other medical practitioner) is performing, a type of tissue being operated on, or a body cavity that is the subject of the surgical procedure. The situational awareness system of the surgical hub can be configured to derive the contextual information from the data received from the data sources in a variety of different ways. In an exemplary embodiment, the contextual information received by the situational awareness system of the surgical hub is associated with a particular control adjustment or set of control adjustments for one or more modular devices. The control adjustments each correspond to a variable parameter. In one example, the situational awareness system includes a pattern recognition system, or machine learning system (e.g., an artificial neural network), that has been trained on training data to correlate various inputs (e.g., data from databases, patient monitoring devices, and/or modular devices) to corresponding contextual information regarding a surgical procedure. In other words, a machine learning system can be trained to accurately derive contextual information regarding a surgical procedure from the provided inputs. In another example, the situational awareness system can include a lookup table storing pre-characterized contextual information regarding a surgical procedure in association with one or more inputs (or ranges of inputs) corresponding to the contextual information. In response to a query with one or more inputs, the lookup table can return the corresponding contextual information for the situational awareness system for controlling at least one modular device. In another example, the situational awareness system includes a further machine learning system, lookup table, or other such system, which generates or retrieves one or more control adjustments for one or more modular devices when provided the contextual information as input.
A surgical hub including a situational awareness system may provide any number of benefits for a surgical system. One benefit includes improving the interpretation of sensed and collected data, which would in turn improve the processing accuracy and/or the usage of the data during the course of a surgical procedure. Another benefit is that the situational awareness system for the surgical hub may improve surgical procedure outcomes by allowing for adjustment of surgical instruments (and other modular devices) for the particular context of each surgical procedure (such as adjusting to different tissue types) and validating actions during a surgical procedure. Yet another benefit is that the situational awareness system may improve surgeon's and/or other medical practitioners' efficiency in performing surgical procedures by automatically suggesting next steps, providing data, and adjusting displays and other modular devices in the surgical theater according to the specific context of the procedure. Another benefit includes proactively and automatically controlling modular devices according to the particular step of the surgical procedure that is being performed to reduce the number of times that medical practitioners are required to interact with or control the surgical system during the course of a surgical procedure, such as by a situationally aware surgical hub proactively activating a generator to which an RF electrosurgical instrument is connected if it determines that a subsequent step of the procedure requires the use of the instrument. Proactively activating the energy source allows the instrument to be ready for use a soon as the preceding step of the procedure is completed.
For example, a situationally aware surgical hub can be configured to determine what type of tissue is being operated on. Therefore, when an unexpectedly high force to close a surgical instrument's end effector is detected, the situationally aware surgical hub can be configured to correctly ramp up or ramp down a motor of the surgical instrument for the type of tissue, e.g., by changing or causing change of at least one variable parameter of an algorithm for the surgical instrument regarding motor speed or torque.
For another example, a type of tissue being operated can affect adjustments that are made to compression rate and load thresholds of a surgical stapler for a particular tissue gap measurement. A situationally aware surgical hub can be configured to infer whether a surgical procedure being performed is a thoracic or an abdominal procedure, allowing the surgical hub to determine whether the tissue clamped by an end effector of the surgical stapler is lung tissue (for a thoracic procedure) or stomach tissue (for an abdominal procedure). The surgical hub can then be configured to cause adjustment of the compression rate and load thresholds of the surgical stapler appropriately for the type of tissue, e.g., by changing or causing change of at least one variable parameter of an algorithm for the surgical stapler regarding compression rate and load threshold.
As yet another example, a type of body cavity being operated in during an insufflation procedure can affect the function of a smoke evacuator. A situationally aware surgical hub can be configured to determine whether the surgical site is under pressure (by determining that the surgical procedure is utilizing insufflation) and determine the procedure type. As a procedure type is generally performed in a specific body cavity, the surgical hub can be configured to control a motor rate of the smoke evacuator appropriately for the body cavity being operated in, e.g., by changing or causing change of at least one variable parameter of an algorithm for the smoke evacuator regarding motor rate. Thus, a situationally aware surgical hub may provide a consistent amount of smoke evacuation for both thoracic and abdominal procedures.
As yet another example, a type of procedure being performed can affect the optimal energy level for an ultrasonic surgical instrument or radio frequency (RF) electrosurgical instrument to operate at. Arthroscopic procedures, for example, require higher energy levels because an end effector of the ultrasonic surgical instrument or RF electrosurgical instrument is immersed in fluid. A situationally aware surgical hub can be configured to determine whether the surgical procedure is an arthroscopic procedure. The surgical hub can be configured to adjust an RF power level or an ultrasonic amplitude of the generator (e.g., adjust energy level) to compensate for the fluid filled environment, e.g., by changing or causing change of at least one variable parameter of an algorithm for the instrument and/or a generator regarding energy level. Relatedly, a type of tissue being operated on can affect the optimal energy level for an ultrasonic surgical instrument or RF electrosurgical instrument to operate at. A situationally aware surgical hub can be configured to determine what type of surgical procedure is being performed and then customize the energy level for the ultrasonic surgical instrument or RF electrosurgical instrument, respectively, according to the expected tissue profile for the surgical procedure, e.g., by changing or causing change of at least one variable parameter of an algorithm for the instrument and/or a generator regarding energy level. Furthermore, a situationally aware surgical hub can be configured to adjust the energy level for the ultrasonic surgical instrument or RF electrosurgical instrument throughout the course of a surgical procedure, rather than just on a procedure-by-procedure basis. A situationally aware surgical hub can be configured to determine what step of the surgical procedure is being performed or will subsequently be performed and then update the control algorithm(s) for the generator and/or ultrasonic surgical instrument or RF electrosurgical instrument to set the energy level at a value appropriate for the expected tissue type according to the surgical procedure step.
As another example, a situationally aware surgical hub can be configured to determine whether the current or subsequent step of a surgical procedure requires a different view or degree of magnification on a display according to feature(s) at the surgical site that the surgeon and/or other medical practitioner is expected to need to view. The surgical hub can be configured to proactively change the displayed view (supplied by, e.g., an imaging device for a visualization system) accordingly so that the display automatically adjusts throughout the surgical procedure.
As yet another example, a situationally aware surgical hub can be configured to determine which step of a surgical procedure is being performed or will subsequently be performed and whether particular data or comparisons between data will be required for that step of the surgical procedure. The surgical hub can be configured to automatically call up data screens based upon the step of the surgical procedure being performed, without waiting for the surgeon or other medical practitioner to ask for the particular information.
As another example, a situationally aware surgical hub can be configured to determine whether a surgeon and/or other medical practitioner is making an error or otherwise deviating from an expected course of action during the course of a surgical procedure, e.g., as provided in a pre-operative surgical plan. For example, the surgical hub can be configured to determine a type of surgical procedure being performed, retrieve a corresponding list of steps or order of equipment usage (e.g., from a memory), and then compare the steps being performed or the equipment being used during the course of the surgical procedure to the expected steps or equipment for the type of surgical procedure that the surgical hub determined is being performed. The surgical hub can be configured to provide an alert (visual, audible, and/or tactile) indicating that an unexpected action is being performed or an unexpected device is being utilized at the particular step in the surgical procedure.
In certain instances, operation of a robotic surgical system, such as any of the various robotic surgical systems described herein, can be controlled by the surgical hub based on its situational awareness and/or feedback from the components thereof and/or based on information from a cloud (e.g., the cloud 713 of FIG. 18 ).
Embodiments of situational awareness systems and using situational awareness systems during performance of a surgical procedure are described further in previously mentioned U.S. patent application Ser. No. 16/729,772 entitled “Analyzing Surgical Trends By A Surgical System” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,747 entitled “Dynamic Surgical Visualization Systems” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,744 entitled “Visualization Systems Using Structured Light” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,778 entitled “System And Method For Determining, Adjusting, And Managing Resection Margin About A Subject Tissue” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,729 entitled “Surgical Systems For Proposing And Corroborating Organ Portion Removals” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,778 entitled “Surgical System For Overlaying Surgical Instrument Data Onto A Virtual Three Dimensional Construct Of An Organ” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,751 entitled “Surgical Systems For Generating Three Dimensional Constructs Of Anatomical Organs And Coupling Identified Anatomical Structures Thereto” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,740 entitled “Surgical Systems Correlating Visualization Data And Powered Surgical Instrument Data” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,737 entitled “Adaptive Surgical System Control According To Surgical Smoke Cloud Characteristics” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,796 entitled “Adaptive Surgical System Control According To Surgical Smoke Particulate Characteristics” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,803 entitled “Adaptive Visualization By A Surgical System” filed Dec. 30, 2019, and U.S. patent application Ser. No. 16/729,807 entitled “Method Of Using Imaging Devices In Surgery” filed Dec. 30, 2019.
Selective Re-Inflation for Visualization or Operation
In one aspect the present disclosure relates to a surgical system having a surgical instrument configured for endoluminal access (e.g., an endoscope) that includes instrument sealing element(s) that allow for selective inflation, or re-inflation, of a portion of a natural body lumen or organ (e.g., a lung, a stomach, a colon, or small intestine), such as for visualization or operational purposes. In some embodiments, the natural body lumen or organ is inflated from a collapsed state (e.g., a deflated configuration relative to a normal configuration), whereas in other embodiments, the natural body lumen or organ is inflated from a non-collapsed state (e.g., a normal configuration).
In certain exemplary aspects, the surgical instrument includes a fluid channel that extends through the surgical instrument and at least one deployable sealing element that is configured to form a first seal at a portion of a natural body lumen or organ, and. The fluid channel is configured to allow fluid ingress and egress distal to the portion of the natural body lumen or organ while the at least one deployable sealing member is in an expanded state. As a result, the natural body lumen distal to the portion can be selectively pressurized. That is, unlike conventional systems (e.g., systems that re-inflate the entire collapsed natural body lumen or organ), the present surgical systems are designed to selectively distend only a portion of the natural body lumen or organ. This distention can increase the surgical working space at the treatment site to thereby improve instrument access and movement (e.g., for dissection and resection), reposition the portion of the natural body lumen or organ or tumor, if present, in such a way that can result in the intra-operative imaging to substantially, or completely, match the pre-operative imaging, allow for pressure testing at the surgical site (e.g., to check for leaks after an anastomosis), or control the environment (e.g., temperature, humidity) within the natural body lumen or organ to increase surgical efficiency (e.g., for dissection).
The terms “filled” or “expanded” are intended to mean that the sealing element(s) has/have fluid therein or added thereto in a desired amount or pressure. These terms are not intended to mean that the sealing element(s) is/are necessarily entirely or 100% filled with a fluid when the sealing element(s) are “expanded” (however, such embodiments are within the scope of the term “filled”). Similarly, the term “unexpanded” does not necessarily mean that the sealing element(s) is/are entirely empty or at 0 pressure. There may be some fluid and the sealing element(s) may have a non-zero pressure in an “unexpanded” state. An “uninflated” sealing element(s) is/are intended to mean that the sealing element(s) does/do not include fluid in an amount or at a pressure that would be desired after the sealing element(s) is/are filled.
An exemplary surgical system can include a variety of features as described herein and illustrated in the drawings. However, a person skilled in the art will appreciate that the surgical systems can include only some of these features and/or it can include a variety of other features known in the art. The surgical systems described herein are merely intended to represent certain exemplary embodiments. Moreover, while the surgical systems are shown and described in connection with a lung and a colon, a person skilled in the art will appreciate that these surgical systems can be used in connection with any other suitable natural body lumen or organ.
A method of inflating a lung to increase visualization and access during a lung surgical procedure is to selectively inflate and deflate a portion of the lung. Similar to the procedure for manipulating the lungs, a bronchoscope is passed through the trachea of a patient and into a respective bronchi of the lung. A sealing element, which can be arranged on the distal end of the scope, can be inflated to locally seal off the scope relative to the bronchus. The scope can then be used to re-inflate a portion of the organ distal to the scope, such as a singular lobe of a lung.
FIG. 22 and FIG. 23 illustrate one embodiment of a surgical system 6100 that is configured for endoluminal access into a lung 6010 and partial inflation thereof. As will be described in more detail below, the surgical system 6100 is used to selectively pressurize a natural body lumen (e.g., first bronchiole 6022) within the lung 6010. For purposes of simplicity, certain components of the surgical system 6100 and the lung 6010 are not illustrated. As shown, the lung 6010 includes an outer tissue surface 6012, a trachea 6014, a right bronchus 6016, and bronchioles 6018. The trachea 6014, right bronchus 6016, and the bronchioles 6018 are fluidly coupled together. Additionally, the lung 6010 includes an upper lobe 6020, which includes first and second bronchioles 6022 and 6024. As illustrated in FIG. 23 , the lung 6010 is in a collapsed state, with the inflated state being represented as a dashed-line border IS. When operating in the thoracic cavity, the lung 6010 is collapsed to provide sufficient working space between the rib cage and the lungs such that the laparoscopically arranged instruments 6110 can easily access and manipulate the lung 6010. In use, as described in more detail below, the surgical system 6100 can partially inflate a portion of the lung 6010.
The surgical system 6100 includes a surgical instrument 6102 configured for endoluminal access through the trachea 6014 and into the lung 6010. In some aspects, the surgical instrument 6102 can have a flexible body 6103 with a distal tip 6104 that configured to be endoscopically inserted through a patient's mouth (not shown) and down the trachea 6014. In use, as shown in FIG. 22 and FIG. 23 , the distal tip 6104 is then passed into the lung 6010 through the right bronchus 6016, and into the first bronchiole 6022 of the upper lobe 6020 The distal tip 6104 can have a variety of configurations. In some embodiments, the distal tip 6104 can be tapered to help navigate through the lung.
As further shown in FIG. 22 and FIG. 23 , the surgical instrument 6102 includes at least one deployable sealing element 6106 operatively coupled to the surgical instrument 6102. The at least one deployable sealing element 6106 can be arranged on or proximal to the distal tip 6104 of the surgical instrument 6102 such that the deployable sealing element 6106 is positioned within the first bronchiole 6022 when the distal tip 6104 is inserted therein. The at least one deployable sealing element 6106 is configured to move between unexpanded and expanded states. The at least one deployable sealing element 6101 can have a variety of configurations. For example, in some embodiments, the at least one deployable sealing element 6106 can be an inflatable balloon. In other embodiments, the at least one deployable sealing element can be a mechanically expandable stent.
In use, when in the expanded state, the at least one deployable sealing element 6106 is configured to form a first seal 6111 within the first bronchiole 6022. More specifically, as shown in FIG. 23 , when the at least one deployable sealing element 6106 is expanded into its expanded state within the first bronchiole 6022, the at least one deployable sealing element 6106 contacts an internal surface 6023 of the first bronchiole 6022. This contact forms the first seal 6111 therebetween. The at least one deployable sealing element 6106 can alternate between its unexpanded and expanded states by passing fluid into or removing fluid from the at least one deployable sealing element 6106 through a first fluid channel 6108 that passes through the length of the surgical instrument 6102. The fluid passed into or out of the at least one deployable sealing element 6106 can be any suitable fluid (e.g., saline, carbon dioxide gas, and the like). The fluid system used to control the ingress or egress of fluid into the deployable sealing element 6106 can include a pump and a fluid reservoir. The pump creates a pressure which pushes the fluid into the deployable sealing element 6106, to expand the deployable sealing element 6106, and creates a suction that draws the fluid from the deployable sealing element 6106 in order to collapse the deployable sealing element 6106.
The surgical instrument 6102 also includes a fluid channel 6107 that extends therethrough. The fluid channel 6107 terminates at an opening 6105 within the distal tip 6104. The opening 6105, in combination with the fluid channel 6107, are configured to allow fluid ingress and egress into and from the sealed portion 6113 of the first bronchiole 6022 (e.g., the portion of the first bronchiole 6022 distal to the first seal 6111). This allows the sealed portion 6113 to be selectively pressurized. The fluid system used to control the ingress or egress of fluid into the sealed portion 6113 can include a pump and a fluid reservoir arranged outside of the body. The pump is configured to create a pressure which forces the fluid into the sealed portion 6113, which thereby pressurizes the sealed portion 6113. Additionally, when pressurization of the sealed portion 6113 is no longer needed, the pump can create a suction that draws the fluid from the sealed portion 6113 and into the fluid reservoir in order to collapse the sealed portion 6113. Thus, in use, once the first seal 6111 within the first bronchiole 6022 is created, the upper lobe 6020 of the lung 6010 can then be at least partially re-inflated via the injection of fluid through the fluid channel 6107 and the opening 6105 and into the first bronchiole 6022. As a result, the inflated upper lobe 6020 is closer to its pre-deflated shape (see FIG. 24 ), and thus, the intra-operative imagining thereof is similar to that of the pre-operative imagining. Further, the fluid can then be subsequently drawn out of the first bronchiole 6022 through opening 6105 and the fluid channel 6107.
The surgical instrument 6102 can further include an optical sensor arranged at the distal tip 6104. The optical sensor can be configured to allow a user to determine the location of the surgical instrument 6102 within the lung 6010 and to help the user position the distal tip 6104 into the desired bronchiole, such as first bronchiole 6022. Views from the optical sensor can be provided in real time to a user (e.g., a surgeon), such as on a display (e.g., a monitor, a computer tablet screen, etc.).
Further, in use, another surgical instrument 6110 can be introduced laparoscopically within the thoracic cavity in order to visual/and or operate on the lung 6010 from the extraluminal space. The surgical instrument 6110 can include a variety of surgical tools, such as graspers, optical sensors, and/or electrosurgical tools. In an exemplary embodiment, where the surgical instrument 6110 is or includes an optical sensor, a user (e.g., a surgeon) can visually inspect the partially inflated lung 6010 (FIG. 24 ) to determine if a leak is present (e.g., in combination with use of a contrast or fluorescing agent mixed with the inflation fluid), identity inadvertent tears and tissue trauma for repair, or both.
FIG. 24 illustrates a schematic view of the lung 6010 depicting a tumor in both a pre-collapsed position and a collapsed position. In order for a medical practitioner to plan the surgical procedure, multiple pre-operative scans (e.g., MRI, CAT Scan, X-Rays) are performed to determine the location of the tumor 6021 as well as the surrounding healthy tissue, which cannot be removed. However pre-operative scans are performed when the lung is inflated and the patient is awake. During surgery, the lung is collapsed, which results in the lung shrinking considerably relative to its original size. This can render the pre-operative scans useless since the tumor 6021 may be in a completely different location. As shown in FIG. 24 , with the lung deflated, the tumor 6021 is arranged at location 6021 a, which is in a lower position within the upper lobe 6020. When the lung 6010 is selectively pressurized, the tumor 6021 will be located in location 6021 b, which is at a higher position within the upper lobe relative to location 6021 a. This would place the tumor 6021 within the first bronchiole 6022 closer to its original spot when compared to pre-operative scans.
In some embodiments where a portion of the lung is removed (e.g., an upper lobe 6020 that is distal to the end of the first bronchiole 6022), the remaining portion of the lung can be pressure tested to ensure that the lung is properly sealed after completion of the surgical dissection, thus helping to identity inadvertent tears and tissue trauma in need of subsequent repair. In some embodiments, a contrast or fluorescing agent can be mixed into the fluid that can help enable real-time visualization of the airways (e.g., with the use of a laparoscopic camera 6112). This real-time visualization can allow for clearer cooperative surgical intervention within the lung that would not otherwise be available in instances where the entire lung is inflated.
In some embodiments, the surgical instrument can include two or more deployable sealing elements arranged on the flexible body. This arrangement can allow two or more portions of the lung to be pressurized (e.g., two or more portions of the first bronchiole 6022). In other embodiments, the flexible body can have channel arms extending outward from a distal end of the flexible body in which each channel arm includes at least one deployable sealing element arranged on or proximal to a respective distal end. Each channel arm can be independently manipulated so that each respective sealing element can be advanced into a respective separate bronchiole. This arrangement can enable multiple portions of the lung to be selectively inflated or deflated through the inflation and deflation of the separate bronchioles while also allowing the mechanical manipulation of the surgical instrument. The cooperative selective inflation and articulation using multiple sealing element can be used to bend or fold the lung, improving access to the surgical site.
In another embodiment, local embolization using a fluid (e.g., saline, carbon dioxide gas, or the like) to expand a local portion of a deflated natural body lumen or organ (e.g., the first bronchiole 6022 in FIG. 22 ) can enable tissue plane separation or dissection of a tumor, if present. In an embodiment where the fluid is saline, a local saline injection could also change the conductivity and contrast properties of the tissue. This change in conductivity and contrast could improve visualization and/or locally advanced energy ablation and cauterization.
In other embodiments, a sealing element can include a laparoscopically deployed portion in addition to an endoscopically arranged sealing element within a natural body lumen or organ. An example of a laparoscopically arranged portion can be a surgical instrument that operates in cooperation with the endoscopically arranged sealing element. The surgical instrument can apply a wrap or band on the external surface of the natural body lumen or organ at the same location as the endoscopically arranged sealing element to prevent over distention of the sealing element.
In other embodiments, sealing one or more ends of a portion of a natural body lumen to be selectively pressurized, and thus inflated, can be accomplished by applying a local concentric suction to an inner surface of the natural body lumen or organ. In order to generate a sufficient seal at one or more ends during a leak test, the suction pressure is greater than the leak test pressure (e.g., the pressure used to inflate the natural body lumen, such as during a leak test). This arrangement can prevent over distension of the natural body lumen or organ since the portion of the natural body lumen has had a partial vacuum applied prior to pressurizing, lowering the required max leak pressure.
As noted above, the present surgical systems can be configured to selectively pressurize other natural body lumens or organs. For example, as discussed below, the present surgical systems can be configured to partially inflate one or more portions of the colon.
Surgery is often the primary treatment for early-stage colon cancers. The type of surgery used depends on the stage (extent) of the cancer, its location in the colon, and the goal of the surgery. Some early colon cancers (stage 0 and some early stage I tumors) and most polyps can be removed during a colonoscopy. However, if the cancer has progressed, a local excision or colectomy, a surgical procedure that removes all or part of the colon, may be required. In certain instances, nearby lymph nodes are also removed. A hemicolectomy, or partial colectomy, can be performed if only part of the colon is removed. In a segmental resection of the colon the surgeon removes the diseased part of the colon along with a small segment of non-diseased colon on either side. Usually, about one-fourth to one-third of the colon is removed, depending on the size and location of the cancer. Major resections of the colon are illustrated in FIG. 25 , in which (i) A-B is a right hemicolectomy, A-C is an extended right hemicolectomy, B-C is a transverse colectomy, C-E is a left hemicolectomy, D-E is a sigmoid colectomy, D-F is an anterior resection, D-G is a (ultra) low anterior resection, D-H is an abdomino-perineal resection, A-D is a subtotal colectomy, A-E is a total colectomy, and A-H is a total procto-colectomy. Once the resection is complete, the remaining intact sections of colon are then reattached.
A colectomy can be performed through an open colectomy, where a single incision through the abdominal wall is used to access the colon for separation and removal of the affected colon tissue, and through a laparoscopic-assisted colectomy. With a laparoscopic-assisted colectomy, the surgery is done through many smaller incisions with instruments and a laparoscope passing through the small incisions to remove the entire colon or a part thereof At the beginning of the procedure, the abdomen is inflated with gas, e.g., carbon dioxide, to provide a working space for the surgeon. The laparoscope transmits images inside the abdominal cavity, giving the surgeon a magnified view of the patient's internal organs on a monitor. Several other trocars are inserted to allow the surgeon to access the body cavity to work inside the body cavity and remove the appropriate part(s) of the colon. Once the diseased parts of the colon are removed, the remaining ends of the colon are attached to each other, e.g., via staples or sutures. The entire procedure may be completed through the cannulas or by lengthening one of the small cannula incisions.
Following a colectomy and reattachment of the colon, it can be beneficial to test for leaks of the colon at the connection site. With conventional systems, leak testing is typically carried out by laparoscopically arranging clamps to the colon to create a seal at the rectum and at the distal end of the colon, and once the seals are created, inflating the entire colon. However, inflation of the entire colon can reduce the working volume space within the abdominal cavity and can be inefficient. As will be described in more detail below, unlike the conventional systems, the present surgical systems can be configured to inflate one or more sections of the colons for leak testing and/or identifying unanticipated tissue damage. While the following discussion is with respect to the colon, a person skilled in the art will appreciate that the present surgical systems can be used in connection with other suitable natural body lumens or organs for leak testing and/or identifying unanticipated tissue damage.
FIG. 26 and FIG. 27 illustrate one embodiment of a surgical system 6200 that is configured for endoluminal access into and partial inflation of a colon 6050. As will be described in more detail below, the surgical system 6200 is used to selectively pressurize a portion of the colon 6050 (e.g., section A). For purposes of simplicity, certain components of the surgical system 6200 and the colon 6050 are not illustrated. While this surgical system 6200 is shown and described in connection with inflation of section A of the colon 6050, a person skilled in the art will appreciate that the surgical system 6200 can be used to additionally, or in the alternative, inflate other sections of the colon 6050.
As shown in FIG. 26 , the colon 6050 includes an intestinal wall 6052 defining a passageway 6053 through the colon 6050. The intestinal wall 6052 further defines different segments of the colon (e.g., cecum, not shown, ascending colon 6054, transverse colon 6055, descending colon 6056, sigmoid colon 6057, and rectum 6058). The rectum 6058 is connected to and extends from a natural orifice 6060 to the sigmoid colon 6050. As illustrated in FIG. 26 , the colon 6050 has undergone a segmental resection in which a portion of the rectum 6058 and sigmoid colon 6057 has been removed and the remaining portions thereof attached at connection point 6059 (e.g., by sutures, staples, or other suitable attachment mechanism(s)). Further, after the segmental resection, the surgical system 6200 can be used to identify any leaks or tissue damage (e.g., at the connection site 6059 or within any one or more segment of the colon, such as segment A, which includes the connection site 6059, segment B, segment C, or segment D illustrated in FIG. 26 ).
The surgical system 6200 can have a variety of configurations. In some embodiments, as shown in FIG. 26 , the surgical system 6200 includes a surgical instrument 6202 configured for endoluminal access through the natural orifice 6060 and into the colon 6050. The surgical instrument, which is shown in more detail in FIG. 27 , includes a flexible body 6204 having an inner tube 6210 and an outer tube 6212 that is disposed about at least a portion of the inner tube 6210. In other embodiments, the flexible body can have other suitable configurations and shapes.
The surgical instrument 6202 can include at least one deployable sealing element. In this illustrated embodiment, the surgical system 6200 includes two deployable sealing elements 6208, 6214. The first deployable sealing element 6208 is coupled to the outer tube 6212 and positioned proximal to the distal end 6211 of the outer tube 6212. The second deployable sealing element 6214 is coupled to the inner tube 6210 and positioned proximal to the distal end 6215 of the inner tube 6210. In this illustrated embodiment, the inner tube 6210 extends through the outer tube 6212 and can move relative to the outer tube 6212.
Due to this arrangement, the second deployable sealing element 6214 can be distally spaced from the sealing element 6208 to allow a sealed portion 6220 between the two sealing elements 6208 and 6214. In order to form the sealed portion 6220, the inner tube 6210 and outer tube 6212 are inserted together as surgical instrument 6202. Once the first deployable sealing element 6208 is in position, the first deployable sealing element 6208 is deployed to contact the internal surface 6051. With the first deployable sealing element 6208 in position, the inner tube 6210 is further inserted into the colon 6050 to place the second sealing element 6214 in a position distal to the first deployable sealing element 6208. When the second deployable sealing element 6214 is in position, the second deployable sealing elements 6214 can be expanded to contact the inner wall 6051. A camera 6222 is arranged in the distal tip 6215 of the inner tube 6210 in order to allow navigation within the colon 6050.
The first and second deployable sealing elements 6208, 6214 are configured to move between respective unexpanded and expanded states. When in the expanded state, the first deployable sealing element 6208 is configured to form a first seal 6226 within the inner tissue surface 6051 of the colon 6050. Similarly, when in an expanded state, the second deployable sealing element 6214 is configured to form a second seal 6228 within the inner tissue surface 6051 at a location distal to the first seal 6226. The first and second deployable sealing elements 6208, 6214 can have a variety of configurations. For example, in some embodiments, the first deployable sealing element 6208, the second deployable sealing elements 6214 can be in the form of an inflatable balloon, or a mechanically expanding stent. In this illustrated embodiment, both the first deployable sealing element 6208 and the second deployable sealing element 6214 are each in the form of an inflatable balloon.
Each of the first and second deployable sealing elements 6208, 6214 can move between respective unexpanded and expanded states. In this illustrated embodiment, the first deployable sealing element 6208 can move between an unexpanded state and an expanded state (FIG. 27 ) by passing fluid (e.g., saline, gas, or any other suitable fluid(s)) into the sealing elements 6208 through a first fluid channel 6207 that is in fluid communication with the first deployable sealing and extends through the outer tube 6212. To move the first deployable sealing element 6208 from an expanded state to an unexpanded state, fluid is removed from the sealing element 6208 though the first fluid channel 6207. Similarly, the second deployable sealing element 6214 can move from an unexpanded state to an expanded state (FIG. 27 ) by passing fluid (e.g., saline, gas, or any other suitable fluid(s)) into the second deployable sealing element through a second fluid channel 6209 that is in fluid communication with the second sealing element and extends through the inner tube. To move the second deployable sealing element from an expanded state to an unexpanded state, fluid is removed from the sealing element 6214 though the second fluid channel 6209.
In use, once the unexpanded first deployable sealing element is positioned at a desired location within the colon, fluid can be passed into the first sealing element to cause it to expand to form a first seal, and thus move from an unexpanded state to an expanded state. Similarly, once an unexpanded second deployable sealing element is positioned at a desired location within the colon, fluid can be passed into the second deployable sealing element to cause the second deployable sealing element to expand to form a second seal, and thus move from an unexpanded state to an expanded state. By creating a seal with the first deployable sealing element at the distal end relative to the colon anastomosis, and creating a seal with the deployable second sealing element proximal end relative to the colon anastomosis, the targeted section of the colon can then be pressurized with a fluid. The fluid can include a dye, contrast agent, or florescence agent, and be passed into the sealed section at a controllable pressure that would allow for the leak testing of the surgical site. Using a laparoscope already inserted from the colectomy, the connection site can be observed for leaking fluid. Additionally, an endoscopy which has multiple sealing elements can be used to isolate the targeted area and locally control temperature, humidity, pressure and/or fluids within the targeted location. Altering these parameters could modify the local environment allowing for improvements for therapeutic treatment either prior, during or after the procedure. This local modifications would allow for better performance to the tissue based on conditions and/or after the intended treatment to reduce inflammation and/or promote blood flow to improve recovery.
When the first and second deployable sealing elements 6208, 6214 are deployed into the colon and expanded, they create a sealed segment 6062 within the colon 6050. Once the sealed segment 6062 is created, a leak test evaluation can be performed thereon. As shown in FIG. 27 , the sealed segment 6062 is located in section A of the colon. However, a person skilled in the art will appreciate that the sealed segment can be positioned in other sections of the colon (e.g., in section B, C, or D in FIG. 26 ), and thus the following discussion is also applicable to such instances.
In use, once the sealed segment 6062 is created, section A of the colon, which includes the connection site 6059, can be inflated to assess for any leaks therein (e.g., at the connection site 6059). As shown, a fluid channel 6213 extends through the inner tube 6210 of the surgical instrument 6202 and has an opening 6230 arranged between the first and second deployable sealing elements 6208, 6214. The opening 6230 is distal to the first seal 6226 created between the inner surface 6051 of the colon 6050 and the first deployed sealing element 6208, and proximal to the second seal 6228 created between the inner surface 6051 of the colon 6050 and the deployed second sealing element 6214. The opening 6230 is configured to allow fluid to pass into and out of the sealed portion 6220, thereby selectively pressurizing section A of the colon 6050. The fluid used to pressurize the sealed portion 6220 can be pressurized fluid that is introduced through a fluid channel 6213 that is in fluid communication with the opening 6230. As a result, the pressurized fluid is expelled into the sealed portion through the opening 6230. In certain embodiments, the fluid is expelled at a controllable rate.
In some embodiments, the fluid 6232 can include a leak assessment fluid (a dye, contrast agent, or florescence agent) that would be visually detectable outside of the colon. In such instances, a first laparoscopic instrument 6312 inserted through a port 630 and into the abdominal cavity can be positioned proximal to the colon 6050 and used to identify any discharge of the leak assessment fluid from the inflated section A of the colon. For example, as illustrated in FIG. 26 and FIG. 27 , the first laparoscopic instrument 6312 can include a camera 6318 configured to detect any leak assessment fluid outside of the colon. For example, the camera 6318 can be used to visually detect (e.g., by a surgeon) any of the leak assessment fluid that passes through the connection 6058. In another embodiment, the camera 6318 can be configured to emit a multi-spectrum wavelength (e.g., near-infrared) if the leak assessment fluid is a florescence agent which must be excited in order to be located. A multi-spectrum wavelength must be applied first to excite the agent within the leak assessment fluid so it becomes visible to the camera. The presence of leak assessment fluid outside the colon can be used to highlight the location of the leak(s) and, in some instances, also highlight the magnitude of the leak(s).
In some embodiments, as shown in FIG. 27 , a second laparoscopic instrument 6314 can be positioned proximal to the colon 6050 and configured to apply agent (e.g., a clot-inducing agent) to one or more leak areas of the colon to form a respective outer seal. The clot-inducing agent can be biologically inert or stable, and have an agent within the fluid, which in direct contact with an agent within the fluid 6232 applied from within the colon 6050, will activate the clot-inducing agent. For example, a plantlet rich plasma can be introduced with leak assessment fluid 6232, and in the event a leak is identified from the laparoscopic side, an oxidized regenerated cellulose (ORC) with or without a freeze dried fibrin and/or thrombin powder could be placed at the leak location via the second laparoscopic instrument 6314. As a result, the plasma would then activate the ORC and the fibrin to form a resilient gel seal. This seal would be treated by the body like a clot or scab and therefore would be remodeled as the body heals. Alternatively, or in addition, an adjunct can be applied laparoscopically to one or more leak area(s) to form a respective outer seal.
As illustrated in FIG. 27 , an observed amount of liquid assessment fluid 6232 a is located on the outside of the colon at, and therefore has leaked through the connection site 6059. This leak is detected by the camera 6318. However, the connection 6061 is not leaking any fluid 6232 while pressurized. In an exemplary embodiment, where a leak is present, the pressure from the fluid 6232 can be relieved and re-applied, or the sealed portion 6062 can remain pressurized, while the adjunct therapy is applied (such as a clotting agent) to insure the adjunct therapy is capable of resisting that level of pressure.
Further, in use, the camera 6215 is arranged endoscopically within the colon 6050 in order to visual/and or operate on the colon 6050 from the intraluminal space. In an exemplary embodiment, a user (e.g., a surgeon) can visually inspect the colon 6050 to determine if a leak is present (e.g., in combination with use of a contrast or fluorescing agent mixed with the inflation fluid), identity inadvertent tears and tissue trauma for repair, or both.
FIG. 28 illustrates a compilation of graphs which are representative of an exemplary leak test evaluation for sections A, B, C, and D of the colon 6050 in FIG. 26 and FIG. 27 . Graph 6402 is representative of the exemplary leak test evaluation on section D of the colon 6050. Line 6404 represents the pressure within section D over the course of the evaluation, a pressure range 6405 is defined between P₁and P_maxand represents the acceptable pressure range for pressurizing section D during the evaluation over time interval t_seal, and line 6406 represents the amount of leak assessment fluid observed outside of the colon during the evaluation. Time interval t_sealrepresents an acceptable duration period to access whether any leaks are present within section D. As illustrated in graph 6402, over time interval t_seal, section D was maintained at a pressure within the pressure range 6405 and no leak assessment fluid was observed outside of section D and therefore no leaks were detected within section D. Graph 6412, which is a representative of section B of the colon 6050 in FIG. 26 , and Graph 6422, which is representative of section C of the colon 6050 in FIG. 26 are similar to graph 6402. That is, lines 6414, 6426 represent the pressure within section C and section B, respectively, over the course of the evaluation, pressure ranges 6415, 6425 represent the acceptable pressure range for pressurizing section C and Section B, respectively over the time interval t_seal, and lines 6416, 6426 represent the amount of leak assessment fluid observed outside of the colon during the evaluation. For the same reasons as section D, no leaks were detected in section C and section B.
Graph 6432 represents the exemplary leak test evaluation of section A of colon 6050 in FIG. 27 . Line 6433 represents the desired pressure within section A over the course of the evaluation, a pressure range 6435 is defined between P₁and Pma_xand represents the acceptable pressure range for pressurizing section A during the evaluation over time interval t_seal, and line 6437 represents the desired amount of leak assessment fluid observed outside of the colon during the evaluation. Time interval t_sealrepresents an acceptable duration period to access whether any leaks are present within section A. As illustrated in graph 6432, over time interval t_seal, section A did not maintain the desired pressure within the pressure range 6435 and leak assessment fluid was observed outside of section A. Line 6434 represents the actual pressure within section A over the course of the evaluation, and line 6436 represents the actual amount of leak assessment fluid observed outside of the colon during the evaluation. As shown in graph 6432, as the pressure 6434 decreased, the observed leak assessment fluid 6436 increased.
In some embodiments, the first laparoscopic instrument 6312, along with the camera 6316, can be configured to detect over-distension of the sealed portion 6062 when a leak test evaluation is being performed. In such embodiments, the over-distension can be measured by 3D structured light scan of colon 6050 prior to pressurization, which can provide a diameter delta limit for 3D surface change of the colon 6050 to prevent inadvertent tissue damage.
In another exemplary embodiment, the arrangement of the sealing elements 6208, 6214 creating a sealed portion 6062 can enable controlled introduction of a fluid or gas within the sealed portion 6062 in order to alter the environment for optimal surgical conditions. Surgical conditions which can affect the efficiency of a procedure can include temperature, pressure, and humidity in order to optimize the environment for tissue dissection. Tissue property values are highly variable, and dependent on time, history, temperature, pressure and hydration. Variations exist between young and old, healthy, diseased and irradiated tissues, which can lead to inefficiencies if constant changes to equipment must be made.
An example of increasing efficiency of a surgical task with a sealed region by altering the conditions within an organ can include the use of bi-polar or mono polar energy applied to tissue for dissection and/or sealing. For bi-polar energy, power is delivered while monitoring the impedance of the tissue, the impedance will go through 3 phases, initially it will decrease impedance for a period of time, then stay at a near constant impedance for a period of time during the desiccating tissue phase until vaporization has occurred and a rapid increase in impedance occurs at which power is stopped. During this type of procedure, tissue that is low in moisture due to a condition of patient, trauma, disease and/or previously treated or altered from treatment can cause the energy cycle to be too short which would reduce the seal and/or alter the intended therapeutic treatment. Modifying the local environmental characteristics (e.g., temperature or humidity) prior or during treatment could improve the efficiency of the energy application and improve sealing and/or dissection optimization of the energy applying instrument.
Another example of increasing surgical efficacy through controlling environmental parameters includes altering the temperature of a region to increase blood flow. Tissue that has low blood flow can alter coagulation properties of that tissue. By modifying the local environments characteristics (e.g., temperature) prior or during treatment could improve the blood flow and improve sealing and/or dissection optimization of an energy applying instrument. Additionally, modifying the local temperature could be done during treatment to reduce blood flow during treatments or targeted areas in which is prone to heavy bleeding.
The instruments disclosed herein can be designed to be disposed of after a single use, or they can be designed to be used multiple times. In either case, however, the instrument can be reconditioned for reuse after at least one use. Reconditioning can include any combination of the steps of disassembly of the instrument, followed by cleaning or replacement of particular pieces and subsequent reassembly. In particular, the instrument can be disassembled, and any number of the particular pieces or parts of the instrument can be selectively replaced or removed in any combination. Upon cleaning and/or replacement of particular parts, the instrument can be reassembled for subsequent use either at a reconditioning facility, or by a surgical team immediately prior to a surgical procedure. Those skilled in the art will appreciate that reconditioning of an instrument can utilize a variety of techniques for disassembly, cleaning/replacement, and reassembly. Use of such techniques, and the resulting reconditioned instrument, are all within the scope of the present application.
Further, in the present disclosure, like-named components of the embodiments generally have similar features, and thus within a particular embodiment each feature of each like-named component is not necessarily fully elaborated upon. Additionally, to the extent that linear or circular dimensions are used in the description of the disclosed systems, devices, and methods, such dimensions are not intended to limit the types of shapes that can be used in conjunction with such systems, devices, and methods. A person skilled in the art will recognize that an equivalent to such linear and circular dimensions can easily be determined for any geometric shape. Sizes and shapes of the systems and devices, and the components thereof, can depend at least on the anatomy of the subject in which the systems and devices will be used, the size and shape of components with which the systems and devices will be used, and the methods and procedures in which the systems and devices will be used.
It will be appreciated that the terms “proximal” and “distal” are used herein with reference to a user, such as a clinician, gripping a handle of an instrument. Other spatial terms such as “front” and “rear” similarly correspond respectively to distal and proximal. It will be further appreciated that for convenience and clarity, spatial terms such as “vertical” and “horizontal” are used herein with respect to the drawings. However, surgical instruments are used in many orientations and positions, and these spatial terms are not intended to be limiting and absolute.
Values or ranges may be expressed herein as “about” and/or from/of “about” one particular value to another particular value. When such values or ranges are expressed, other embodiments disclosed include the specific value recited and/or from/of the one particular value to another particular value. Similarly, when values are expressed as approximations, by the use of antecedent “about,” it will be understood that here are a number of values disclosed therein, and that the particular value forms another embodiment. It will be further understood that there are a number of values disclosed therein, and that each value is also herein disclosed as “about” that particular value in addition to the value itself. In embodiments, “about” can be used to mean, for example, within 10% of the recited value, within 5% of the recited value or within 2% of the recited value.
For purposes of describing and defining the present teachings, it is noted that unless indicated otherwise, the term “substantially” is utilized herein to represent the inherent degree of uncertainty that may be attributed to any quantitative comparison, value, measurement, or other representation. The term “substantially” is also utilized herein to represent the degree by which a quantitative representation may vary from a stated reference without resulting in a change in the basic function of the subject matter at issue.
One skilled in the art will appreciate further features and advantages of the invention based on the above-described embodiments. Accordingly, the invention is not to be limited by what has been particularly shown and described, except as indicated by the appended claims. All publications and references cited herein are expressly incorporated herein by reference in their entirety. Any patent, publication, or information, in whole or in part, that is said to be incorporated by reference herein is only to the extent that the incorporated material does not conflict with existing definitions, statements, or other disclosure material set forth in this document. As such the disclosure as explicitly set forth herein supersedes any conflicting material incorporated herein by reference.

Claims

What is claimed is:

1. A surgical system, comprising:

a surgical instrument configured for endoluminal access, the surgical instrument having:

at least one deployable sealing element operatively coupled to the surgical instrument and configured to move between unexpanded and expanded states, and when in the expanded state, the at least one deployable sealing element is configured to form a first seal at a portion of a natural body lumen or an organ; and

a fluid channel extending through the surgical instrument and having an opening distal to the first seal and configured to allow fluid ingress and egress distal to the portion of the natural body lumen or the organ while the at least one deployable sealing member is in the expanded state thereby selectively pressurizing the natural body lumen or the organ distal to the portion.

2. The surgical system of claim 1, wherein at least one deployable sealing element is configured to expand to contact an internal surface of the natural body lumen or the organ.

3. The surgical system of claim 1, wherein at least one deployable sealing element is an inflatable balloon configured to be filled with a fluid to move from an unexpanded state to an expanded state.

4. The surgical system of claim 1, wherein the surgical instrument further includes an optical sensor arranged at a distal end thereof.

5. The surgical system of claim 1, wherein the surgical instrument further includes a second deployable sealing element coupled to the surgical instrument and distal to the opening of the fluid channel.

6. The surgical system of claim 5, wherein the second deployable sealing element is configured to move between unexpanded and expanded states.

7. The surgical system of claim 6, wherein when in the expanded state, the second deployable sealing element is configured to form a second seal at a second portion of the natural body lumen or the organ.

8. The surgical system of claim 5, wherein the at least one deployable sealing element and the second deployable sealing element can be expanded separately.

9. The surgical system of claim 5, wherein the at least one deployable sealing element and the second deployable sealing element can be expanded simultaneously.

10. The surgical system of claim 1, wherein when pressurized the portion of the natural body lumen or the organ distal to the at least one deployable sealing element has a first pressure, and an area outside of the natural body lumen or the organ has a second pressure, different than the first pressure.

11. A method, comprising:

inserting a surgical instrument into a natural body lumen or an organ, the surgical instrument having a fluid channel extending therethrough and at least one deployable sealing element operatively coupled to the surgical instrument;

expanding a first deployable sealing element of the at least one deployable sealing element from an unexpanded state to an expanded state to form a first seal within the natural body lumen or the organ;

injecting fluid through the fluid channel and into a portion of the natural body lumen or the organ distal to the first seal to thereby inflate the portion of the natural body lumen or the organ; and

pressurizing the portion of the natural body lumen or the organ.

12. The method of claim 11, further comprising expanding at least one deployable sealing element to contact an internal surface of the natural body lumen or the organ.

13. The method of claim 11, further comprising filing at least one deployable sealing element with a fluid to move from an unexpanded state to the expanded state.

14. The method of claim 11, wherein the at least one deployable sealing element includes a second deployable sealing element coupled to the surgical instrument and positioned distal to the first deployable sealing element, the method further comprises transitioning the second deployable sealing element from an unexpanded state to an expanded state.

15. The method of claim 14, further comprising expanding the second deployable sealing element to an expanded state to form a second seal within the natural body lumen, wherein the portion of the natural body lumen or the organ is located between the first and second deployable sealing elements.

16. The method of claim 11, further comprising creating a pressure differential within the portion of the natural body lumen or organ relative to an area arranged outside of the natural body lumen or the organ.

17. The method of claim 15, further comprising expanding the first deployable sealing element and the second deployable sealing element separately.

18. The method of claim 15, further comprising expanding at least one deployable seal and the second deployable seal simultaneously.