WO2018005928A1 - Systèmes et procédés de commande d'instrument souple assistée par ordinateur - Google Patents

Systèmes et procédés de commande d'instrument souple assistée par ordinateur Download PDF

Info

Publication number
WO2018005928A1
WO2018005928A1 PCT/US2017/040214 US2017040214W WO2018005928A1 WO 2018005928 A1 WO2018005928 A1 WO 2018005928A1 US 2017040214 W US2017040214 W US 2017040214W WO 2018005928 A1 WO2018005928 A1 WO 2018005928A1
Authority
WO
WIPO (PCT)
Prior art keywords
flexible elongate
elongate instrument
mode
computer
rigidity
Prior art date
Application number
PCT/US2017/040214
Other languages
English (en)
Inventor
Nicola Diolaiti
Samuel Y. CHANG
Vincent Duindam
Original Assignee
Intuitive Surgical Operations, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intuitive Surgical Operations, Inc. filed Critical Intuitive Surgical Operations, Inc.
Priority to US16/311,514 priority Critical patent/US11344376B2/en
Priority to EP24154210.9A priority patent/EP4335403A3/fr
Priority to CN202211616017.0A priority patent/CN116115337A/zh
Priority to EP17821332.8A priority patent/EP3478149B1/fr
Priority to CN201780039384.XA priority patent/CN109310287B/zh
Publication of WO2018005928A1 publication Critical patent/WO2018005928A1/fr
Priority to US17/733,446 priority patent/US20220273377A1/en

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00006—Operational features of endoscopes characterised by electronic signal processing of control signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00064—Constructional details of the endoscope body
    • A61B1/00071—Insertion part of the endoscope body
    • A61B1/00078—Insertion part of the endoscope body with stiffening means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00147—Holding or positioning arrangements
    • A61B1/0016—Holding or positioning arrangements using motor drive units
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/005—Flexible endoscopes
    • A61B1/0051—Flexible endoscopes with controlled bending of insertion part
    • A61B1/0057—Constructional details of force transmission elements, e.g. control wires
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/35—Surgical robots for telesurgery
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/37—Master-slave robots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A61B34/71—Manipulators operated by drive cable mechanisms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A61B34/74—Manipulators with manual electric input means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/06—Measuring instruments not otherwise provided for
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M25/00—Catheters; Hollow probes
    • A61M25/01—Introducing, guiding, advancing, emplacing or holding catheters
    • A61M25/0105—Steering means as part of the catheter or advancing means; Markers for positioning
    • A61M25/0133—Tip steering devices
    • A61M25/0147—Tip steering devices with movable mechanical means, e.g. pull wires
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/24—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes
    • G02B23/2476—Non-optical details, e.g. housings, mountings, supports
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
    • A61B17/00234—Surgical instruments, devices or methods, e.g. tourniquets for minimally invasive surgery
    • A61B2017/00292—Surgical instruments, devices or methods, e.g. tourniquets for minimally invasive surgery mounted on or guided by flexible, e.g. catheter-like, means
    • A61B2017/00336—Surgical instruments, devices or methods, e.g. tourniquets for minimally invasive surgery mounted on or guided by flexible, e.g. catheter-like, means with a protective sleeve, e.g. retractable or slidable
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2051—Electromagnetic tracking systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2055—Optical tracking systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2059—Mechanical position encoders
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2061—Tracking techniques using shape-sensors, e.g. fiber shape sensors with Bragg gratings
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B2034/301—Surgical robots for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B2034/305—Details of wrist mechanisms at distal ends of robotic arms
    • A61B2034/306—Wrists with multiple vertebrae
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A61B34/74—Manipulators with manual electric input means
    • A61B2034/742—Joysticks
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/06—Measuring instruments not otherwise provided for
    • A61B2090/061—Measuring instruments not otherwise provided for for measuring dimensions, e.g. length

Definitions

  • the present disclosure is directed to systems and methods for computer-assisted medical procedures, and more particularly to systems and methods for controlling a flexible elongate instrument.
  • Minimally invasive medical techniques are intended to reduce the amount of tissue that is damaged during medical procedures, thereby reducing patient recovery time, discomfort, and harmful side effects.
  • Such minimally invasive techniques may be performed through natural orifices in a patient anatomy or through one or more surgical incisions. Through these natural orifices or incisions physicians may insert minimally invasive medical instruments (including surgical, diagnostic, therapeutic, or biopsy instruments) to reach a tissue location. To assist with reaching the target location, the location and movement of the medical instruments may be correlated with pre-operative or intra-operative images of the patient anatomy.
  • the instruments may navigate natural or surgically created passageways in anatomic systems such as the intestines, the kidneys, the brain, the heart, the circulatory system, lungs, urethras, arteries, umbilical lines, and/or the like. Medical tools are needed that are flexible enough to safely navigate the tight beds of the anatomic passageways while providing sufficient rigidity to ensure a predictable performance direction when deployed from a delivery catheter toward the target tissue.
  • a computer-assisted medical system comprises a flexible elongate instrument.
  • the flexible elongate instrument comprises a plurality of wires extending from a proximal end of the flexible elongate instrument to a distal end of the flexible elongate instrument. Each wire of the plurality of wires may be used to steer the distal end.
  • the system also comprises a control system coupled to the flexible elongate instrument.
  • the control system is configured to monitor movement of the flexible elongate instrument along a longitudinal central axis and determine an extent of motion of the flexible elongate instrument in a first direction along the longitudinal central axis based on the monitoring.
  • the control system is also configured to alter a rigidity of the flexible elongate instrument based on a rigidity profile relative to the extent of motion by adjusting one or more forces applied by the plurality of wires to the distal end of the flexible elongate instrument.
  • a method of controlling a medical device comprises monitoring commanded movement of a flexible elongate instrument.
  • the flexible elongate instrument comprises a plurality of wires extending from a proximal end of the flexible elongate instrument to a distal end of the flexible elongate instrument. Each of the plurality of wires may be used to steer the distal end of the flexible elongate instrument.
  • the commanded movement is provided through an input device.
  • the method also comprises determining an extent of a commanded motion of the flexible elongate instrument based on the monitoring and altering a rigidity of the flexible elongate instrument based on a rigidity profile relative to the extent of the motion by adjusting one or more forces applied by each of the plurality of wires to the distal end of the flexible elongate instrument.
  • a non-transitory machine-readable medium comprises a plurality of machine-readable instructions which, when executed by one or more processors associated with a flexible elongate instrument, are adapted to cause the one or more processors to perform a method.
  • the method comprises monitoring movement of the flexible elongate instrument.
  • the flexible elongate instrument comprises a plurality of wires extending from a proximal end of the flexible elongate instrument to a distal end of the flexible elongate instrument. Each of the plurality of wires may be used to steer the distal end of the flexible elongate instrument.
  • the method also comprises determining an extent of a motion of the flexible elongate instrument based on the monitoring and altering a rigidity of the flexible elongate instrument based on a rigidity profile relative to the extent of the motion by adjusting one or more forces applied by each of the plurality of wires to the distal end of the flexible elongate instrument.
  • a computer-assisted medical system comprises a flexible elongate instrument comprising a plurality of wires extending from a proximal end of the flexible elongate instrument to a distal end of the flexible elongate instrument. Each wire of the plurality of wires may be used to steer the distal end.
  • the system also comprises a control system coupled to the flexible elongate instrument. The control system is configured to monitor movement of the flexible elongate instrument and determine a mode of operation for the flexible elongate instrument. The mode of operation corresponds to one of a retraction mode, an insertion mode, and a parking mode.
  • the control system In response to a determination that the mode of operation is the retraction mode, the control system is configured to determine an extent of a retraction of the flexible elongate instrument based on the monitoring and reduce a rigidity of the flexible elongate instrument based on the extent of the retraction by adjusting one or more forces applied by the plurality of wires to the distal end of the flexible elongate instrument. In response to a determination that the mode of operation is the insertion mode, the control system is configured to determine an extent of a insertion of the flexible elongate instrument based on the monitoring.
  • the control system is also configured to increase the rigidity of the flexible elongate instrument based on the extent of the insertion by adjusting the one or more forces applied by the plurality of wires to the distal end of the flexible elongate instrument.
  • the control system is configured to determine an extent of a parking of the flexible elongate instrument based on the monitoring and increase or decrease the rigidity of the flexible elongate instrument based on the extent of the parking by adjusting the one or more forces applied by the plurality of wires to the distal end of the flexible elongate instrument.
  • FIG. 1 is an exemplary teleoperated medical system.
  • FIG. 2A is an exemplary medical instrument system.
  • FIG. 2B is an exemplary distal end of a flexible elongate instrument.
  • FIG. 3 is an exemplary actuator.
  • FIGS. 4A and 4B illustrate side views of a patient coordinate space including a medical instrument mounted on an insertion assembly.
  • FIGS. 5A and 5B are exemplary rigidity profiles for a flexible elongate instrument.
  • FIG. 6 is a flowchart illustrating an exemplary method for controlling a flexible body.
  • FIG. 7 illustrates a rigidity multiplier profile applied during an exemplary application of the method of FIG. 6 shown in relation to a retraction distance.
  • FIGS. 9-10 illustrates sub-mode rigidity profiles.
  • position refers to the location of an object or a portion of an object in a three-dimensional space (e.g., three degrees of translational freedom along Cartesian x-, y-, and z-coordinates).
  • position refers to the location of an object or a portion of an object in a three-dimensional space (e.g., three degrees of translational freedom along Cartesian x-, y-, and z-coordinates).
  • orientation refers to the rotational placement of an object or a portion of an object (three degrees of rotational freedom - e.g., roll, pitch, and yaw).
  • pose refers to the position of an object or a portion of an object in at least one degree of translational freedom and to the orientation of that object or portion of the object in at least one degree of rotational freedom (up to six total degrees of freedom).
  • shape refers to a set of poses, positions, or orientations measured along an object. Components described as “coupled” may be electrically or mechanically directly coupled, or they may be indirectly coupled via one or more intermediate components.
  • Long, flexible devices such as catheters, have a small cross section that include one or more hollow openings along their length and are desirable for inserting medical instruments and/or devices into narrow passageways of a patient, such as airways in lungs and/or the like. Steering the leading or distal end of a catheter through passageways makes it easier to control which passageways the catheter enters.
  • One possible way of steering the leading end of these catheters is by actuators pushing and/or pulling a series of wires that run along the long, flexible device. Using these wires, an operator and/or an automated system is able to pull and/or push harder on one side of the leading end relative to another so as bend the leading end in a desired direction relative to the rest of the device.
  • leading end In addition to steering the leading end, pushing and/or pulling the wires harder also affects how rigid or stiff the leading end becomes. This is not generally a problem when the bend in the leading end aligns with the passageway in which the leading end is inserted, but this may not be the case when the leading end is further inserted into and/or retracted from the passageways without aligning it with the current passageway.
  • the leading end could be bent over into a "hook” shape to insert the leading end around a curve in the passageways, but when the leading end is later retracted, this "hook” may prevent the leading end from being negotiated through a straighter and/or differently curved region of the passageways.
  • One way of accounting for the possibility of hooking is to reverse the steering used to insert the leading end when the leading end is later retracted.
  • Another approach is to reduce the rigidity of the leading end so that as it is retracted, it can more easily adjust its shape based on the current passageway.
  • the rigidity of the leading end may be adjusted by changing the amount of force applied by the wires used to steer the leading end. For example, to reduce the rigidity, the amount of force applied to the wires is reduced accordingly.
  • the amount of force applied by the wires may be controlled using motors and/or actuators that can push and/or pull on the wires.
  • the rigidity can be adjusted by altering a force and/or a torque applied by the motors/actuators to control the pushing and/or pulling of one or more of the wires in the catheter.
  • a controlled reduction in the rigidity may reduce the likelihood of disorientation of an operator controlling movement of the catheter.
  • Adjustment of rigidity may proceed according to a predefined plan or profile of how quickly the rigidity is adjusted.
  • a rigidity profile may have different zones of behavior for the rigidity of the catheter; for example, the rigidity may be decreased faster in one zone than in other zones.
  • the rigidity may be increased to a maximum allowable rigidity for the catheter.
  • the rigidity can be adjusted to higher or lower levels. Rigidity may be increased when the catheter is not moving much in order to provide stability to an instalment being deployed through the catheter while a procedure is being performed.
  • FIG. 1 is a simplified diagram of a teleoperated medical system 100 according to some embodiments.
  • teleoperated medical system 100 may be suitable for use in, for example, surgical, diagnostic, therapeutic, or biopsy procedures.
  • medical system 100 generally includes a teleoperational manipulator assembly 102 for operating a medical instrument 104 in performing various procedures on a patient P.
  • Teleoperational manipulator assembly 102 is mounted to or near an operating table T.
  • a master assembly 106 allows an operator (e.g., a surgeon, a clinician, or a physician O as illustrated in FIG. 1) to view the interventional site and to control teleoperational manipulator assembly 102.
  • Master assembly 106 may be located at a surgeon's console which is usually located in the same room as operating table T, such as at the side of a surgical table on which patient P is located. However, it should be understood that physician O can be located in a different room or a completely different building from patient P. Master assembly 106 generally includes one or more control devices for controlling teleoperational manipulator assembly 102.
  • the control devices may include any number of a variety of input devices, such as joysticks, trackballs, scroll wheels, data gloves, trigger-guns, hand-operated controllers, voice recognition devices, body motion or presence sensors, and/or the like which may be operated by the physician O to provide a commanded motion.
  • control devices may be provided with the same degrees of freedom as the associated medical instrument 104. In this manner, the control devices provide physician O with telepresence or the perception that the control devices are integral with medical instruments 104.
  • control devices may have more or fewer degrees of freedom than the associated medical instrument 104 and still provide physician O with telepresence.
  • the control devices may optionally be manual input devices which move with six degrees of freedom, and which may also include an actuatable handle for actuating instruments (for example, for closing grasping jaws, applying an electrical potential to an electrode, delivering a medicinal treatment, and/or the like).
  • Teleoperational manipulator assembly 102 supports medical instrument 104 and may include a kinematic structure of one or more non-servo controlled links (e.g., one or more links that may be manually positioned and locked in place, generally referred to as a set-up structure) and a teleoperational manipulator.
  • Teleoperational manipulator assembly 102 may optionally include a plurality of actuators or motors that drive inputs on medical instrument 104 in response to commands from the control system (e.g., a control system 112).
  • the actuators may optionally include drive systems that when coupled to medical instrument 104 may advance medical instrument 104 into a naturally or surgically created anatomic orifice.
  • Other drive systems may move the distal end of medical instrument 104 in multiple degrees of freedom, which may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and in three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes).
  • degrees of linear motion e.g., linear motion along the X, Y, Z Cartesian axes
  • rotational motion e.g., rotation about the X, Y, Z Cartesian axes
  • the actuators can be used to actuate an articulable end effector of medical instrument 104 for grasping tissue in the jaws of a biopsy device and/or the like.
  • Actuator position sensors such as resolvers, encoders, potentiometers, and other mechanisms may provide sensor data to medical system 100 describing the rotation and orientation of the motor shafts. This position sensor data may be used to determine motion of the objects manipulated by the actuators.
  • Teleoperated medical system 100 may include a sensor system 108 with one or more sub-systems for receiving information about the instruments of teleoperational manipulator assembly 102.
  • Such sub-systems may include a position/location sensor system (e.g., an electromagnetic (EM) sensor system); a shape sensor system for determining the position, orientation, speed, velocity, pose, and/or shape of a distal end and/or of one or more segments along a flexible body that may make up medical instrument 104; and/or a visualization system for capturing images from the distal end of medical instrument 104.
  • EM electromagnetic
  • Teleoperated medical system 100 also includes a display system 110 for displaying an image or representation of the surgical site and medical instrument 104 generated by sub-systems of sensor system 108.
  • Display system 110 and master assembly 106 may be oriented so physician O can control medical instrument 104 and master assembly 106 with the perception of telepresence.
  • medical instrument 104 may have a visualization system (discussed in more detail below), which may include a viewing scope assembly that records a concurrent or real-time image of a surgical site and provides the image to the operator or physician O through one or more displays of medical system 100, such as one or more displays of display system 110.
  • the concurrent image may be, for example, a two or three dimensional image captured by an endoscope positioned within the surgical site.
  • the visualization system includes endoscopic components that may be integrally or removably coupled to medical instrument 104. However in some embodiments, a separate endoscope, attached to a separate manipulator assembly may be used with medical instrument 104 to image the surgical site.
  • the endoscope may include one or more mechanisms for cleaning one or more lenses of the endoscope when the one or more lenses become partially and/or fully obscured by fluids and/or other materials encountered by the endoscope.
  • the one or more cleaning mechanisms may optionally include an air and/or other gas delivery system that is usable to emit a puff of air and/or other gasses to blow the one or more lenses clean. Examples of the one or more cleaning mechanisms are discussed in more detail in International Publication No. WO/2017/025465 (filed August 11, 2016) (disclosing "Systems and Methods for Cleaning an Endoscopic Instrument"), which is incorporated by reference herein in its entirety.
  • the visualization system may be implemented as hardware, firmware, software or a combination thereof which interact with or are otherwise executed by one or more computer processors, which may include the processors of a control system 112.
  • Display system 110 may also display an image of the surgical site and medical instruments captured by the visualization system.
  • teleoperated medical system 100 may configure medical instrument 104 and controls of master assembly 106 such that the relative positions of the medical instruments are similar to the relative positions of the eyes and hands of physician O.
  • physician O can manipulate medical instrument 104 and the hand control as if viewing the workspace in substantially true presence.
  • true presence it is meant that the presentation of an image is a true perspective image simulating the viewpoint of a physician that is physically manipulating medical instrument 104.
  • display system 110 may present images of a surgical site recorded pre-operatively or intra-operatively using image data from imaging technology such as, computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and/or the like.
  • CT computed tomography
  • MRI magnetic resonance imaging
  • OCT optical coherence tomography
  • thermal imaging impedance imaging
  • laser imaging nanotube X-ray imaging
  • display system 110 may present images of a surgical site recorded pre-operatively or intra-operatively using image data from imaging technology such as, computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and/or the like.
  • the pre-operative or intra-operative image data may be presented as two-dimensional, three-dimensional, or four-
  • display system 110 may display a virtual navigational image in which the actual location of medical instrument 104 is registered (i.e., dynamically referenced) with the preoperative or concurrent images/model. This may be done to present the physician O with a virtual image of the internal surgical site from a viewpoint of medical instrument 104.
  • the viewpoint may be from a tip of medical instrument 104.
  • An image of the tip of medical instrument 104 and/or other graphical or alphanumeric indicators may be superimposed on the virtual image to assist physician O controlling medical instrument 104.
  • medical instrument 104 may not be visible in the virtual image.
  • display system 110 may display a virtual navigational image in which the actual location of medical instrument 104 is registered with preoperative or concurrent images to present the physician O with a virtual image of medical instrument 104 within the surgical site from an external viewpoint.
  • An image of a portion of medical instrument 104 or other graphical or alphanumeric indicators may be superimposed on the virtual image to assist physician O in the control of medical instrument 104.
  • representations of data points may be rendered to display system 110.
  • measured data points, moved data points, registered data points, and other data points described herein may be displayed on display system 110 in a visual representation.
  • the data points may be visually represented in a user interface by a plurality of points or dots on display system 110 or as a rendered model, such as a mesh or wire model created based on the set of data points.
  • the data points may be color coded according to the data they represent.
  • a visual representation may be refreshed in display system 110 after each processing operation has been implemented to alter data points.
  • Teleoperated medical system 100 may also include control system 112.
  • Control system 112 includes at least one memory and at least one computer processor (not shown) for effecting control between medical instrument 104, master assembly 106, sensor system 108, and display system 110.
  • Control system 112 also includes programmed instructions (e.g., a non- transitory machine-readable medium storing the instructions) to implement some or all of the methods described in accordance with aspects disclosed herein, including instructions for providing information to display system 110. While control system 112 is shown as a single block in the simplified schematic of FIG.
  • control system 112 may include two or more data processing circuits with one portion of the processing optionally being performed on or adjacent to teleoperational manipulator assembly 102, another portion of the processing being performed at master assembly 106, and/or the like.
  • the processors of control system 112 may execute instructions comprising instruction corresponding to processes disclosed herein and described in more detail below. Any of a wide variety of centralized or distributed data processing architectures may be employed. Similarly, the programmed instructions may be implemented as a number of separate programs or subroutines, or they may be integrated into a number of other aspects of the teleoperational systems described herein.
  • control system 112 supports wireless communication protocols such as Bluetooth, IrDA, HomeRF, IEEE 802.11, DECT, and Wireless Telemetry.
  • control system 112 may receive force and/or torque feedback from medical instrument 104. Responsive to the feedback, control system 112 may transmit signals to master assembly 106. In some examples, control system 112 may transmit signals instructing one or more actuators of teleoperational manipulator assembly 102 to move medical instrument 104. Medical instrument 104 may extend into an internal surgical site within the body of patient P via openings in the body of patient P. Any suitable conventional and/or specialized actuators may be used. In some examples, the one or more actuators may be separate from, or integrated with, teleoperational manipulator assembly 102. In some embodiments, the one or more actuators and teleoperational manipulator assembly 102 are provided as part of a teleoperational cart positioned adjacent to patient P and operating table T.
  • Control system 112 may optionally further include a virtual visualization system to provide navigation assistance to physician O when controlling medical instrument 104 during an image-guided surgical procedure.
  • Virtual navigation using the virtual visualization system may be based upon reference to an acquired preoperative or intraoperative dataset of anatomic passageways.
  • the virtual visualization system processes images of the surgical site imaged using imaging technology such as computerized tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and/or the like.
  • CT computerized tomography
  • MRI magnetic resonance imaging
  • fluoroscopy thermography
  • ultrasound ultrasound
  • OCT optical coherence tomography
  • thermal imaging impedance imaging
  • laser imaging laser imaging
  • nanotube X-ray imaging and/or the like.
  • Software which may be used in combination with manual inputs, is used to convert the recorded images into segmented two dimensional or three dimensional composite representation of a partial or an entire anatomic organ or anatomic region.
  • An image data set is associated with the composite representation.
  • the composite representation and the image data set describe the various locations and shapes of the passageways and their connectivity.
  • the images used to generate the composite representation may be recorded preoperatively or intra-operatively during a clinical procedure.
  • a virtual visualization system may use standard
  • the composite representation and any virtual images generated by the composite representation may represent the static posture of a deformable anatomic region during one or more phases of motion (e.g., during an inspiration/ expiration cycle of a lung).
  • sensor system 108 may be used to compute an approximate location of medical instrument 104 with respect to the anatomy of patient P.
  • the location can be used to produce both macro-level (external) tracking images of the anatomy of patient P and virtual internal images of the anatomy of patient P.
  • the system may implement one or more electromagnetic (EM) sensor, fiber optic sensors, and/or other sensors to register and display a medical implement together with preoperatively recorded surgical images. , such as those from a virtual visualization system, are known.
  • EM electromagnetic
  • Teleoperated medical system 100 may further include optional operations and support systems (not shown) such as illumination systems, steering control systems, irrigation systems, and/or suction systems.
  • teleoperated medical system 100 may include more than one teleoperational manipulator assembly and/or more than one master assembly. The exact number of
  • Master assembly 106 may be collocated or they may be positioned in separate locations. Multiple master assemblies allow more than one operator to control one or more teleoperational manipulator assemblies in various combinations.
  • FIG. 2A is a simplified diagram of a medical instrument system 200 according to some embodiments.
  • medical instrument system 200 may be used as medical instrument 104 in an image-guided medical procedure performed with teleoperated medical system 100.
  • medical instrument system 200 may be used for non- teleoperational exploratory procedures or in procedures involving traditional manually operated medical instruments, such as endoscopy.
  • medical instrument system 200 may be used to gather (i.e., measure) a set of data points corresponding to locations within anatomic passageways of a patient, such as patient P.
  • Medical instrument system 200 includes elongate device 202, such as a flexible catheter, coupled to a drive unit 204.
  • Elongate device 202 includes a flexible body 216 having proximal end 217 and distal end 218 (or tip portion 218).
  • flexible body 216 has an approximately 3 mm outer diameter. Other flexible body outer diameters may be larger or smaller.
  • Medical instrument system 200 further includes a tracking system 230 for determining the position, orientation, speed, velocity, pose, and/or shape of distal end 218 and/or of one or more segments 224 along flexible body 216 using one or more sensors and/or imaging devices as described in further detail below.
  • tracking system 230 may optionally be implemented as hardware, firmware, software or a combination thereof which interact with or are otherwise executed by one or more computer processors, which may include the processors of control system 112 in FIG. 1.
  • Tracking system 230 may optionally track distal end 218 and/or one or more of the segments 224 using a shape sensor 222.
  • Shape sensor 222 may optionally include an optical fiber aligned with flexible body 216 (e.g., provided within an interior channel (not shown) or mounted externally). In one embodiment, the optical fiber has a diameter of approximately 200 ⁇ . In other embodiments, the dimensions may be larger or smaller.
  • the optical fiber of shape sensor 222 forms a fiber optic bend sensor for determining the shape of flexible body 216.
  • optical fibers including Fiber Bragg Gratings (FBGs) are used to provide strain measurements in structures in one or more dimensions.
  • FBGs Fiber Bragg Gratings
  • Patent Application No. 11/180,389 (filed July 13, 2005) (disclosing "Fiber optic position and shape sensing device and method relating thereto”); U.S. Patent Application No. 12/047,056 (filed on Jul. 16, 2004) (disclosing "Fiber-optic shape and relative position sensing”); and U.S. Patent No. 6,389,187 (filed on Jun. 17, 1998) (disclosing "Optical Fibre Bend Sensor"), which are all incorporated by reference herein in their entireties. Sensors in some embodiments may employ other suitable strain sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and Fluorescence scattering. In some embodiments, the shape of the elongate device may be determined using other techniques.
  • a history of the distal end pose of flexible body 216 can be used to reconstruct the shape of flexible body 216 over the interval of time.
  • tracking system 230 may optionally and/or additionally track distal end 218 using a position sensor system 220.
  • Position sensor system 220 may use any appropriate sensing technology or combination of sensing technologies, such as: OFDR (optical frequency domain reflectometry) techniques such as those using Fiber Bragg gratings, Raleigh scattering, or some other applicable reflection approach; position sensors enabled by EM
  • position sensor system 220 may comprise, or be a component of, an electromagnetic (EM) sensor system including one or more conductive coils that may be subjected to an externally generated electromagnetic field. Each coil of such an EM sensor system used to implement position sensor system 220 then produces an induced electrical signal having characteristics that depend on the position and orientation of the coil relative to the externally generated electromagnetic field.
  • EM electromagnetic
  • position sensor system 220 may be configured and positioned to measure six degrees of freedom, e.g., three position coordinates X, Y, Z and three orientation angles indicating pitch, yaw, and roll of a base point or five degrees of freedom, e.g., three position coordinates X, Y, Z and two orientation angles indicating pitch and yaw of a base point. Further description of a position sensor system is provided in U.S. Patent No. 6,380,732 (filed August 11, 1999) (disclosing "Six- Degree of Freedom Tracking System Having a Passive Transponder on the Object Being
  • tracking system 230 may alternately and/or additionally rely on historical pose, position, or orientation data stored for a known point of an instrument system along a cycle of alternating motion, such as breathing. This stored data may be used to develop shape information about flexible body 216.
  • a series of position sensors such as electromagnetic (EM) sensors similar to the sensors used in some embodiments of position sensor system 220 may be positioned along flexible body 216 and then used for shape sensing.
  • EM electromagnetic
  • a history of data from one or more of these sensors taken during a procedure may be used to represent the shape of elongate device 202, particularly if an anatomic passageway is generally static.
  • Flexible body 216 includes a channel 221 sized and shaped to receive a medical instrument 226.
  • FIG. 2B is a simplified diagram of flexible body 216 with medical instrument 226 extended according to some embodiments.
  • medical instrument 226 may be used for procedures such as surgery, biopsy, ablation, illumination, irrigation, or suction.
  • Medical instrument 226 can be deployed through channel 221 of flexible body 216 and used at a target location within the anatomy.
  • Medical instrument 226 may include, for example, image capture probes, biopsy instruments, laser ablation fibers, and/or other surgical, diagnostic, or therapeutic tools. Medical tools may include end effectors having a single working member such as a scalpel, a blunt blade, an optical fiber, an electrode, and/or the like.
  • medical instrument 226 is a biopsy instrument, which may be used to remove sample tissue or a sampling of cells from a target anatomic location. Medical instrument 226 may be used with an image capture probe also within flexible body 216.
  • medical instrument 226 may be an image capture probe that includes a distal portion with a stereoscopic or monoscopic camera at or near distal end 218 of flexible body 216 for capturing images (including video images) that are processed by a visualization system 231 for display and/or provided to tracking system 230 to support tracking of distal end 218 and/or one or more of the segments 224.
  • the image capture probe may include a cable coupled to the camera for transmitting the captured image data.
  • the image capture instrument may be a fiber-optic bundle, such as a fiberscope, that couples to visualization system 231.
  • the image capture instrument may be single or multi- spectral, for example capturing image data in one or more of the visible, infrared, and/or ultraviolet spectrums.
  • medical instrument 226 may itself be the image capture probe. Medical instrument 226 may be advanced from the opening of channel 221 to perform the procedure and then retracted back into the channel when the procedure is complete. Medical instrument 226 may be removed from proximal end 217 of flexible body 216 or from another optional instrument port (not shown) along flexible body 216. [0046] Medical instalment 226 may additionally house cables, linkages, or other actuation controls (not shown) that extend between its proximal and distal ends to controllably the bend distal end of medical instrument 226. Steerable instruments are described in detail in U.S. Patent No. 7,316,681 (filed on Oct. 4, 2005) (disclosing "Articulated Surgical Instrument for
  • Flexible body 216 may also house cables, linkages, or other steering controls (not shown) that extend between drive unit 204 and distal end 218 to controllably bend distal end 218 as shown, for example, by broken dashed line depictions 219 of distal end 218.
  • at least four cables are used to provide independent "up-down" steering to control a pitch of distal end 218 and "left-right” steering to control a yaw of distal end 281.
  • Steerable elongate devices are described in detail in U.S. Patent Application No. 13/274,208 (filed Oct. 14, 2011) (disclosing "Catheter with Removable Vision Probe"), which is incorporated by reference herein in its entirety.
  • drive unit 204 may include drive inputs that removably couple to and receive power from drive elements, such as actuators, of the teleoperational assembly.
  • medical instrument system 200 may include gripping features, manual actuators, or other components for manually controlling the motion of medical instrument system 200.
  • the actuator(s) can be used to actuate an articulable end effector of medical instrument 226, such as for grasping tissue in the jaws of a biopsy device or the like.
  • One or more actuator position sensors such as resolvers, encoders, potentiometers, and/or other mechanisms may provide sensor data to the teleoperational assembly including position and/or rotation information for the actuators and/or the one or more wires 240. In some examples, this sensor data may be used to determine motion of distal end 218 and/or one or more end effectors being manipulated by the actuators. In some examples, this sensor data may be used to calculate forces and/or torques being applied by wires 240. Actuators are discussed in more detail below.
  • Elongate device 202 may be steerable or, alternatively, the system may be non- steerable with no integrated mechanism for operator control of the bending of distal end 218.
  • one or more lumens, through which medical instruments can be deployed and used at a target surgical location are defined in the walls of flexible body 216.
  • medical instalment system 200 may include a flexible bronchial instrument, such as a bronchoscope or bronchial catheter, for use in examination, diagnosis, biopsy, or treatment of a lung. Medical instrument system 200 is also suited for navigation and treatment of other tissues, via natural or surgically created connected
  • passageways in any of a variety of anatomic systems, including the colon, the intestines, the kidneys and kidney calices, the brain, the heart, the circulatory system including vasculature, and/or the like.
  • the information from tracking system 230 may be sent to a navigation system 232 where it is combined with information from visualization system 231 and/or the preoperatively obtained models to provide the physician or other operator with real-time position information.
  • the real-time position information may be displayed on display system 110 of FIG. 1 for use in the control of medical instrument system 200.
  • control system 116 of FIG. 1 may utilize the position information as feedback for positioning medical instrument system 200.
  • Various systems for using fiber optic sensors to register and display a surgical instrument with surgical images are provided in U.S. Patent Application No.
  • medical instrument system 200 may be teleoperated within medical system 100 of FIG. 1.
  • teleoperational manipulator assembly 102 of FIG. 1 may be replaced by direct operator control.
  • the direct operator control may include various handles and operator interfaces for hand-held operation of the instrument.
  • control system 112 is configured to apply commands to teleoperational manipulator assembly 102 and/or to one or more actuators, which control bending of distal end 218 of flexible body 216 through the use of steer controls, which may include one or more gears, levers, pulleys, cables, wires, rods, bands, and/or the like and/or any combination of these approaches.
  • steer controls which may include one or more gears, levers, pulleys, cables, wires, rods, bands, and/or the like and/or any combination of these approaches.
  • the steering controls are then used to transmit actions from the one or more inputs along the shaft of the surgical instrument and to actuate distal end 218.
  • the commands include one or more instructions stored in a memory. These instructions may be executed by a processor (e.g., a processor of the control system 112).
  • flexible body 216 may be steered within some passageway of patient P. As flexible body 216 makes contact with walls of the passageway, there may be external forces which push against flexible body 216 from the passageway, and vice versa. In some examples, the external forces may be determined using one or more force and/or pressure sensors located on or inside flexible body 216. Depending upon the magnitude of force applied to the pushing and/or pulling on wires 240, a rigidity or stiffness of flexible body 216 may optionally be controlled so that the ability of flexible body to resist and/or reduce the external forces, which may cause flexible body to bend, may optionally be adjusted. In some examples, the higher the magnitude of the force applied to the wires 240, the higher the rigidity exhibited by flexible body 216.
  • FIG. 3 illustrates an exemplary actuator 400, such as an actuator contained within drive unit 204.
  • actuator 400 as shown in FIG. 3, is based on a rotational actuation approach in which a rotating spindle 410 is rotated to actuate a controllable degree of freedom (DOF).
  • Rotating spindle 410 is coupled to a drive shaft 420 which may be the drive shaft of a motor, servo, active actuator, hydraulic actuator, pneumatic actuator, and/or the like and/or any combination of these approaches (not shown).
  • a wire 430 attached to rotating spindle 410 and/or drive shaft 420 may be further wrapped around and/or unwrapped from around rotating spindle 410 and/or drive shaft 420.
  • wire 430 may correspond to any of wires 240-242 and/or wires 310-340.
  • rotation of rotating spindle 410 and drive shaft 420 and the corresponding wrapping and/or unwrapping of wire 430 may result in a pushing force and/or pulling force on wire 430.
  • monitoring a rotation angle and/or rotational velocity of rotating spindle 410 and/or drive shaft 420 may also provide an indication of how far wire 430 is being released and/or pulled.
  • the rotation angle and/or rotational velocity of rotating spindle 410 and/or drive shaft 420 and/or the torque applied by actuator 400 to drive shaft 420 may provide useful feedback on the forces applied to wire 430 and thus the steering to be applied at the distal end (e.g., distal end 218) of medical instrument system 200 by wire 430.
  • distal end 218 which way distal end 218 will bend may depend upon the placement of wire 430 with respect to other wires that are also contributing to steering.
  • releasing or reducing the force in the wires of the catheter body may cause a corresponding reduced stiffness or rigidity in the catheter.
  • applying or increasing a pulling or pushing force in the wires of the catheter body may cause an increase in stiffness or rigidity of the catheter.
  • This stiffness may be the physical stiffness or rigidity of the catheter body material.
  • the material of the catheter body may become stiff er with multiple steering wires pulled concurrently.
  • the stiffness or rigidity of the catheter may be a closed-loop stiffness or rigidity controlled by the control system.
  • the closed-loop catheter control system and methods are described, for example, in U.S. Patent Application No. 13/274,198 (filed October 14, 2011) (disclosing "Catheters with Control Modes for
  • drive shaft 420 may provide force and/or torque feedback information to medical instrument 104 and control system 112.
  • position sensors such as resolvers, encoders, potentiometers, and other mechanisms may provide data to the teleoperational assembly regarding the rotation and/or orientation of drive shaft 420 and/or rotating spindle 410.
  • actuator 400 may be used in conjunction with the wires 240 to respond to external forces acting on distal end 218 and measured by one or more force and/or pressure sensors.
  • an external force detected by the one or more force and/or pressure sensors may optionally be compensated for by actuating a corresponding wire, to steer distal end 218 toward or away from the external force.
  • FIGS. 4 A and 4B are simplified diagrams of side views of a patient coordinate space including a medical instrument mounted on an insertion assembly, according to some embodiments.
  • a surgical environment 500 includes a patient P positioned on platform 502.
  • Patient P may be stationary within the surgical environment in the sense that gross patient movement is limited by sedation, restraint, and/or other means. Cyclic anatomic motion including respiration and cardiac motion of patient P may continue, unless patient is asked to hold his or her breath to temporarily suspend respiratory motion.
  • data may be gathered at a specific phase in respiration and tagged and identified with that phase.
  • the phase during which data is collected may be inferred from physiological information collected from patient P.
  • a point gathering instrument 504 is coupled to an instrument carriage 506.
  • point gathering instrument 504 may use EM sensors, shape-sensors, and/or other sensor modalities.
  • the measurement points are collected from within the passageways of patient P, the points are stored in a data storage device, such as a memory.
  • the set of measured points may be stored in a database that includes at least some, but may include all, of the measured points obtained during the procedure or immediately before the procedure.
  • each of the points may be represented by data comprising coordinates of the point, a timestamp, and/or a relative sensor position or individual sensor ID (when multiple sensors distributed along a length of the point gathering instrument 504 are used to determine the location of several points simultaneously).
  • Instrument carriage 506 is mounted to an insertion stage 508 fixed within surgical environment 500.
  • insertion stage 508 may be movable but may also have a known location (e.g., via a tracking sensor or other tracking device) within surgical environment 500.
  • Instrument carriage 506 may be a component of a teleoperational manipulator assembly (e.g., teleoperational manipulator assembly 102) that couples to point gathering instrument 504 to control insertion motion (i.e., motion along the A axis) and, optionally, motion of distal end 518 of a flexible elongate instrument 510 in multiple directions including yaw, pitch, and roll.
  • flexible elongate instrument 510 corresponds with instrument system 200 and/or flexible body 300.
  • Instrument carriage 506 or insertion stage 508 may include actuators, such as servomotors, (not shown) that control motion of instrument carriage 506 along insertion stage 508.
  • Elongate instrument 510 is coupled to an instrument body 512.
  • Instrument body 512 is coupled and fixed relative to instrument carriage 506.
  • an optical fiber shape sensor 514 is fixed at a proximal point 516 on instrument body 512.
  • proximal point 516 of optical fiber shape sensor 514 may be movable along instrument body 512 and the location of proximal point 516 may be known (e.g., via a tracking sensor or other tracking device). Shape sensor 514 measures a shape from proximal point 516 to another point such as distal end 518 of elongate instrument 510.
  • point gathering instrument 504 may be similar to medical instrument system 200.
  • a position measuring device 520 provides information about the position of the instrument body 512 as it moves on insertion stage 508 along a retraction and/or insertion axis A (such as a direction along the longitudinal central axis of the instrument body).
  • the position measuring device 520 may include resolvers, encoders, potentiometers, and other sensors that determine the rotation and orientation of drive shafts controlling the motion of the instrument carriage 506 and consequently the motion of the instrument body 512.
  • insertion stage 508 is linear.
  • insertion stage 508 may be curved or have a combination of curved and linear sections.
  • FIG. 4A shows instrument body 512 and instrument carriage 506 in a retracted position along insertion stage 508.
  • proximal point 516 is at a position Lo on axis A.
  • an A component of the location of proximal point 516 may be set to a zero and/or another reference value to provide a base reference to describe the position of instrument carriage 506, and thus proximal point 516, on insertion stage 508.
  • distal end 518 of elongate instrument 510 may be positioned just inside an entry orifice of patient P.
  • instrument body 512 and instrument carriage 506 have advanced along the linear track of insertion stage 508 and distal end 518 of flexible elongate instrument 510 has advanced into patient P.
  • the proximal point 516 is at a position Li on the A-axis.
  • encoder and/or other position data from one or more actuators controlling movement of instrument carriage 506 along insertion stage 508 and/or one or more position sensors associated with instrument carriage 506 and/or insertion stage 508 is used to determine the position LA of proximal point 516 relative to position L 0 .
  • position Li may further be used as an indicator of the distance or the insertion depth to which distal end 518 of elongate instrument 510 is inserted into the passageways of the anatomy of patient P.
  • the flexible elongate instrument 510 can be advanced to a position causing the proximal point 516 to be at a position beyond L 1 .
  • Flexible elongate instrument 510 may be advanced from position L 0 , advanced or retracted from any position between L 0 and the advanced position Li, or advanced or retracted from any position beyond L 1 .
  • distal end 518 may have a decreased rigidity so as to reduce the risk of harm to patient P and/or damage to flexible elongate instrument 510. Rigidity in the flexible instrument during retraction could cause a force to be applied by distal end 518 against the walls of the passageways, resulting in abrasion.
  • the rigidity of distal end 518 may be decreased during the retraction such that any contact with the walls of the passageway and resulting force from the contact may be reduced.
  • retraction occurs in the context of making minor position adjustments or small, quick reciprocal motion.
  • an abrupt full slackening of the flexible instrument during retraction may be undesirable because full slackening could change the orientation (i.e., the pointing direction) of the distal end of the instrument. Therefore, in some embodiments, adjusting the rigidity of the flexible instrument gradually based on an elapsed time or distance traveled during a movement mode may be more suitable.
  • the rigidity adjustment may have a stepwise profile with no rigidity adjustment for a period of time until a threshold time, distance, or speed is achieved, followed by full slackening after the threshold is reached.
  • flexible elongate instrument 510 may be parked within passageways of the anatomy of patient P such that it is not being inserted or retracted. While parked, it may be desirable to increase the rigidity, or place the elongate instrument in a "locked" mode in order to provide stability to an instrument deployed through flexible elongate instrument 510 as a procedure is performed. Alternatively, it may be desirable to keep the rigidity relatively constant, decrease the rigidity so that any contact with the walls of the passageway and resulting force from this contact may be reduced or, as with the retraction or insertion, it would be advantageous to control the rigidity of flexible elongate instrument 510 according to a rigidity profile.
  • FIGS. 5A and 5B are exemplary rigidity profiles 600 and 650, respectively, for a flexible elongate instrument, such as flexible body 216 and/or flexible elongate instrument 510.
  • Rigidity profiles 600 and 650 are graphical representations of possible variations of the rigidity or stiffness, illustrated by curves 610 and 660, respectively. Curves 610 and 660 show how the rigidity, designated by k, may be adjusted as a function of a parameter designated by S representing an extent of a movement mode such as insertion, retraction, or park.
  • control system 112 is configured to apply one or more of rigidity profiles 600 and/or 650 to the flexible elongate instrument during a procedure.
  • S may represent a distance or a change in distance, for example a change in the insertion depth or a retraction distance for the flexible elongate instalment or the distance traveled as a percentage of a largest insertion depth value by the flexible elongate instrument through patient passageways.
  • S may represent other variables that may be used to characterize an extent of the retraction, insertion, and/or parking for the flexible elongate instrument.
  • S may represent an elapsed insertion time, elapsed retraction time, elapsed parking time, change in insertion depth, retraction distance, velocity, speed, and/or the like and/or any combination thereof.
  • S may represent an elapsed time since the insertion, retraction, or parking began.
  • Rigidity value k may range between an initial value (e.g., ki, k 3 ) and a final value (e.g., k 2 , k 4 ) over the course of an instrument movement mode such as retraction, insertion or parking.
  • the lowest rigidity value e.g. k 2 , k 3
  • the lowest rigidity value may correspond to a desired nominal value, such as zero or a small but non-zero rigidity.
  • the desired nominal rigidity value may be operator-selected, based on the size of one or more actuators actuating the flexible elongate instrument, based on the ability of the flexible elongate instrument to bend in response to anatomic motions, based on a procedure being performed, and/or the like and/or any combination of these approaches.
  • the rigidity profiles 600 and 650 can be organized in different zones of behavior for the rigidity of the flexible elongate instrument as exemplified by curves 610 and 660.
  • FIG. 5A includes zones 620, 630, 640 and FIG. 5B includes zones 670, 680, 690.
  • Each zone may represent an elapsed S period such as a different distance traveled or a different period of time, and the curves 610 and 660 can represent different rates of change of rigidity value k in different zones.
  • zones 620, 640, 670, 690 may serve as transition zones where the rigidity value k is kept relatively constant as S increases, so that the rigidity profile may smoothly transition at the beginning and ending of a movement mode such as retraction, insertion or parking.
  • the rate of change in rigidity value k is greater.
  • the zones of greatest rigidity rate change occur near the middle of a movement mode, but in other examples, the zone of greatest rigidity rate change may occur at other periods of the movement mode.
  • rigidity profiles 600 and/or 650 may optionally include additional zones (not shown) with corresponding functions of rigidity.
  • a rigidity profile may feature a combination of rigidity profiles 600 and 650.
  • rigidity profiles 600 and/or 650 may optionally include any combination of linear, non-linear, exponential, logarithmic, step, piece-wise, hyperbolic, parabolic, periodic/trigonometric, inverse hyperbolic, polynomial, modular, other monotonic functions, and/or the like and/or any combination thereof.
  • rigidity profiles may also include a hysteresis or memory feature in which the adjustment in stiffness in one direction S (for example when moving from small to large values) is different than in the opposite direction of S.
  • rigidity adjustment profiles may depend in a linear or nonlinear way on variables other than S, such as the derivative of S, the integral of S, or any combination thereof, or on other additional variables.
  • Rigidity profile 600 and/or 650 may be used to adjust forces applied by one or more actuators to one or more wires, such as any of wires 240, 430 used to control the steering of the distal end of the flexible elongate instrument.
  • rigidity profile 600 and/or 650 may be used to adjust a force and/or a torque applied by the one or more actuators to control the pushing and/or pulling of one or more of the one or more wires within the flexible elongate instrument.
  • rigidity profile 600 and/or 650 may implement a scaling factor and/or torque multiplier used to scale a force and/or torque that is applied by the one or more actuators.
  • the scaling factor or torque multiplier may range from 1.0 or near 1.0 for ki to a nominal, possibly non-zero value for k 2 . In some examples, the scaling factor or torque multiplier may range from a nominal, possibly non-zero value for k 3 to 1.0 or near 1.0 for k 4 .
  • rigidity profiles 600 and/or 650 may represent a function that is dependent upon additional factors such as an external force applied to the flexible elongate instrument, a shape of the flexible elongate instrument, a sensitivity of the anatomy forming the passageways, a curvature of the anatomy, one or more operator preferences, and/or the like and/or any combination of these approaches.
  • the rigidity may correspond to an aggregation of the individual forces on each wire, e.g., by average, weighted sum, and/or the like and/or any combination of these
  • FIG. 5A an example of a rigidity profile is shown during the retraction mode of an elongate flexible device.
  • rigidity profile 600 By applying rigidity profile 600 during the retraction mode, the rigidity of the flexible elongate instrument is decreased, and consequently any contact force from the flexible elongate instrument applied to the anatomic tissue, such as a wall of a passageway, may be reduced.
  • the controlled reduction in the rigidity may be particularly suitable when the instrument is being adjusted, being moved in a reciprocal motion, or otherwise being retracted relatively small distances before becoming parked or transitioning to an insertion movement.
  • an instrument with an applied rigidity profile 600 would experience only a slight decrease in rigidity in zone 620, a greater decrease in rigidity in zone 630, and a relatively steady relaxes state in zone 640. More specifically, in a first zone 620, the rigidity is
  • a second zone 630 starting at Si, the rigidity is decreased at a faster rate than at first zone 620 as S further increases and additional retraction occurs with the likelihood of contact between a distal end of the flexible elongate instrument and walls of the passageways increasing. Rigidity may be decreased at a faster rate, causing the instrument to more quickly lose stiffness and therefore reduce the possible damage to the patient that a rigid instrument would cause.
  • a third zone 640 starting at S 2 , the rigidity is tapered to a near constant desired final value, k 2 , as the retraction distance continues to increase.
  • rigidity profile 650 is a mirror image of rigidity profile 600, and may optionally be used to increase the rigidity during the insertion and/or parking of the flexible elongate instrument in order to allow control of the distal end as the flexible elongate instrument advances and/or is held steady within a passageway.
  • an instrument with an applied rigidity profile 650 would experience only a slight increase in rigidity in zone 670, a greater increase in rigidity in zone 680, and a relatively steady rigid state in zone 690.
  • the rigidity is approximately maintained at an initial rigidity, k 3 , or increases slightly above k 3 as S is increased when the insertion and/or parking begins.
  • the rigidity is increased at a faster rate than at first zone 670 as S further increases.
  • the rigidity is increased at a faster rate at second zone 680 as S further increases.
  • the rigidity is tapered to a near constant desired final value, k 4 , as S continues to increase.
  • k 4 may correspond to a desired maximum rigidity.
  • the desired maximum rigidity may be operator-selected, based on the size of one or more actuators actuating the flexible elongate instrument, based on a procedure being performed, a safety ceiling, and/or the like and/or any combination of these approaches.
  • a rigidity profile similar to rigidity profile 600 or rigidity profile 650 may be applied when the flexible instrument enters a parking mode. For example, if the flexible instrument enters the parking mode from a retraction mode, the flexible instrument may be stiffened while in the parking mode. With some rigidity profiles applied during the parking mode, the rigidity may increase or decrease more quickly because the flexible instrument is not moving.
  • first zone 620/670 and/or third zone 640/690 are optional.
  • Rigidity profile 600/650 may have different shapes and/or regions than those depicted in curve 610/660. As shown, rigidity profile 600/650 is a smooth function, such that it is at least continuously differentiable within its zone, or rigidity profile 600/650 may be one continuous function through zones 620/670-640/690. Alternatively, rigidity profile 600/650 is not continuously differentiable at starting and/or ending points between one or more of zones 620/670-640/690. In some examples, rigidity profile 600/650 is a constant at first zone 620/670 at an initial value ki.
  • Rigidity profile 600/650 may optionally include a series of steps, starting at an initial value ki at first zone 620/670, and stepping down or up to a desired final value k 2 at second zone 630/680.
  • shape of curve 610/660 is representative of a logistic function.
  • rigidity profile 600/650 is a monotonic decreasing or increasing function to ensure that there is a decrease, increase and/or no change in the rigidity throughout the motion of the flexible elongate instrument.
  • FIG. 6 is a flowchart illustrating an exemplary method 700 for controlling a flexible body, such as flexible body 216 and/or flexible elongate instrument 510.
  • the method 700 is illustrated in FIG. 6 as a set of operations or processes 702-710. Not all of the illustrated processes 702-710 may be performed in all embodiments of method 700. Additionally, one or more processes that are not expressly illustrated in FIG. 6 may be included before, after, in between, or as part of the processes 702-710.
  • one or more of the processes 702-710 may be implemented, at least in part, in the form of executable code stored on non-transitory, tangible, computer readable media that when run by one or more processors (e.g., the processors of control system 112) may cause the one or more processors to perform one or more of the processes 702-710.
  • the method may begin when the flexible body is initiated into a passageway.
  • anatomic passageways may include the intestines, the kidneys, the brain, the heart, the circulatory system, lungs, urethras, arteries, umbilical lines, and/or the like.
  • movement of the flexible body is monitored. Movement of the flexible body may be measured by a sensor system, such as sensor system 108.
  • a sensor system such as sensor system 108.
  • One or more sensors such as position sensor system 220, shape sensor 222, or position measuring device 520 may be used to determine a position of the flexible body. The position may correspond to an insertion depth, such as the position of instrument body 512 along axis A.
  • the monitoring may occur continuously, periodically, and/or at set markers.
  • process 702 may be performed concurrently with other steps in method 700.
  • movement can be determined based on commanded motion from an input device such as those described with reference to master assembly 106.
  • detection of an input device which would correlate with insertion or retraction can be used to determine movement.
  • a user can provide an input by depressing a button or clicking on a button on a touchscreen to indicate an intended change in direction.
  • Monitoring the movement of the flexible body may include recording state properties of the flexible body such as a current position, a current time, and/or a current control configuration (e.g., current rigidity caused by closed-loop steering wire control) for the flexible body.
  • monitoring the movement of the flexible body may include recording an input magnitude of the input device determined by a measure of size, speed, or velocity of the operator input at the control device, time an input device has been actuated in one direction, pressure placed on an input device, loss of contact between an operators hand and the input device, and/or the like.
  • Recording the current position may include measuring an insertion depth, such as the position of instrument body 512 along axis A.
  • Recording the current rigidity may include measuring one or more forces applied by one or more actuators to one or more wires, such as any of wires 240-242, wires 310-340, and/or wire 430, used to control the steering of the distal end, such as distal end 218, of the flexible body.
  • the rigidity may correspond to an aggregation of the individual forces on each wire, e.g., by average, weighted sum, and/or the like.
  • Measuring the rigidity may also include taking into account external forces applied to flexible body, which may be measured by one or more force and/or pressure sensors, such as one or more strain gauges.
  • measuring the rigidity may also include taking into account a measured shape of the distal end, a moment of inertia, Young's modulus, the length of the bending portion of the distal end, and/or the like and/or any combination of these approaches.
  • Measuring the rigidity may include measuring a motor current or an applied actuator torque, such as the torque applied by the actuator to drive shaft 420, used to control the force applied to the one or more wires.
  • a rigidity profile of a previous mode may be used to determine the current rigidity based on monitored movement during process 702.
  • a mode of operation for the flexible body is determined.
  • the mode may correspond to one of retraction, insertion, and parking modes.
  • the mode may be determined based on the movement monitored during process 702.
  • the mode determination may be based on elapsed time, velocity, distance traveled of the flexible body, and/or the like and/or any combination thereof.
  • Distance traveled may be equal to the displacement between a current position and the insertion depth recorded during process 702.
  • the current position may correspond to an insertion depth of the flexible body.
  • the elapsed time may be equal to the difference between a current time and the time recorded during process 702.
  • the velocity may be calculated by a sensor, using numerical differentiation, dividing a distance traveled by an elapsed time, and/or by exponential smoothing of velocity sample values, low pass filtering, and/or any combination of these approaches.
  • Numerical differentiation may include backward difference approximation and/or the like.
  • the mode determination may be based on input magnitude of the input device determined by a measure of size, speed, or velocity of the operator input at the control device, time an input device has been actuated in one direction, pressure placed on an input device, loss of contact between an operators hand and the input device, and/or the like.
  • the movement mode may be determined to be the insertion mode when the velocity of the flexible body exceeds a threshold value in an insertion direction or an input device is actuated at a velocity in an insertion direction over a threshold value. Additionally or alternatively, the mode may be determined to be the insertion mode when the velocity of the flexible body or the velocity at which an input device is actuated exceeds a threshold value in the insertion direction for a threshold time. In some examples, the threshold time is between 1 and 10 seconds. In some examples, the threshold velocity is between .1-5 millimeters per second in the insertion direction. In some examples, the flexible body is considered to be in the insertion mode until the velocity of the flexible body in the retraction direction exceeds a threshold value. In some examples, the actuation of the input device is considered to be in the insertion mode until the velocity of the actuation of the input device in the retraction direction exceeds a threshold value.
  • the mode may be determined to be the retraction mode when the velocity of the flexible body exceeds a threshold value in a retraction direction or an input device is actuated at a velocity in a retraction direction over a threshold value. Additionally or alternatively, the mode may be determined to be the retraction mode when the velocity of the flexible body or the velocity at which an input device is actuated exceeds a threshold value in a retraction direction for a threshold time.
  • the threshold time is between 1 and 10 seconds.
  • the threshold velocity is between .1-5 millimeters per second in a retraction direction.
  • the flexible body is considered to be in the retraction mode until the velocity of the flexible body in the insertion direction exceeds a threshold value.
  • the actuation of the input device is considered to be in the insertion mode until the velocity of the actuation of the input device in the retraction direction exceeds a threshold value.
  • the mode may be determined to be the retraction mode when the retracted distance traveled since the last forward motion exceeds a threshold value such as 5 or 10 mm.
  • the mode may be determined to be in the retraction mode when the flexible body is commanded via the user input to retract a distance since the last forward motion that exceeds a threshold value such as 5 or 10 mm.
  • the mode may be determined to be the parking mode when the magnitude of the velocity of the flexible body falls below a threshold value or when the input device is actuated at a velocity that falls below a threshold value. Additionally or alternatively, the mode may be determined to be the parking mode when the velocity of the flexible body or the velocity at which the input device is actuated falls below a threshold value for a threshold time. In some examples, the threshold time is between 1 and 10 seconds. In some examples, the magnitude of the threshold velocity is between .1-5 millimeters per second.
  • the conditions associated with a threshold velocity and/or a threshold time before determining a change in the mode may be to account for perturbations, sensed noise, cyclic anatomic motion (e.g., respiration and cardiac motion), movement within an anatomy, and/or other movement caused by environmental displacement and/or combinations thereof.
  • the mode may be determined to be the parking mode due to loss of operator contact with the input device.
  • an extent of movement (including lack of movement) in the mode of operation is determined.
  • the extent of the movement may include measurement of an elapsed time, distance traveled, and/or the like and/or any combination thereof.
  • the distance traveled may be equal to the displacement between a current insertion depth of the flexible body and the insertion depth recorded during process 702 or a displacement in a current input device position and a previous input device position.
  • the elapsed time may be equal to the difference between a current time and the time recorded during process 702.
  • the rigidity of the flexible body is adjusted based on the extent of the movement.
  • the rigidity may be adjusted by controlling the forces applied by one or more actuators on one more wires used to control the steering of the distal end of the flexible body.
  • Each wire may have the force adjusted by its own force profile that is substantially similar to the rigidity profile.
  • the rigidity may be modified by adjusting a force and/or a torque applied by the one or more actuators to control the pushing and/or pulling of one or more of the one or more wires in the flexible body.
  • an actual force or torque applied by an actuator is adjusted.
  • a maximum force and/or torque limit is placed on a controller controlling each of the one or more actuators.
  • the rigidity of the flexible body may be decreased according to a rigidity profile, such as rigidity profile 600.
  • Ki may be set to the rigidity recorded during process 702 and decreased until the rigidity is at a final rigidity value such as k 2 from rigidity profile 600.
  • the final rigidity may be set at a nominal, possibly non-zero value, to retain at least some control over a bend in the distal end of the flexible body, an instrument deployed at the distal end of the flexible body, and/or the like.
  • the rigidity of the flexible body may be increased according to a rigidity profile, such as rigidity profile 650.
  • K 3 may be set to the rigidity recorded during process 702 and increased until the rigidity is at a final rigidity such as k 4 from rigidity profile 650.
  • the final rigidity may provide sufficient stiffness to the flexible body to steer the advancing flexible body and control the orientation of a medical tool deployed from the flexible body.
  • rigidity may be increased in a locked mode so that the flexible body may be properly articulated or may act as an effective stationary platform.
  • the rigidity may be adjusted based upon additional factors such as by one or more external forces applied to the flexible body, a shape of the flexible body, a sensitivity of the anatomy forming the passageways, a curvature of the anatomy, one or more operator preferences, and/or the like and/or any combination of these approaches.
  • a change in movement mode it is determined whether a change in movement mode occurs. For example, a change from insertion mode to retraction or park mode may be determined or a change from retraction mode to insertion or park mode may be determined.
  • the change in mode may be determined based on an elapsed time, velocity, distance traveled and/or the like and/or any combination thereof.
  • a change in mode is determined based on a change in direction of movement of flexible body.
  • a change in mode is determined when the velocity of the flexible body exceeds a threshold value in a direction opposite of the previously determined movement mode.
  • the conditions associated with a threshold velocity and/or a threshold time before determining a change in the mode may be needed to account for perturbations, sensed noise, cyclic anatomic motion (e.g., respiration and cardiac motion), movement within an anatomy, and/or other movement caused by environmental displacement and/or combinations thereof.
  • the parking mode is optional with the mode of operation being limited to the retraction mode and insertion mode.
  • the method may be repeated starting at process 704. If there is no change in mode, the method may be repeated starting at process 706.
  • the rigidity profile of the first mode may continue as if no change in mode occurred.
  • the rigidity may continue to be increased according to the rigidity profile of the insertion mode as if insertion is continuing without interruption.
  • the rigidity may continue to be increased according to the rigidity profile of the parking mode as if parking is continuing without interruption.
  • the rigidity may continue to be decreased according to the rigidity profile of the retraction mode as if retraction is continuing without interruption. In some examples, during a transition from the parking mode to the retraction mode, the rigidity may continue to be decreased according to the rigidity profile of the parking mode as if parking is continuing without interruption.
  • one or more of the processes 702-710 of method 700 may be implemented, at least in part, in the form of executable code stored on non-transient, tangible, machine-readable media that when run by one or more processors (e.g., control system 112) may cause the one or more processors to perform one or more of the processes 702-710.
  • processors e.g., control system 112
  • one or more elements in embodiments and examples may be implemented in software to execute on the one or more processors. When implemented in software, the elements of the
  • embodiments are essentially the code segments to perform the necessary tasks.
  • the program or code segments can be stored in a non-transitory, processor-readable storage medium or device, including any medium that can store information including an optical medium, semiconductor medium, and magnetic medium.
  • Non-transitory, processor-readable storage device examples include an electronic circuit, a semiconductor device, a semiconductor memory device, a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), a floppy diskette, a CD-ROM, an optical disk, a hard disk, or other storage device.
  • the code segments may be downloaded via computer networks such as the Internet, Intranet, etc.
  • operations of accessing, detecting, initiating, registered, displaying, receiving, generating, determining, moving data points, segmenting, matching, etc. may be performed at least in part by control system 112 or one or more processors thereof.
  • the previously described rigidity profile 600 and/or 650 may implement a scaling factor and/or torque multiplier used to scale a force and/or torque that is applied by the one or more actuators.
  • FIG. 7 illustrates a rigidity multiplier profile 800 applied during an exemplary application of method of retraction over a retraction distance 802 and over a retraction distance 804.
  • a multiplier m may be representative of a scaling factor and/or torque multiplier used to scale a force and/or torque applied by the one or more actuators during processes 708 to adjust the rigidity of the flexible elongate instrument.
  • Rigidity multiplier profile 800 is a graphical representation of a possible variation of the multiplier m, illustrated by curve 810 according to retraction distance, designated by d. Times t 0 -t 7 show the relationships between retraction distances do-d fina i and multiplier m over time.
  • the multiplier m may range from a value mi of 1.0 or near 1.0 at which the flexible instrument is relatively rigid to a value m 2 of zero or near zero at which the flexible instrument is relatively slack.
  • the torque is a function of the multiplier and the maximum torque, x max that may be applied by the actuator, illustrated by Equation 1.
  • the torque determined by Equation 1 may be applied by the respective controllers either as an actual torque to be applied by the respective actuators or as a maximum torque limit to be applied on the respective actuators.
  • ⁇ and x max are replaced by / and max. accordingly; with an actual force or maximum force limit being set.
  • the flexible instrument has been inserted into an anatomic passageway and placed in a park mode with a multiplier set at m 1 .
  • the flexible instrument may be inserted 100 mm into the anatomic passageways when parked.
  • the flexible instrument then enters a retraction mode.
  • the flexible instrument moves from the initial position d 0 and a retracted distance d 1 while the multiplier remains unchanged or approximately unchanged at or near the value m 1 .
  • the extent of the flexible instrument retraction between do and di may be approximately 5 mm.
  • the distance between d 0 and d 1 may be traversed over a time period between t 0 and ti.
  • the multiplier m begins to reduce. Over a retraction distance between di and d 2 , the multiplier m is reduced from the value mi (e.g., relatively rigid) to the value m 2 (e.g., relatively slack).
  • the distance between di and d 2 may be traversed over a time period between t 1 and t 2 .
  • the extent of the flexible instrument retraction may be approximately 10 mm between di and d 2 .
  • the multiplier value remains at m 2 . In one example, the flexible instrument may retract approximately 35 mm between d 2 and d 3 .
  • the distance between d 2 and d 3 may be traversed over a time period between t 2 and t 3 .
  • the flexible instrument exits the retraction mode and enters a park mode.
  • the park mode the flexible instrument remains at the distance d 3 for a time period between t 3 and t 4 .
  • the multiplier value ramps up to mi (e.g., relatively rigid). This transition from multiplier value m 2 to mi may occur abruptly over a short period of time or may follow a more gradual transition profile based on elapsed time.
  • the flexible instrument exits the park mode and enters the retraction mode again.
  • the flexible instrument moves from the parked distance d 3 to a retracted distance d 4 while the multiplier remains unchanged or approximately unchanged at or near the value m 1 .
  • the flexible instrument may retract approximately 5 mm between d 3 and d 4 .
  • the distance between d 3 and d 4 may be traversed over a time period between t 4 and t 5 .
  • the multiplier m begins to reduce.
  • the multiplier m is again reduced from the value mi (e.g., relatively rigid) to the value m 2 (e.g., relatively slack).
  • the distance between d 4 and d 5 may be traversed over a time period between t 5 and t 6 .
  • the flexible instrument may retract approximately 10 mm between d 4 and d 5 .
  • the multiplier value remains at m 2 .
  • the flexible instrument may retract approximately 35 mm between d 2 and d 3 .
  • the distance between d 5 and d fina i may be traversed over a time period between t 6 and t 7 .
  • various sub-modes of the control system may be selected to alter the rigidity of the flexible body based on a sub-mode specific rigidity profile.
  • a sub-mode of a drive movement mode is a full passive sub-mode.
  • the full passive sub-mode may be operator-selected using for example, an identifying input to the control device at the master assembly 106 (e.g. a double or multiple tap of a "passive" button located on the control device), a voice command, or an anatomical gesture command.
  • the rigidity profile 900 is linear but in alternative embodiments, the rigidity profile in full passive sub-mode may be non-linear.
  • the full passive sub-mode may be suitable for use when the procedure is complete, and the operator is retracting the flexible body from the patient anatomy using the master assembly.
  • the full passive sub-mode may be suitable when the flexible catheter is positioned within a tight bend in an anatomic passageway, and the operator would like to relieve rigidity in the flexible catheter (and therefore the anatomic passageway) to, for example, allow passage of a biopsy needle.
  • the control system may remain in the full passive sub-mode until the operator selects a different mode or until another predefined condition occurs.
  • the slow passive rigidity profile reduces torque at a slower rate.
  • the rigidity profile 910 is linear but in alternative embodiments, the rigidity profile in full passive sub-mode may be nonlinear.
  • FIG. 9 illustrates such a non-linear profile 950.
  • the rigidity of the flexible body is reduced as the actuator torque drops quickly between an initial interval (to-ti).
  • the actuator torque is relatively stable between an interval (ti-t 2 ) and drops quickly to a torque of approximately zero over an interval (t 2 -t 3 ).
  • the slow passive sub-mode may be suitable for use when the operator makes slight adjustments to the flexible body position in the patient anatomy where the short incremental distances moved by the flexible body are too small to effectively reduce the actuator torque under a normal drive movement mode such as insertion or retraction.
  • a normal drive movement mode such as insertion or retraction.
  • the control system may remain in the slow passive sub-mode until, for example, the operator releases the passive button.
  • the control system may return to the previous mode of operation by ramping up torque to a previous torque level or to the operating torque level using a linear or non-linear ramp up profile.
  • a sub-mode of a drive movement mode is an insertion clutch sub- mode.
  • the insertion clutch sub-mode may be operator-selected using for example, an
  • the rigidity profile 920 is linear but in alternative embodiments, the rigidity profile in insertion clutch sub-mode may be non-linear over a relatively short time period.
  • the master assembly 106 may not accept user commands to control the manipulator or an attached instrument, and the actuators that provide motion along an insertion path may be disabled.
  • the insertion clutch sub-mode may be suitable for use when the operator wants to quickly remove the flexible body from the patient anatomy in an emergency situation or for another clinical purpose.
  • the operator may manually retract the flexible body by moving an insertion stage of the manipulator assembly 102 to which the instrument 104 is coupled.
  • the control system may remain in the insertion clutch sub-mode until the operator releases the clutch input button or another release condition is met.
  • a sub-mode of a drive movement mode is an auto-passive sub- mode.
  • the auto-passive passive sub-mode may be operator-selected using for example, an identifying input to the control device at the master assembly 106 (e.g. a single tap of a "passive" button located on the control device), a voice command, or an anatomical gesture command.
  • FIG. 10 illustrates a rigidity profile 960 that may be used in the auto-passive sub-mode.
  • the rigidity of the flexible body is reduced as the actuator torque is reduced from an operating torque 905 to a torque of approximately zero based on an input magnitude.
  • the input magnitude may be a measure of size or speed of the operator input at the control device of the master assembly 106 correlating to a measure of desired distance the flexible body is commanded to be moved.
  • the control device is a scroll wheel on the master assembly 106
  • small retractions on the scroll wheel are considered a small input magnitude in the auto-passive sub-mode correlating with a short commanded movement distance
  • large retractions on the scroll wheel are considered a large input magnitude in the auto-passive sub-mode correlating with a large commanded movement distance.
  • the rigidity profile 960 small input magnitudes cause the flexible body to remain under active control, at the operating torque 905. This allows an operator to perform fine adjustments in position without losing the orientation of the distal end of the flexible body. With the rigidity profile 960, large input magnitudes cause the actuator torque to become reduced from an operating torque to a torque of approximately zero. This allows the operator to move the flexible body large distances or remove the flexible body from the patient anatomy with little or no rigidity.
  • the rigidity profile 960 is linear but in alternative embodiments, the rigidity profile in auto-passive sub-mode may be non-linear. The control system may remain in the auto-passive sub-mode during a particular movement mode (e.g.
  • the control system switches out of the auto-passive sub-mode and the actuator torque returns to the operating torque 905 gradually or according to a configurable linear or non-linear transition profile.
  • exiting the auto-passive sub-mode may be accomplished by another type of user input such as a switch.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Physics & Mathematics (AREA)
  • Robotics (AREA)
  • Optics & Photonics (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Signal Processing (AREA)
  • Astronomy & Astrophysics (AREA)
  • General Physics & Mathematics (AREA)
  • Pulmonology (AREA)
  • Anesthesiology (AREA)
  • Hematology (AREA)
  • Mechanical Engineering (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Endoscopes (AREA)
  • Manipulator (AREA)
  • Media Introduction/Drainage Providing Device (AREA)

Abstract

La présente invention concerne un système médical assisté par ordinateur comprenant un instrument allongé souple. L'instrument allongé souple comprend une pluralité de fils s'étendant depuis une extrémité proximale de l'instrument allongé souple vers une extrémité distale de l'instrument allongé souple. Chaque fil de la pluralité de fils peut être utilisé pour diriger l'extrémité distale. Le système comprend également un système de commande accouplé à l'instrument allongé souple. Le système de commande est configuré pour surveiller le mouvement de l'instrument allongé souple le long d'un axe central longitudinal et déterminer une étendue de mouvement de l'instrument allongé souple dans un premier sens le long de l'axe central longitudinal sur la base de la surveillance. Le système de commande est également configuré pour modifier une rigidité de l'instrument allongé souple sur la base d'un profil de rigidité par rapport l'étendue de mouvement en réglant une ou plusieurs forces appliquées par la pluralité de fils à l'extrémité distale de l'instrument allongé souple.
PCT/US2017/040214 2016-07-01 2017-06-30 Systèmes et procédés de commande d'instrument souple assistée par ordinateur WO2018005928A1 (fr)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US16/311,514 US11344376B2 (en) 2016-07-01 2017-06-30 Systems and methods for flexible computer-assisted instrument control
EP24154210.9A EP4335403A3 (fr) 2016-07-01 2017-06-30 Systèmes et procédés de commande d'instrument assistée par ordinateur flexible
CN202211616017.0A CN116115337A (zh) 2016-07-01 2017-06-30 一种计算机辅助医疗系统及其控制方法
EP17821332.8A EP3478149B1 (fr) 2016-07-01 2017-06-30 Systèmes de commande d'instrument souple assistée par ordinateur
CN201780039384.XA CN109310287B (zh) 2016-07-01 2017-06-30 一种计算机辅助医疗系统及其控制方法
US17/733,446 US20220273377A1 (en) 2016-07-01 2022-04-29 Systems and methods for flexible computer-assisted instrument control

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662357555P 2016-07-01 2016-07-01
US62/357,555 2016-07-01

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US16/311,514 A-371-Of-International US11344376B2 (en) 2016-07-01 2017-06-30 Systems and methods for flexible computer-assisted instrument control
US17/733,446 Continuation US20220273377A1 (en) 2016-07-01 2022-04-29 Systems and methods for flexible computer-assisted instrument control

Publications (1)

Publication Number Publication Date
WO2018005928A1 true WO2018005928A1 (fr) 2018-01-04

Family

ID=60787795

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2017/040214 WO2018005928A1 (fr) 2016-07-01 2017-06-30 Systèmes et procédés de commande d'instrument souple assistée par ordinateur

Country Status (4)

Country Link
US (2) US11344376B2 (fr)
EP (2) EP4335403A3 (fr)
CN (2) CN109310287B (fr)
WO (1) WO2018005928A1 (fr)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10512515B2 (en) 2017-07-31 2019-12-24 Intuitive Surgical Operations, Inc. Systems and methods for steerable elongate device
WO2019246240A1 (fr) * 2018-06-19 2019-12-26 Intuitive Surgical Operations, Inc. Systèmes et procédés de maintien d'un dispositif allongé flexible dans une pose
CN110891514A (zh) * 2018-02-13 2020-03-17 奥瑞斯健康公司 用于驱动医疗器械的系统和方法
WO2020072415A1 (fr) * 2018-10-04 2020-04-09 Intuitive Surgical Operations, Inc. Systèmes et procédés de commande de dispositifs orientables
US11529038B2 (en) 2018-10-02 2022-12-20 Elements Endoscopy, Inc. Endoscope with inertial measurement units and / or haptic input controls
US11678788B2 (en) 2018-07-25 2023-06-20 Intuitive Surgical Operations, Inc. Systems and methods for use of a variable stiffness flexible elongate device
US11826017B2 (en) 2017-07-31 2023-11-28 Intuitive Surgical Operations, Inc. Systems and methods for safe operation of a device
US12005574B2 (en) * 2018-10-04 2024-06-11 Intuitive Surgical Operations, Inc. Systems and methods for motion control of steerable devices

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10575754B2 (en) 2015-09-23 2020-03-03 Covidien Lp Catheter having a sensor and an extended working channel
EP3503834B1 (fr) * 2016-08-23 2024-06-12 Intuitive Surgical Operations, Inc. Systèmes permettant de surveiller les mouvements d'un patient pendant un acte médical
US11793394B2 (en) 2016-12-02 2023-10-24 Vanderbilt University Steerable endoscope with continuum manipulator
WO2018122976A1 (fr) * 2016-12-27 2018-07-05 オリンパス株式会社 Appareil d'insertion de tuyau flexible
US11529190B2 (en) 2017-01-30 2022-12-20 Covidien Lp Enhanced ablation and visualization techniques for percutaneous surgical procedures
WO2019055701A1 (fr) * 2017-09-13 2019-03-21 Vanderbilt University Robots continuum à mouvement multi-échelle par modulation d'équilibre
JP7167127B2 (ja) * 2018-03-06 2022-11-08 オリンパス株式会社 可撓管挿入装置、剛性制御装置、挿入部の挿入方法、及び剛性制御プログラムを記録した記録媒体
US10872449B2 (en) 2018-05-02 2020-12-22 Covidien Lp System and method for constructing virtual radial ultrasound images from CT data and performing a surgical navigation procedure using virtual ultrasound images
US11071591B2 (en) 2018-07-26 2021-07-27 Covidien Lp Modeling a collapsed lung using CT data
US11975157B2 (en) 2019-04-12 2024-05-07 Covidien Lp Method of manufacturing an elongated catheter having multiple sensors for three-dimensional location of the catheter
CA3108966A1 (fr) * 2020-02-17 2021-08-17 Melissa Jones Catheter endovasculaire orientable
US12064191B2 (en) 2020-06-03 2024-08-20 Covidien Lp Surgical tool navigation using sensor fusion
US11701492B2 (en) 2020-06-04 2023-07-18 Covidien Lp Active distal tip drive
CN116669651A (zh) 2020-11-30 2023-08-29 直观外科手术操作公司 用于缩回和调节细长柔性装置的系统和相关方法
CN116531107A (zh) * 2020-12-29 2023-08-04 武汉联影智融医疗科技有限公司 力感知装置、医疗器械、医疗操控系统及主从式医疗操控系统
CN113057570B (zh) * 2021-03-16 2023-06-02 上海微创微航机器人有限公司 气管镜、持镜臂、可控鞘管、操作方法及患者端装置
CN113712674B (zh) * 2021-09-13 2023-05-09 上海微创微航机器人有限公司 导管机器人及系统与控制方法、可读存储介质及电子设备
CN115342953B (zh) * 2022-06-30 2023-06-02 中国科学院自动化研究所 柔性可控器械拉线的拉力传感器
CN115445039B (zh) * 2022-09-19 2023-09-29 无锡仁诺科技发展有限公司 自动位移自动搜寻声门并置入气管导管的智能插管系统
WO2024081745A2 (fr) * 2022-10-14 2024-04-18 Canon U.S.A., Inc. Localisation et ciblage de petites lésions pulmonaires
CN115554568A (zh) * 2022-11-28 2023-01-03 深圳市精锋医疗科技股份有限公司 导管弯曲转向控制方法、导管系统及存储介质

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06292652A (ja) 1993-04-09 1994-10-21 Toshiba Corp 内視鏡
US6380732B1 (en) 1997-02-13 2002-04-30 Super Dimension Ltd. Six-degree of freedom tracking system having a passive transponder on the object being tracked
US6389187B1 (en) 1997-06-20 2002-05-14 Qinetiq Limited Optical fiber bend sensor
US20060013523A1 (en) 2004-07-16 2006-01-19 Luna Innovations Incorporated Fiber optic position and shape sensing device and method relating thereto
US20070135683A1 (en) * 2005-12-09 2007-06-14 Konstantin Bob Alternating propulsion type endoscope and continuous drive type endoscope
US7316681B2 (en) 1996-05-20 2008-01-08 Intuitive Surgical, Inc Articulated surgical instrument for performing minimally invasive surgery with enhanced dexterity and sensitivity
US20080212082A1 (en) 2004-07-16 2008-09-04 Luna Innovations Incorporated Fiber optic position and/or shape sensing based on rayleigh scatter
US20100016757A1 (en) * 2008-07-10 2010-01-21 Superdimension, Ltd. Integrated Multi-Functional Endoscopic Tool
US20120289777A1 (en) 2011-05-13 2012-11-15 Intuitive Surgical Operations, Inc. Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery
US20130096497A1 (en) 2011-10-14 2013-04-18 Intuitive Surgical Operations, Inc. Catheters with control modes for interchangeable probes
US20130096377A1 (en) 2011-10-14 2013-04-18 Intuitive Surgical Operations, Inc. Catheter with removable vision probe
US20150011830A1 (en) 2010-08-27 2015-01-08 Massachusetts Institute Of Technology Tip actuated disposable endoscope
WO2015153174A1 (fr) * 2014-04-02 2015-10-08 Intuitive Surgical Operations, Inc. Dispositifs, systèmes et procédés utilisant un stylet orientable et une aiguille souple
US9259274B2 (en) 2008-09-30 2016-02-16 Intuitive Surgical Operations, Inc. Passive preload and capstan drive for surgical instruments
WO2016025465A1 (fr) 2014-08-14 2016-02-18 Intuitive Surgical Operations, Inc. Systèmes et procédés de nettoyage d'un instrument endoscopique
WO2016040079A1 (fr) * 2014-09-09 2016-03-17 Intuitive Surgical Operations, Inc. Instrument médical souple
WO2017109986A1 (fr) 2015-12-25 2017-06-29 オリンパス株式会社 Dispositif d'insertion de tube souple

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2567149B1 (fr) 1984-07-06 1986-12-05 Solvay Procede pour l'extraction de poly-beta-hydroxybutyrates au moyen d'un solvant a partir d'une suspension aqueuse de micro-organismes
US7775972B2 (en) * 1998-02-24 2010-08-17 Hansen Medical, Inc. Flexible instrument
US6974411B2 (en) * 2000-04-03 2005-12-13 Neoguide Systems, Inc. Endoscope with single step guiding apparatus
WO2014021222A1 (fr) * 2012-07-31 2014-02-06 オリンパス株式会社 Manipulateur médical
JP5702818B2 (ja) 2013-03-21 2015-04-15 富士フイルム株式会社 内視鏡挿入部及びその接合方法

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06292652A (ja) 1993-04-09 1994-10-21 Toshiba Corp 内視鏡
US7316681B2 (en) 1996-05-20 2008-01-08 Intuitive Surgical, Inc Articulated surgical instrument for performing minimally invasive surgery with enhanced dexterity and sensitivity
US6380732B1 (en) 1997-02-13 2002-04-30 Super Dimension Ltd. Six-degree of freedom tracking system having a passive transponder on the object being tracked
US6389187B1 (en) 1997-06-20 2002-05-14 Qinetiq Limited Optical fiber bend sensor
US20060013523A1 (en) 2004-07-16 2006-01-19 Luna Innovations Incorporated Fiber optic position and shape sensing device and method relating thereto
US20080212082A1 (en) 2004-07-16 2008-09-04 Luna Innovations Incorporated Fiber optic position and/or shape sensing based on rayleigh scatter
US20070135683A1 (en) * 2005-12-09 2007-06-14 Konstantin Bob Alternating propulsion type endoscope and continuous drive type endoscope
US20100016757A1 (en) * 2008-07-10 2010-01-21 Superdimension, Ltd. Integrated Multi-Functional Endoscopic Tool
US9259274B2 (en) 2008-09-30 2016-02-16 Intuitive Surgical Operations, Inc. Passive preload and capstan drive for surgical instruments
US20150011830A1 (en) 2010-08-27 2015-01-08 Massachusetts Institute Of Technology Tip actuated disposable endoscope
US20120289777A1 (en) 2011-05-13 2012-11-15 Intuitive Surgical Operations, Inc. Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery
US20130096377A1 (en) 2011-10-14 2013-04-18 Intuitive Surgical Operations, Inc. Catheter with removable vision probe
US20130096497A1 (en) 2011-10-14 2013-04-18 Intuitive Surgical Operations, Inc. Catheters with control modes for interchangeable probes
WO2015153174A1 (fr) * 2014-04-02 2015-10-08 Intuitive Surgical Operations, Inc. Dispositifs, systèmes et procédés utilisant un stylet orientable et une aiguille souple
WO2016025465A1 (fr) 2014-08-14 2016-02-18 Intuitive Surgical Operations, Inc. Systèmes et procédés de nettoyage d'un instrument endoscopique
WO2016040079A1 (fr) * 2014-09-09 2016-03-17 Intuitive Surgical Operations, Inc. Instrument médical souple
WO2017109986A1 (fr) 2015-12-25 2017-06-29 オリンパス株式会社 Dispositif d'insertion de tube souple

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11826017B2 (en) 2017-07-31 2023-11-28 Intuitive Surgical Operations, Inc. Systems and methods for safe operation of a device
US11648076B2 (en) 2017-07-31 2023-05-16 Intuitive Surgical Operations, Inc. Input control system console drape
US10512515B2 (en) 2017-07-31 2019-12-24 Intuitive Surgical Operations, Inc. Systems and methods for steerable elongate device
US11284959B2 (en) 2017-07-31 2022-03-29 Intuitive Surgical Operations, Inc. Method for protecting an input control console with a drape
US10842581B2 (en) 2017-07-31 2020-11-24 Intuitive Surgical Operations, Inc. Panel and drape for input control console of elongate device
CN110891514B (zh) * 2018-02-13 2023-01-20 奥瑞斯健康公司 用于驱动医疗器械的系统和方法
CN110891514A (zh) * 2018-02-13 2020-03-17 奥瑞斯健康公司 用于驱动医疗器械的系统和方法
CN111989022A (zh) * 2018-06-19 2020-11-24 直观外科手术操作公司 用于将柔性细长装置保持在姿势中的系统和方法
WO2019246240A1 (fr) * 2018-06-19 2019-12-26 Intuitive Surgical Operations, Inc. Systèmes et procédés de maintien d'un dispositif allongé flexible dans une pose
US12004710B2 (en) 2018-06-19 2024-06-11 Intuitive Surgical Operations, Inc. Systems and methods for holding a flexible elongate device in a pose
US11678788B2 (en) 2018-07-25 2023-06-20 Intuitive Surgical Operations, Inc. Systems and methods for use of a variable stiffness flexible elongate device
US11529038B2 (en) 2018-10-02 2022-12-20 Elements Endoscopy, Inc. Endoscope with inertial measurement units and / or haptic input controls
CN112804960A (zh) * 2018-10-04 2021-05-14 直观外科手术操作公司 用于控制可转向装置的系统和方法
WO2020072415A1 (fr) * 2018-10-04 2020-04-09 Intuitive Surgical Operations, Inc. Systèmes et procédés de commande de dispositifs orientables
US12005574B2 (en) * 2018-10-04 2024-06-11 Intuitive Surgical Operations, Inc. Systems and methods for motion control of steerable devices
US12023803B2 (en) 2018-10-04 2024-07-02 Intuitive Surgical Operations, Inc. Systems and methods for control of steerable devices
CN112804960B (zh) * 2018-10-04 2024-09-13 直观外科手术操作公司 用于控制可转向装置的系统和方法

Also Published As

Publication number Publication date
EP4335403A2 (fr) 2024-03-13
EP4335403A3 (fr) 2024-06-05
CN116115337A (zh) 2023-05-16
US20220273377A1 (en) 2022-09-01
EP3478149A4 (fr) 2020-01-15
EP3478149B1 (fr) 2024-03-20
US11344376B2 (en) 2022-05-31
CN109310287A (zh) 2019-02-05
EP3478149A1 (fr) 2019-05-08
US20190231449A1 (en) 2019-08-01
CN109310287B (zh) 2023-01-03

Similar Documents

Publication Publication Date Title
US20220273377A1 (en) Systems and methods for flexible computer-assisted instrument control
US20240268636A1 (en) Systems and methods for instrument buckling detection
US11992188B2 (en) Systems and methods for detecting environmental forces on an elongate device
US11607107B2 (en) Systems and methods for medical instrument force sensing
EP3503834B1 (fr) Systèmes permettant de surveiller les mouvements d'un patient pendant un acte médical
EP3478210B1 (fr) Interface utilisateur graphique pour l'affichage d'informations de guidage pendant une intervention guidée par imagerie
US11832891B2 (en) Systems and methods for fault reaction mechanisms for medical robotic systems
CN107072717B (zh) 用于自适应输入映射的系统和方法
US20200078103A1 (en) Graphical user interface for displaying guidance information in a plurality of modes during an image-guided procedure
US20200008678A1 (en) Systems and methods for medical procedures using optical coherence tomography sensing
US20240308063A1 (en) Systems and methods for control of steerable devices
CN118845238A (zh) 用于自适应输入映射的系统和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17821332

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017821332

Country of ref document: EP

Effective date: 20190201