WO2014049598A1

WO2014049598A1 - Directing and maneuvering articulating a laparoscopic surgery tool

Info

Publication number: WO2014049598A1
Application number: PCT/IL2013/050806
Authority: WO
Inventors: Gal ATAROT; Yaron LEVINSON
Original assignee: M.S.T. Medical Surgery Technologies Ltd.
Priority date: 2012-09-30
Filing date: 2013-09-30
Publication date: 2014-04-03
Also published as: EP2900130A4; DE202013012276U1; EP2900130A1

Abstract

A surgical controlling system comprising: a. a surgical tool adapted to be inserted into a surgical environment of a human body for assisting a surgical procedure, said surgical tool being an articulating tool; b. a location estimating means adapted to real-time locate the 3D spatial position of said surgical tool at any given time 't'; c. a movement detection means; and, d. a controller having a processing means communicable with a controller's database, said controller adapted to control the spatial position of said surgical tool; said controller's database is in communication with said movement detection means; said controller adapted to provide instructions for moving said surgical tool.

Description

DEVICE AND METHOD FOR ASISSTING LAPAROSCOPIC SURGERY - DIRECTING

AND MANEUVERING ARTICULATING TOOL

FIELD OF THE INVENTION

The present invention generally pertains to a system and method for directing and maneuvering an articulating tool such as an endoscope during laparoscopic surgery.

BACKGROUND OF THE INVENTION

In laparoscopic surgery, the surgeon performs the operation through small holes using long instruments and observing the internal anatomy with an endoscope camera.

Laparoscopic surgery is becoming increasingly popular with patients because the scars are smaller and their period of recovery is shorter. Laparoscopic surgery requires special training for the surgeon and the theatre nursing staff. The equipment is often expensive and is not available in all hospitals.

During laparoscopic surgery, it is often required to shift the spatial placement of the endoscope in order to present the surgeon with an optimal view. Conventional laparoscopic surgery makes use of either human assistants that manually shift the instrumentation or, alternatively, robotic automated assistants. Automated assistants utilize interfaces that enable the surgeon to direct the mechanical movement of the assistant, achieving a shift in the camera view.

Research has suggested that these systems divert the surgeon's focus from the major task at hand. Therefore, technologies assisted by magnets and image processing have been developed to simplify interfacing control. In all such systems, the endoscope must be maneuvered such that it does not come into contact with other objects in the surgical field, such as other tools or the patient's organs, which can significantly complicate the maneuvering of the endoscope.

Therefore, there is need for a system in which the system comprises an endoscope or other surgical tool that can change shape, size or angulation so as to simplify maneuvering of the system.

Hence, there is still a long felt need for a method of directing a laparoscopic system to a desired location that includes control of the size, shape or angulation of at least one surgical tool.

SUMMARY OF THE INVENTION It is an object of the present invention to disclose a system and method for directing and maneuvering an articulating tool such as an endoscope during laparoscopic surgery.

It is another object of the present invention to disclose the surgical controlling system, additionally comprising at least one endoscope adapted to provide a real time image of said surgical environment.

It is another object of the present invention to disclose the surgical controlling system, wherein said tool is an endoscope.

It is another object of the present invention to disclose the surgical controlling system, wherein said tool comprises at least one proximity sensor positioned on the outer circumference of the same.

It is another object of the present invention to disclose the surgical controlling system, wherein said instructions comprise a predetermined set of rules selected from a group consisting of: most used tool rule, right tool rule, left tool rule, field of view rule, no fly zone rule, a route rule, environmental rule, operator input rule, proximity rule; collision prevention rule, history-based rule, tool-dependent ALLOWED and RESTRICTED movements rule, preferred volume zone rule, preferred tool rule, movement detection rule, tagged tool rule, change of speed rule and any combination thereof.

It is another object of the present invention to disclose the surgical controlling system, wherein said route rule comprises a communicable database storing predefined route in which said at least one surgical tool is adapted to move within said surgical environment; said predefined route comprises n 3D spatial positions of said at least one surgical tool; n is an integer greater than or equal to 2; said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said n 3D spatial positions of said predefined route, and said RESTRICTED movements are movements in which said location of said at least one surgical tool is substantially different from said n 3D spatial positions of said predefined route.

It is another object of the present invention to disclose the surgical controlling system, wherein said environmental rule comprises a comprises a communicable database; said communicable database adapted to receive at least one real-time image of said surgical environment and is adapted to perform real-time image processing of the same and to determine the 3D spatial position of hazards or obstacles in said surgical environment; said environmental rule is adapted to determine said ALLOWED and RESTRICTED movements according to said hazards or obstacles in said surgical environment, such that said RESTRICTED movements are movements in which said at least one surgical tool is located substantially in at least one of said 3D spatial positions, and said ALLOWED movements are movements in which the location of said at least one surgical tool is substantially different from said 3D spatial positions.

It is another object of the present invention to disclose the surgical controlling system, wherein said hazards or obstacles in said surgical environment are selected from a group consisting of tissue, a surgical tool, an organ, an endoscope and any combination thereof.

It is another object of the present invention to disclose the surgical controlling system, wherein said operator input rule comprises a communicable database; said communicable database is adapted to receive an input from the operator of said system regarding said ALLOWED and RESTRICTED movements of said at least one surgical tool.

It is another object of the present invention to disclose the surgical controlling system, wherein said input comprises n 3D spatial positions; n is an integer greater than or equal to 2; wherein at least one of which is defined as ALLOWED location and at least one of which is defined as RESTRICTED location, such that said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said at least one surgical tool is substantially different from said n 3D spatial positions.

It is another object of the present invention to disclose the surgical controlling system, wherein said input comprises at least one rule according to which ALLOWED and RESTRICTED movements of said at least one surgical tool are determined, such that the spatial position of said at least one surgical tool is controlled by said controller according to said ALLOWED and RESTRICTED movements.

It is another object of the present invention to disclose the surgical controlling system, wherein said predetermined set of rules comprises a member of a group consisting of: most used tool, right tool rule, left tool rule, field of view rule, no fly zone rule, route rule, environmental rule, operator input rule, proximity rule, collision prevention rule, preferred volume zone rule, preferred tool rule, movement detection rule, history-based rule, tool-dependent ALLOWED and RESTRICTED movements rule, and any combination thereof.

It is another object of the present invention to disclose the surgical controlling system, wherein said operator input rule converts an ALLOWED movement to a RESTRICTED movement and a RESTRICTED movement to an ALLOWED movement.

It is another object of the present invention to disclose the surgical controlling system, wherein said proximity rule is adapted to define a predetermined distance between at least two surgical tools; said ALLOWED movements are movements which are within the range or out of the range of said predetermined distance, and said RESTRICTED movements are movements which are out of the range or within the range of said predetermined distance.

It is another object of the present invention to disclose the surgical controlling system, wherein said proximity rule is adapted to define a predetermined angle between at least three surgical tools; said ALLOWED movements are movements which are within the range or out of the range of said predetermined angle, and said RESTRICTED movements are movements which are out of the range or within the range of said predetermined angle.

It is another object of the present invention to disclose the surgical controlling system, wherein said collision prevention rule is adapted to define a predetermined distance between said at least one surgical tool and an anatomical element within said surgical environment; said ALLOWED movements are movements which are in a range that is larger than said predetermined distance, and said RESTRICTED movements are movements which is in a range that is smaller than said predetermined distance.

It is another object of the present invention to disclose the surgical controlling system, wherein said anatomical element is selected from a group consisting of tissue, organ, another surgical tool and any combination thereof.

It is another object of the present invention to disclose the surgical controlling system, wherein at least one of the following is being held true (a) said system additionally comprises an endoscope; said endoscope is adapted to provide real-time image of said surgical environment; (b) at least one of said surgical tools is an endoscope adapted to provide at least one real-time image of said surgical environment.

It is another object of the present invention to disclose the surgical controlling system, wherein said right tool rule is adapted to determine said ALLOWED movement of said endoscope according to the movement of the surgical tool positioned to right of said endoscope; further wherein said left tool rule is adapted to determine said ALLOWED movement of said endoscope according to the movement of the surgical tool positioned to left of said endoscope.

It is another object of the present invention to disclose the surgical controlling system, wherein said tagged tool rule comprises means adapted to tag at least one surgical tool within said surgical environment and to determine said ALLOWED movement of said endoscope to constantly track the movement of said tagged surgical tool. It is another object of the present invention to disclose the surgical controlling system, wherein said field of view rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; the combination of all of said n 3D spatial positions provides a predetermined field of view; said field of view rule is adapted to determine said ALLOWED movement of said endoscope within said n 3D spatial positions so as to maintain a constant field of view, such that said ALLOWED movements are movements in which said endoscope is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said endoscope is substantially different from said n 3D spatial positions.

It is another object of the present invention to disclose the surgical controlling system, wherein said preferred volume zone rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; said n 3D spatial positions provides said preferred volume zone; said preferred volume zone rule is adapted to determine said ALLOWED movement of said endoscope within said n 3D spatial positions and RESTRICTED movement of said endoscope outside said n 3D spatial positions, such that said ALLOWED movements are movements in which said endoscope is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said endoscope is substantially different from said n 3D spatial positions.

It is another object of the present invention to disclose the surgical controlling system, wherein said preferred tool rule comprises a communicable database, said database stores a preferred tool; said preferred tool rule is adapted to determine said ALLOWED movement of said endoscope to constantly track the movement of said preferred tool.

It is another object of the present invention to disclose the surgical controlling system, wherein said no fly zone rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; said n 3D spatial positions define a predetermined volume within said surgical environment; said no fly zone rule is adapted to determine said RESTRICTED movement if said movement is within said no fly zone and ALLOWED movement if said movement is outside said no fly zone, such that said RESTRICTED movements are movements in which said at least one of said surgical tool is located substantially in at least one of said n 3D spatial positions, and said ALLOWED movements are movements in which the location of said at least one endoscope is substantially different from said n 3D spatial positions. It is another object of the present invention to disclose the surgical controlling system, wherein said most used tool rule comprises a communicable database counting the amount of movement of each said surgical tool; said most used tool rule is adapted to constantly position said endoscope to track the movement of the most moved surgical tool.

It is another object of the present invention to disclose the surgical controlling system, wherein said system further comprises a maneuvering subsystem communicable with said controller, said maneuvering subsystem is adapted to spatially reposition said at least one surgical tool during a surgery according to said predetermined set of rules; further wherein said system is adapted to alert the physician of said RESTRICTED movement of said at least one surgical tool.

It is another object of the present invention to disclose the surgical controlling system, wherein said alert is selected from a group consisting of audio signaling, voice signaling, light signaling, flashing signaling and any combination thereof.

It is another object of the present invention to disclose the surgical controlling system, wherein said ALLOWED movement is permitted by said controller and said RESTRICTED movement is denied by said controller.

It is another object of the present invention to disclose the surgical controlling system, wherein said history-based rule comprises a communicable database storing each 3D spatial position of each said surgical tool, such that each movement of each surgical tool is stored; said history-based rule is adapted to determine said ALLOWED and RESTRICTED movements according to historical movements of said at least one surgical tool, such that said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said 3D spatial positions, and said RESTRICTED movements are movements in which the location of said at least one surgical tool is substantially different from said n 3D spatial positions.

It is another object of the present invention to disclose the surgical controlling system, wherein said tool-dependent ALLOWED and RESTRICTED movements rule comprises a communicable database; said communicable database is adapted to store predetermined characteristics of at least one of said surgical tool; said tool-dependent ALLOWED and RESTRICTED movements rule is adapted to determine said ALLOWED and RESTRICTED movements according to said predetermined characteristics of said surgical tool; such that ALLOWED movements are movements of said endoscope which track said surgical tool having said predetermined characteristics. It is another object of the present invention to disclose the surgical controlling system, wherein said predetermined characteristics of said surgical tool are selected from a group consisting of: physical dimensions, structure, weight, sharpness, and any combination thereof.

It is another object of the present invention to disclose the surgical controlling system, wherein said movement detection rule comprises a communicable database comprising the real-time 3D spatial positions of each said surgical tool; said movement detection rule is adapted to detect movement of said at least one surgical tool when a change in said 3D spatial positions is received, such that said ALLOWED movements are movements in which said endoscope is re-directed to focus on said moving surgical tool.

It is another object of the present invention to disclose the surgical controlling system, further comprising a maneuvering subsystem communicable with said controller, said maneuvering subsystem is adapted to spatially reposition said at least one surgical tool during a surgery according to said predetermined set of rules, such that if said movement of said at least one surgical tool is a RESTRICTED movement, said maneuvering subsystem prevents said movement.

It is another object of the present invention to disclose the surgical controlling system, wherein said at least one location estimating means comprises at least one endoscope adapted to acquire realtime images of said surgical environment within said human body; and at least one surgical instrument spatial location software adapted to receive said real-time images of said surgical environment and to estimate said 3D spatial position of said at least one surgical tool.

It is another object of the present invention to disclose the surgical controlling system, wherein said at least one location estimating means comprises (a) at least one element selected from a group consisting of optical imaging means, radio frequency transmitting and receiving means, at least one mark on said at least one surgical tool and any combination thereof; and, (b) at least one surgical instrument spatial location software adapted to estimate said 3D spatial position of said at least one surgical tool by means of said element.

It is another object of the present invention to disclose the surgical controlling system, wherein said at least one location estimating means is an interface subsystem between a surgeon and said at least one surgical tool, the interface subsystem comprising: a. at least one array comprising N regular or pattern light sources, where N is a positive integer; b. at least one array comprising M cameras, each of the M cameras, where M is a positive integer; c. optional optical markers and means for attaching the optical marker to the at least one surgical tool; and; d. a computerized algorithm operable via the controller, the computerized algorithm adapted to record images received by each camera of each of the M cameras and to calculate therefrom the position of each of the tools, and further adapted to provide automatically the results of the calculation to the human operator of the interface.

It is another object to disclose the method, additionally comprising steps of providing a real time image of said surgical environment using at least one endoscope.

It is another object to disclose the method, additionally comprising steps of selecting said tool to be an endoscope.

It is another object to disclose the method, additionally comprising steps of positioning at least one proximity sensor on the outer circumference of said tool.

It is another object to disclose the method, additionally comprising steps of selecting said instructions from a predetermined set of rules selected from a group consisting of: most used tool rule, right tool rule, left tool rule, field of view rule, no fly zone rule, a route rule, environmental rule, operator input rule, proximity rule; collision prevention rule, history-based rule, tool- dependent ALLOWED and RESTRICTED movements rule, preferred volume zone rule, preferred tool rule, movement detection rule, tagged tool rule, change of speed rule and any combination thereof.

It is another object to disclose the method, wherein said route rule comprises steps of: providing a communicable database; storing a predefined route in which said at least one surgical tool is adapted to move within said surgical environment; comprising said predefined route of n 3D spatial positions of said at least one surgical tool, n is an integer greater than or equal to 2; said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said n 3D spatial positions of said predefined route, and said RESTRICTED movements are movements in which said location of said at least one surgical tool is substantially different from said n 3D spatial positions of said predefined route.

It is another object to disclose the method, wherein said environmental rule comprises steps of: providing a communicable database; receiving at least one real-time image of said surgical environment in said communicable database; performing real-time image processing of the same and determining the 3D spatial position of hazards or obstacles in said surgical environment; determining said ALLOWED and RESTRICTED movements according to said hazards or obstacles in said surgical environment, such that said RESTRICTED movements are movements in which said at least one surgical tool is located substantially in at least one of said 3D spatial positions, and said ALLOWED movements are movements in which the location of said at least one surgical tool is substantially different from said 3D spatial positions.

It is another object to disclose the method, additionally comprising steps of selecting said hazards or obstacles in said surgical environment from a group consisting of tissue, a surgical tool, an organ, an endoscope and any combination thereof.

It is another object to disclose the method, wherein said operator input rule comprises steps of: providing a communicable database; and receiving input from an operator of said system regarding said ALLOWED and RESTRICTED movements of said at least one surgical tool.

It is another object to disclose the method, additionally comprising steps of: comprising said input of n 3D spatial positions, n is an integer greater than or equal to 2; defining at least one of said spatial positions as an ALLOWED location; defining at least one of said spatial positions as a RESTRICTED location; such that said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said at least one surgical tool is substantially different from said n 3D spatial positions.

It is another object to disclose the method, additionally comprising steps of: comprising said input of at least one rule according to which ALLOWED and RESTRICTED movements of said at least one surgical tool are determined, such that the spatial position of said at least one surgical tool is controlled by said controller according to said ALLOWED and RESTRICTED movements.

It is another object to disclose the method, additionally comprising steps of selecting said predetermined set of rules from a group consisting of: most used tool, right tool rule, left tool rule, field of view rule, no fly zone rule, route rule, environmental rule, operator input rule, proximity rule, collision prevention rule, preferred volume zone rule, preferred tool rule, movement detection rule, history-based rule, tool-dependent ALLOWED and RESTRICTED movements rule, and any combination thereof.

It is another object to disclose the method, wherein said operator input rule comprises steps of: converting an ALLOWED movement to a RESTRICTED movement and converting a RESTRICTED movement to an ALLOWED movement.

It is another object to disclose the method, wherein said proximity rule comprises steps of: defining a predetermined distance between at least two surgical tools; said ALLOWED movements are movements which are within the range or out of the range of said predetermined distance, and said RESTRICTED movements are movements which are out of the range or within the range of said predetermined distance.

It is another object to disclose the method, wherein said proximity rule comprises steps of: defining a predetermined angle between at least three surgical tools; said ALLOWED movements are movements which are within the range or out of the range of said predetermined angle, and said RESTRICTED movements are movements which are out of the range or within the range of said predetermined angle.

It is another object to disclose the method, wherein said collision prevention rule comprises steps of: defining a predetermined distance between said at least one surgical tool and an anatomical element within said surgical environment; said ALLOWED movements are movements which are in a range that is larger than said predetermined distance, and said RESTRICTED movements are movements which is in a range that is smaller than said predetermined distance.

It is another object to disclose the method, additionally comprising steps of selecting said anatomical element from a group consisting of tissue, organ, another surgical tool and any combination thereof.

It is another object to disclose the method, wherein at least one of the following is being held true (a) additionally providing an endoscope for said system; and provide at least one real-time image of said surgical environment by means of said endoscope; (b) selecting at least one of said surgical tools to be an endoscope and providing at least one real-time image of said surgical environment by means of said endoscope.

It is another object to disclose the method, wherein said right tool rule comprises steps of: determining said ALLOWED movement of said endoscope according to the movement of the surgical tool positioned to right of said endoscope; further wherein said left tool rule comprises steps of: determining said ALLOWED movement of said endoscope according to the movement of the surgical tool positioned to left of said endoscope.

It is another object to disclose the method, wherein said tagged tool rule comprises steps of: tagging at least one surgical tool within said surgical environment and determining said ALLOWED movements of said endoscope to be movements that constantly track the movement of said tagged surgical tool.

It is another object to disclose the method, wherein said field of view rule comprises steps of: providing a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; generating a field of view from the combination of all of said n 3D spatial positions; maintaining a constant field of view by determining said ALLOWED movement of said endoscope to be within said n 3D spatial positions, such that said ALLOWED movements are movements in which said endoscope is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said endoscope is substantially different from said n 3D spatial positions.

It is another object to disclose the method, wherein said preferred volume zone rule comprises steps of: providing a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; generating said preferred volume zone from said n 3D spatial positions; determining said ALLOWED movement of said endoscope to be within said n 3D spatial positions and said RESTRICTED movement of said endoscope to be outside said n 3D spatial positions, such that said ALLOWED movements are movements in which said endoscope is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said endoscope is substantially different from said n 3D spatial positions.

It is another object to disclose the method, wherein said preferred tool rule comprises steps of: providing a communicable database, storing a preferred tool in said database; determining said ALLOWED movement of said endoscope so as to constantly track the movement of said preferred tool.

It is another object to disclose the method, wherein said no fly zone rule comprises steps of: providing a communicable database comprising n 3D spatial positions, n is an integer greater than or equal to 2; defining a predetermined volume within said surgical environment from said n 3D spatial positions; determining said RESTRICTED movement to be said movement within said no fly zone; determining said ALLOWED movement to be said movement outside said no fly zone, such that said RESTRICTED movements are movements in which said at least one of said surgical tool is located substantially in at least one of said n 3D spatial positions, and said ALLOWED movements are movements in which the location of said at least one endoscope is substantially different from said n 3D spatial positions.

It is another object to disclose the method, wherein said most used tool rule comprises steps of: providing a communicable database; counting the amount of movement of each said surgical tool; constantly positioning said endoscope to track movement of the most moved surgical tool.

It is another object to disclose the method, additionally comprising steps of providing a maneuvering subsystem communicable with said controller, spatially repositioning said at least one surgical tool during a surgery according to said predetermined set of rules; and alerting the physician of said RESTRICTED movement of said at least one surgical tool.

It is another object to disclose the method, additionally comprising steps of selecting said alert from a group consisting of: audio signaling, voice signaling, light signaling, flashing signaling and any combination thereof.

It is another object to disclose the method, additionally comprising steps of defining said ALLOWED movement as a movement permitted by said controller and defining said RESTRICTED movement as a movement denied by said controller.

It is another object to disclose the method, wherein said history-based rule comprises steps of: providing a communicable database storing each 3D spatial position of each said surgical tool, such that each movement of each surgical tool is stored; determining said ALLOWED and RESTRICTED movements according to historical movements of said at least one surgical tool, such that said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said 3D spatial positions, and said RESTRICTED movements are movements in which the location of said at least one surgical tool is substantially different from said n 3D spatial positions.

It is another object to disclose the method, wherein said tool-dependent ALLOWED and RESTRICTED movements rule comprises steps of: providing a communicable database; storing predetermined characteristics of at least one said surgical tool; determining said ALLOWED and RESTRICTED movements according to said predetermined characteristics of said surgical tool; such that ALLOWED movements are movements of said endoscope which track said surgical tool having said predetermined characteristics.

It is another object to disclose the method, additionally comprising steps of selecting said predetermined characteristics of said surgical tool from a group consisting of: physical dimensions, structure, weight, sharpness, and any combination thereof.

It is another object to disclose the method, wherein said movement detection rule comprises steps of: providing a communicable database comprising the real-time 3D spatial positions of each said surgical tool; detecting movement of said at least one surgical tool when a change in said 3D spatial positions is received, such that said ALLOWED movements are movements in which said endoscope is re-directed to focus on said moving surgical tool.

It is another object to disclose the method, additionally comprising steps of providing a maneuvering subsystem communicable with said controller, spatially repositioning said at least one surgical tool during a surgery according to said predetermined set of rules, such that if said movement of said at least one surgical tool is a RESTRICTED movement, said maneuvering subsystem prevents said movement.

It is another object to disclose the method, additionally comprising steps of comprising said at least one location estimating means of at least one endoscope adapted to acquire real-time images of said surgical environment within said human body; providing at least one surgical instrument spatial location software; receiving said real-time images of said surgical environment from said endoscope and estimating said 3D spatial position of said at least one surgical tool using said spatial location software.

It is another object to disclose the method, additionally comprising steps of providing said at least one location estimating means comprising (a) at least one element selected from a group consisting of optical imaging means, radio frequency transmitting and receiving means, at least one mark on said at least one surgical tool and any combination thereof; and, (b) at least one surgical instrument spatial location software adapted to estimate said 3D spatial position of said at least one surgical tool by means of said element.

It is another object to disclose the method, additionally comprising steps of selecting said at least one location estimating means to be an interface subsystem between a surgeon and said at least one surgical tool, the interface subsystem comprising: a. at least one array comprising N regular or pattern light sources, where N is a positive integer; b. at least one array comprising M cameras, each of the M cameras, where M is a positive integer; c. optional optical markers and means for attaching the optical marker to the at least one surgical tool; and; d. a computerized algorithm operable via the controller, the computerized algorithm adapted to record images received by each camera of each of the M cameras and to calculate therefrom the position of each of the tools, and further adapted to provide automatically the results of the calculation to the human operator of the interface.

It is another object of the present invention to disclose the surgical controlling system, wherein said articulating tool has articulations substantially at the tip of said tool, substantially along the body of said too, and any combination thereof. It is another object of the present invention to disclose the surgical controlling system, wherein control of articulation is selected from a group consisting of hardware control, software control and any combination thereof.

It is another object of the present invention to disclose the surgical controlling system, wherein said tool has articulation in a regions selected from a group consisting of near the tip of said tool, on the body of said tool, and any combination thereof.

It is another object of the present invention to disclose the method, additionally comprising steps of providing said tool with articulations substantially at the tip of said tool, substantially along the body of said too, and any combination thereof.

It is another object of the present invention to disclose the method, additionally comprising steps of controlling articulation by means of a method selected from a group consisting of hardware control, software control and any combination thereof.

It is another object of the present invention to disclose the method, additionally comprising steps of providing a toll articulated at a region selected from a group consisting of near the tip of said tool, on the body of said tool, and any combination thereof.

BRIEF DESCRIPTION OF THE FIGURES

In order to understand the invention and to see how it may be implemented in practice, and by way of non-limiting example only, with reference to the accompanying drawing, in which

Fig. 1 depicts a direction indicator;

Fig. 2A-B presents a means to control the articulation of an articulating endoscope;

Fig. 3 illustrates the use of the endoscope articulation control;

Fig. 4 illustrates articulation of the endoscope;

Fig. 5 illustrates one embodiment of the present invention;

Fig. 6A-D schematically illustrates operation of an embodiment of a tracking system with collision avoidance system;

Fig. 7A-D schematically illustrates operation of an embodiment of a tracking system with no fly zone rule/function;

Fig. 8A-D schematically illustrates operation of an embodiment of a tracking system with preferred volume zone rule/function;

Fig. 9 schematically illustrates operation of an embodiment of the organ detection function/rule; Fig. 10 schematically illustrates operation of an embodiment of the tool detection function/rule; Fig. 11A-B schematically illustrates operation of an embodiment of the movement detection function/rule;

Fig. 12A-D schematically illustrates operation of an embodiment of the prediction function/rule;

Fig. 13 schematically illustrates operation of an embodiment of the right tool function/rule;

Fig. 14A-B schematically illustrates operation of an embodiment of the field of view function/rule;

Fig. 15 schematically illustrates operation of an embodiment of the tagged tool function/rule;

Fig. 16A-C schematically illustrates operation of an embodiment of the proximity function/rule;

Fig. 17A-B schematically illustrates operation of an embodiment of the operator input function/rule;

Figs. 18A-D schematically illustrate an embodiment of a tracking system with a constant field of view rule/function;

Fig. 19 schematically illustrates an embodiment of a tracking system with a change of speed rule/function;

Fig. 20A-B schematically illustrates movement of an articulated tool; and

Fig. 21 schematically illustrates movement of an articulated tool.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIEMNTS

The following description is provided, alongside all chapters of the present invention, so as to enable any person skilled in the art to make use of said invention and sets forth the best modes contemplated by the inventor of carrying out this invention. Various modifications, however, will remain apparent to those skilled in the art, since the generic principles of the present invention have been defined specifically to provide a means and method for directing a laparoscopic system comprising at least one articulating tool.

The term 'articulation' refers hereinafter to any device which has more than 1 degree of freedom. Thus, said tool can bend either in the tip thereof or any location in the body of the same.

The term 'toggle' refers hereinafter to switching between one tagged surgical tool to another.

The term 'surgical environment' refers hereinafter to any anatomical part within the human body which may be in surrounding a surgical instrument. The environment may comprise: organs, body parts, walls of organs, arteries, veins, nerves, a region of interest, or any other anatomical part of the human body.

The term 'endoscope' refers hereinafter to any means adapted for looking inside the body for medical reasons. This may be any instrument used to examine the interior of a hollow organ or cavity of the body. The endoscope may also refer to any kind of a laparascope. It should be pointed that the following description may refer to an endoscope as a surgical tool.

The term 'region of interest' refers hereinafter to any region within the human body which may be of interest to the operator of the system of the present invention. The region of interest may be, for example, an organ to be operated on, a RESTRICTED area to which approach of a surgical instrument is RESTRICTED, a surgical instrument, or any other region within the human body.

The term 'spatial position' refers hereinafter to a predetermined spatial location and/or orientation of an object (e.g., the spatial location of the endoscope, the angular orientation of the endoscope, and any combination thereof).

The term 'prohibited area' refers hereinafter to a predetermined area to which a surgical tool (e.g., an endoscope) is prohibited to be spatially positioned in.

The term 'preferred area' refers hereinafter to predetermined area to which a surgical tool (e.g., an endoscope) is allowed and/or preferred to be spatially positioned in.

The term 'automated assistant' refers hereinafter to any mechanical device (including but not limited to a robotic device) that can maneuver and control the position of a surgical or endoscopic instrument, and that can in addition be adapted to receive commands from a remote source.

The term 'tool' or 'surgical instrument' refers hereinafter to any instrument or device introducible into the human body. The term may refer to any location on the tool. For example it can refer to the tip of the same, the body of the same and any combination thereof. It should be further pointed that the following description may refer to a surgical tool/instrument as an endoscope.

The term 'provide' refers hereinafter to any process (visual, tactile, or auditory) by which an instrument, computer, controller, or any other mechanical or electronic device can report the results of a calculation or other operation to a human operator.

The term 'automatic' or 'automatically' refers to any process that proceeds without the necessity of direct intervention or action on the part of a human being.

The term 'ALLOWED movement' refers hereinafter to any movement of a surgical tool which is permitted according to a predetermined set of rules.

The term 'RESTRICTED movement' refers hereinafter to any movement of a surgical tool which is forbidden according to a predetermined set of rules. For example, one rule, according to the present invention, provides a preferred volume zone rule which defines a favored zone within the surgical environment. Thus, according to the present invention an ALLOWED movement of a surgical tool or the endoscope is a movement which maintains the surgical tool within the favored zone; and a RESTRICTED movement of a surgical tool is a movement which extracts (or moves) the surgical tool outside the favored zone.

The term 'time step' refers hereinafter to the working time of the system. At each time step, the system receives data from sensors and commands from operators and processes the data and commands and executes actions. The time step size is the elapsed time between time steps.

The term 'proximity sensor' hereinafter refers to a sensor able to detect the presence of nearby objects without physical contact. Proximity sensors are sometimes referred to as 'force sensors'. A proximity sensor often emits an electromagnetic field or a beam of electromagnetic radiation (infrared, for instance), and looks for changes in the field or return signal. The object being sensed is often referred to as the proximity sensor's target. Different proximity sensor targets demand different sensors. For example, a capacitive photoelectric sensor might be suitable for a plastic target; an inductive proximity sensor always requires a metal target. Proximity sensors can be introduced into the body and used for detecting metal fragments during surgery. See, for example, Sakthivel, M., A new inductive proximity sensor as a guiding tool for removing metal shrapnel during surgery, Instrumentation and Measurement Technology Conference (I2MTC), 2013 IEEE International, pp. 53-57. ISSN: 1091-5281, print ISBN: 978-1-4673-4621-4. INSPEC Accession Number: 13662555.

Laparoscopic surgery, also called minimally invasive surgery (MIS), is a modern surgical technique in which operations in the abdomen are performed through small incisions (usually 0.5-1.5cm) as compared to larger incisions needed in traditional surgical procedures. The key element in laparoscopic surgery is the use of a laparoscope, which is a device adapted for viewing the scene within the body, at the distal end of the laparoscope. Either an imaging device is placed at the end of the laparoscope, or a rod lens system or fiber optic bundle is used to direct this image to the proximal end of the laparoscope. Also attached is a light source to illuminate the operative field, inserted through a 5 mm or 10 mm cannula or trocar to view the operative field.

The abdomen is usually injected with carbon dioxide gas to create a working and viewing space. The abdomen is essentially blown up like a balloon (insufflated), elevating the abdominal wall above the internal organs like a dome. Within this space, various medical procedures can be carried out.

In many cases, the laparoscope cannot view the entire working space within the body, so the laparoscope is repositioned to allow the surgeon to view regions of interest within the space. In some laparoscopic system, this requires the surgeon to instruct an assistant to manually move the laparoscope. In other systems, the surgeon himself instructs the laparoscope to move, by a manual control system such as a button, joystick or slider attached to the surgeon or to a surgical tool, by contact with a touchscreen, or by voice commands.

In all such systems, in directing and maneuvering the surgical controlling system, the controller needs to avoid obstacles such as body organs and tools or other surgical equipment in the body cavity. Its speed should be controlled so that, on the one hand, the speed is low enough to make avoidance routine and to ensure that the instrument accurately reaches the desired location and, on the other hand, the speed needs to be great enough that maneuvers are accomplished in a reasonable time.

In order to avoid the obstacles, in a conventional system, the endoscope must be routed around them, increasing the complexity of maneuvering and the time taken for maneuvering.

In the present, system, the system comprises at least one articulating section, typically an articulating tool such as an articulating endoscope. The articulating tool can have an articulating tip, where the articulations are near the tip, it can have an articulating body, where the articulations are in the body or shaft of the tool, or both. The articulations allow bending in at least two degrees of freedom (DOF), preferably in four DOF, and possibly in all six DOF (bending in all three directions and rotating in all three directions).

In comparison to a rigid tool, during maneuvering, an articulating toll can use more direct routes, as the articulating section enables removal of the tip of an articulating tool from the region of an obstacle. For example, instead of routing an endoscope around a body organ, the endoscope can articulate such that its tip is withdrawn to a sufficient height that the route of the endoscope can be directly across the organ.

Furthermore, the system has more flexibility in positioning. For example, the angle of the field of view can be changed by changing the articulation of the endoscope, with only minimal change of the position of the main part of the endoscope.

In some embodiments, the device of the present invention additionally comprises a touchscreen used as the display screen on which the image of the field of view of the laparoscope is displayed. In these embodiments, in order to direct the laparoscope, the surgeon touches the portion of the image toward which he wants the laparoscope to move and automatic control software controls the motion of the laparoscope towards the goal. Thus, in preferred embodiments, the surgeon need not concern himself with the mechanics of repositioning; a brief touch on the display screen and he can return his hand to the instrument while the laparoscope automatically repositions itself.

In preferred variants of embodiments including a touchscreen, the surgeon directs the instrument to the desired location by touching the portion of the screen showing the image of the desired location. For example, to direct the laparoscope to put the tip of the appendix in the center of the screen, the surgeon would touch the image of the tip of appendix on the screen. In these embodiments, the surgeon touches the screen only briefly; continued pressure is not needed to direct the laparoscope to the desired position.

In other variants of embodiments including a touchscreen, the screen contains at least one graphical direction indicator, which can be at least one arrow, line or pointer or, preferably, a direction rose with 4, 8 or 16 indicators. In some variants of these embodiments, the surgeon touches the appropriate indicator, for non-limiting example, the one pointing at 45° clockwise from the vertical, and the laparoscope moves so that the center of its field of view moves towards the upper right portion of the image. In these embodiments, the surgeon needs to keep his hand on the touchscreen until the maneuver is complete.

In other variants of embodiments with graphical indicators on the touchscreen, the indicator comprises a direction rose (100), the surgeon touches a position anywhere on the graphical indicator and the laparoscope moves so that the center of its field of view moves towards the direction indicated by the position of the touch. For example, in the direction rose (100) shown in Fig. 1, the uppermost point (110) indicates movement towards the top of the screen, the rightmost point (120), movement towards the right, the lowest point (130), movement towards the bottom of the screen, and the leftmost point (140), movement towards the left. If the surgeon touches a position 55° clockwise from the vertical, the laparoscope will move so that the center of its field of view moves towards the upper right portion of the image, at an angle 55° clockwise from the vertical. In these embodiments, the surgeon needs to keep his hand on the touchscreen until the maneuver is complete.

In other variants of embodiments with graphical indicators on the touchscreen, the location of the touch on the indicator defines the speed at which the center of the field of view moves. For non- limiting example, the further from the center of the direction rose, the faster the motion.

In yet other embodiments with a touchscreen, the direction of motion is indicated by words appearing on the screen such as, but not limited to, left, right, up, down, forward, back, zoom, zoom in, zoom out, and any combination thereof.

Combinations of the above embodiments will be obvious to one skilled in the art.

Many other means of indication direction of movement via a touchscreen will be obvious to one skilled in the art. In yet other embodiments, voice commands are used to direct the endoscope. In such embodiments, the direction of motion can be indicated by words spoken by the surgeon such as, but not limited to, left, right, up, down, forward, back, zoom, zoom in, zoom out, and any combination thereof.

In some variants of embodiments employing voice commands, the surgeon can provide an angular designation, such as, but not limited to, a numerical value or a compass rose designation. Non- limiting examples of numerical values include 60°, 75° clockwise, 30° west of north. Other examples will be obvious to one skilled in the art. Non-limiting examples of compass rose designations are north-northwest, NNW, and southeast by south.

In still other embodiments, eye movements are used to direct the endoscope. Typically, in such embodiments, the endoscope moves in the direction in which the surgeon moves his eyes. For non-limiting example, if the surgeon looks to the right, the endoscope moves to the right of the field of view, if the surgeon looks up, the endoscope moves towards the top of the field of view, and similarly for eye movements to the left or downward.

According to different embodiments of the present invention, the surgical controlling system comprises the following components: a. at least one surgical tool adapted to be inserted into a surgical environment of a human body for assisting a surgical procedure, at least one said tool being an articulating tool; b. at least one location estimating means adapted to real-time estimate/locate the location (i.e., the 3D spatial position) of the at least one surgical tool at any given time t; c. at least one movement detection means communicable with a movement-database and with said location estimating means; said movement-database is adapted to store said 3D spatial position of said at least one surgical tool at time tf and at time to_,- where tf > to,' said movement detection means is adapted to detect movement of said at least one surgical tool if the 3D spatial position of said at least one surgical tool at time tf is different than said 3D spatial position of said at least one surgical tool at time to_,- and, d. a controller having a processing means communicable with a database, the controller adapted to control the spatial position of the at least one surgical tool.

The initial time to can be the beginning of the surgical procedure, it can be the time at which the tool entered the body, it can be the time at the beginning of the current movement, or it can be the previous timestep in the current maneuver. In preferred embodiments, the processor will reset to as necessary during the surgical procedure. For non-limiting example, the difference in position between the location of the tool at the previous timestep and its location at the current timestep can be used to calculate the tool's current velocity while the difference in position between its current position and its position at the start of the current maneuver can be used to calculate the tool's overall direction of motion.

The location of the tool can be the location of the tool's tip, the location of a predetermined point on the tool's body, or the location of a predetermined point on the tool's handle. The position defining the location of the tool can be changed as needed, e.g., from the location of the body to the location of the tip.

In some embodiments, the surgical controlling system additionally comprises a touchscreen adapted to accept input of a location within the body, that location indicated by pressure on the portion of the touchscreen showing the image of the location.

In order to facilitate control, a number of motion control rules have been implemented, as described hereinbelow.

It is within the scope of the present invention that the database is adapted to store a predetermined set of rules according to which ALLOWED and RESTRICTED movements of the at least one surgical tool are determined, such that the spatial position of the at least one surgical tool is controlled by the controller according to the ALLOWED and RESTRICTED movements. In other words, each detected movement by said movement detection means of said at least one surgical tool is determined as either an ALLOWED movement or as a RESTRICTED movement according to said predetermined set of rules.

Thus, the present invention stores the 3D spatial position of each surgical tool at a current at time tf and at time to; where tf > to_. If the 3D spatial position of said at least one surgical tool at time tf is different than said 3D spatial position of said at least one surgical tool at time to movement of the tool is detected. Next the system analyses said movement according to said set of rule and process whether said movement is ALLOWED movement or RESTRICTED movement.

According to one embodiment of the present invention, the system prevents said movement, if said movement is a RESTRICTED movement. Said movement prevention is obtained by controlling a maneuvering system which prevents the movement of said surgical tool.

According to one embodiment of the present invention, the system does not prevent said movement, (if said movement is a RESTRICTED movement), but merely signals/alerts the user (i.e., the physician) of said RESTRICTED movement. According to another embodiment of the present invention, said surgical tool is an endoscope.

According to different embodiments of the present invention, the controller may provide a suggestion to the operator as to which direction the surgical tool has to move to or may be moved to.

Thus, according to a preferred embodiment of the present invention, the present invention provides a predetermined set of rules which define what is an "ALLOWED movement" of any surgical tool within the surgical environment and what is a "RESTRICTED movement" of any surgical tool within the surgical environment.

According to some embodiments the system of the present invention comprises a maneuvering subsystem communicable with the controller, the maneuvering subsystem is adapted to spatially reposition the at least one surgical tool during surgery according to the predetermined set of rules.

According to some embodiments, the controller may provide instructions to a maneuvering subsystem for spatially repositioning the location of the surgical tool. According to these instructions, only ALLOWED movements of the surgical tool will be performed. Preventing RESTRICTED movements is performed by: detecting the location of the surgical tool; processing all current rules; analyzing the movement of the surgical tool and preventing the movement if the tool's movement is a RESTRICTED movement.

According to some embodiments, system merely alerts the physician of a RESTRICTED movement of at least one surgical tool (instead of preventing said RESTRICTED movement).

Alerting the physician of RESTRICTED movements (or, alternatively preventing a RESTRICTED movement) is performed by: detecting the location of the surgical tool; processing all current rules; analyzing the movement of the surgical tool and informing the surgeon (the user of the system) if the tool's movement is an ALLOWED movement or a RESTRICTED movement.

Thus, according to a preferred embodiment of the present invention, if RESTRICTED movements are prevented, the same process (of detecting the location of the surgical tool; processing all current rules and analyzing the movement of the surgical tool) is followed except for the last movement, where the movement is prevented if the tool's movement is a RESTRICTED movement. The surgeon can also be informed that the movement is being prevented.

According to another embodiment, the above (alerting the physician and/or preventing the movement) is performed by detecting the location of the surgical tool and analyzing the surgical environment of the surgical tool. Following analysis of the surgical environment and detection of the location of the surgical tool, the system may assess all the risks which may follow a movement of the surgical tool in the predetermined direction. Therefore, each location in the surgical environment has to be analyzed so that any possible movement of the surgical tool will be classified as an ALLOWED movement or a RESTRICTED movement.

According to one embodiment of the present invention, the location of each tool is determined using image processing means and determining in real-time what is the 3D spatial location of each tool. It should be understood that the above mentioned "tool" may refer to the any location on the tool. For example, it can refer to the tip of the same, the body of the same and any combination thereof.

In some embodiments, avoidance of body organs is facilitated by means of a proximity sensor on the circumference of at least one tool. In these embodiments, if the distance between the tool and another object in the surgical environment, such as, but not limited to, an organ or another tool, is less than a predetermined distance, the proximity sensor activates, thereby notifying the control system that at least one tool is too close to another object in the surgical environment.

In some variants of embodiments with proximity sensors, the proximity sensor not only determined whether an object is within a predetermined distance of the sensor, it also determines, for objects within the predetermined distance, the distance between the sensor and the object.

Hereinbelow, determination of the 3D location of each tool includes determination by means of a proximity sensor as well as determination by means of image processing.

The predetermined set of rules which are the essence of the present invention are adapted to take into consideration all the possible factors which may be important during the surgical procedure. The predetermined set of rules may comprise the following rules or any combination thereof: a. a route rule; b. an environment rule; c. an operator input rule; d. a proximity rule; e. a collision prevention rule; f. a history based rule; g- a tool-dependent ALLOWED and RESTRICTED movements rule. h. a most used tool rule; i. a right tool rule; j- a left tool rule; k. a field of view rule;

1. a no fly zone rule; m. an operator input rule; n. a preferred volume zone rule; o. a preferred tool rule;

P- a movement detection rule, and q- a tagged tool rule.

Thus, for example, the collision prevention rule defines a minimum distance below which two or more tools should not be brought together (i.e., there is minimum distance between two or more tools that should be maintained). If the movement of one tool will cause it to come dangerously close to another tool (i.e., the distance between them, after the movement, is smaller than the minimum distance defined by the collision prevention rule), the controller either alerts the user that the movement is a RESTRICTED movement or does not permit the movement.

It should be emphasized that all of the above (and the following disclosure) is enabled by constantly monitoring the surgical environment, and identifying and locating the 3D spatial location of each element/tool in the surgical environment.

The identification is provided by conventional means known to any skilled in the art (e.g., image processing, optical means etc.).

The following provides explanations for each of the above mentioned rules and its functions:

According to some embodiments, the route rule comprises a predefined route in which the at least one surgical tool is adapted to move within the surgical environment; the ALLOWED movements are movements in which the at least one surgical tool is located within the borders of the predefined route, and the RESTRICTED movements are movements in which the at least one surgical tool is located out of the borders of the predefined route. Thus, according to this embodiment, the route rule comprises a communicable database storing at least one predefined route in which the at least one surgical tool is adapted to move within the surgical environment; the predefined route comprises n 3D spatial positions of the at least one surgical tool in the route; n is an integer greater than or equal to 2; ALLOWED movements are movements in which the at least one surgical tool is located substantially in at least one of the n 3D spatial positions of the predefined route, and RESTRICTED movements are movements in which the location of the at least one surgical tool is substantially different from the n 3D spatial positions of the predefined route.

In other words, according to the route rule, each of the surgical tool's courses (and path in any surgical procedure) is stored in a communicable database. ALLOWED movements are defined as movements in which the at least one surgical tool is located substantially in at least one of the stored routes; and RESTRICTED movements are movements in which the at least one surgical tool is in a substantially different location than any location in any stored route.

According to some embodiments, the environmental rule is adapted to determine ALLOWED and RESTRICTED movements according to hazards or obstacles in the surgical environment as received from an endoscope or other sensing means. Thus, according to this embodiment, the environmental rule comprises a comprises a communicable database; the communicable database is adapted to received real-time images of the surgical environment and is adapted to perform realtime image processing of the same and to determine the 3D spatial position of hazards or obstacles in the surgical environment; the environmental rule is adapted to determine ALLOWED and RESTRICTED movements according to hazards or obstacles in the surgical environment, such that RESTRICTED movements are movements in which at least one surgical tool is located substantially in at least one of the 3D spatial positions, and ALLOWED movements are movements in which the location of at least one surgical tool is substantially different from the 3D spatial positions.

In other words, according to the environment rule, each element in the surgical environment is identified so as to establish which is a hazard or obstacle (and a path in any surgical procedure) and each hazard and obstacle (and path) is stored in a communicable database. RESTRICTED movements are defined as movements in which the at least one surgical tool is located substantially in the same location as that of the hazards or obstacles; and the ALLOWED movements are movements in which the location of the at least one surgical tool is substantially different from that of all of the hazards or obstacles.

According to other embodiments, hazards and obstacles in the surgical environment are selected from a group consisting of tissues, surgical tools, organs, endoscopes and any combination thereof.

According to some embodiments, the operator input rule is adapted to receive an input from the operator of the system regarding the ALLOWED and RESTRICTED movements of the at least one surgical tool. Thus, according to this embodiment, the operator input rule comprises a communicable database; the communicable database is adapted to receive an input from the operator of the system regarding ALLOWED and RESTRICTED movements of the at least one surgical tool.

According to other embodiments, the input comprises n 3D spatial positions; n is an integer greater than or equal to 2; wherein at least one of which is defined as an ALLOWED location and at least one of which is defined as a RESTRICTED location, such that the ALLOWED movements are movements in which the at least one surgical tool is located substantially in at least one of the n 3D ALLOWED spatial positions, and the RESTRICTED movements are movements in which the location of the at least one surgical tool is substantially different from the n 3D ALLOWED spatial positions.

According to other embodiments, the input comprises at least one rule according to which ALLOWED and RESTRICTED movements of the at least one surgical tool are determined, such that the spatial position of the at least one surgical tool is controlled by the controller according to the ALLOWED and RESTRICTED movements.

According to other embodiments, the operator input rule can convert an ALLOWED movement to a RESTRICTED movement and a RESTRICTED movement to an ALLOWED movement.

According to some embodiments, the proximity rule is adapted to define a predetermined distance between the at least one surgical tool and at least one another surgical tool; the ALLOWED movements are movements which are within the range or out of the range of the predetermined distance, and the RESTRICTED movements which are out of the range or within the range of the predetermined distance; the ALLOWED movements and the RESTRICTED movements are defined according to different ranges. Thus, according to this embodiment, the proximity rule is adapted to define a predetermined distance between at least two surgical tools. In a preferred embodiment, the ALLOWED movements are movements which are within the range of the predetermined distance, while the RESTRICTED movements which are out of the range of the predetermined distance. In another preferred embodiment, the ALLOWED movements are movements which are out of the range of the predetermined distance, while the RESTRICTED movements are within the range of the predetermined distance

It should be pointed out that the above mentioned distance can be selected from the following:

(a) the distance between the tip of the first tool and the tip of the second tool;

(b) the distance between the body of the first tool and the tip of the second tool; (c) the distance between the body of the first tool and the body of the second tool;

(d) the distance between the tip of the first tool and the body of the second tool; and any combination thereof.

According to another embodiment, the proximity rule is adapted to define a predetermined angle between at least three surgical tools; ALLOWED movements are movements which are within the range or out of the range of the predetermined angle, and RESTRICTED movements are movements which are out of the range or within the range of the predetermined angle.

According to some embodiments, the collision prevention rule is adapted to define a predetermined distance between the at least one surgical tool and an anatomical element within the surgical environment (e.g. tissue, organ, another surgical tool or any combination thereof); the ALLOWED movements are movements which are in a range that is larger than the predetermined distance, and the RESTRICTED movements are movements which is in a range that is smaller than the predetermined distance.

According to another embodiment, the anatomical element is selected from a group consisting of tissue, organ, another surgical tool or any combination thereof.

According to some embodiments, the surgical tool is an endoscope. The endoscope is adapted to provide real-time images of the surgical environment.

According to some embodiments, the right tool rule is adapted to determine the ALLOWED movement of the endoscope according to the movement of a surgical tool in a specified position in relation to the endoscope, preferably positioned to right of the same. According to this rule, the tool which is defined as the right tool is constantly tracked by the endoscope. According to some embodiments, the right tool is defined as the tool positioned to the right of the endoscope; according to other embodiments, any tool can be defined as the right tool. An ALLOWED movement, according to the right tool rule, is a movement in which the endoscope field of view is moved to a location substantially the same as the location of the right tool, thereby tracking the right tool. A RESTRICTED movement, according to the right tool rule, is a movement in which the endoscope field of view is moved to a location substantially different from the location of the right tool.

According to some embodiments, the left tool rule is adapted to determine the ALLOWED movement of the endoscope according to the movement of a surgical tool in a specified position in relation to the endoscope, preferably positioned to left of the same. According to this rule, the tool which is defined as the left tool is constantly tracked by the endoscope. According to some embodiments, the left tool is defined as the tool positioned to the left of the endoscope; according to other embodiments, any tool can be defined as the left tool. An ALLOWED movement, according to the left tool rule, is a movement in which the endoscope field of view is moved to a location substantially the same as the location of the left tool. A RESTRICTED movement, according to the left tool rule, is a movement in which the endoscope field of view is moved to a location substantially different from the location of the left tool.

According to some embodiments, the field of view rule is adapted to define a field of view and maintain that field of view. The field of view rule is defined such that if the endoscope is adapted to track a predetermined set of tools in a desired field of view, when one of those tools is no longer in the field of view, the rule instructs the endoscope to zoom out so as to reintroduce the tool into the field of view. Thus, according to this embodiment, the field of view rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; the combination of all of the n 3D spatial positions provides a predetermined field of view; the field of view rule is adapted to determine the ALLOWED movement of the endoscope within the n 3D spatial positions so as to maintain a constant field of view, such that the ALLOWED movements are movements in which the endoscope is located substantially in at least one of the n 3D spatial positions, and the RESTRICTED movements are movements in which the location of the endoscope is substantially different from the n 3D spatial positions.

Thus, according to another embodiment of the field of view rule, the field of view rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; the combination of all of the n 3D spatial positions provides a predetermined field of view. The field of view rule further comprises a communicable database of m tools and the 3D spacial locations of the same, where m is an integer greater than or equal to 1 and where a tool can be a surgical tool, an anatomical element and any combination thereof. The combination of all of the n 3D spatial positions provides a predetermined field of view. The field of view rule is adapted to determine ALLOWED movement of the endoscope such that the m 3D spatial positions of the tools comprise at least one of the n 3D spatial positions of the field of view, and RESTRICTED movements are movements in which the 3D spatial position of at least one tool is substantially different from the n 3D spatial positions of the field of view.

According to another embodiment, the preferred volume zone rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; the n 3D spatial positions provides the preferred volume zone; the preferred volume zone rule is adapted to determine the ALLOWED movement of the endoscope within the n 3D spatial positions and RESTRICTED movement of the endoscope outside the n 3D spatial positions, such that the ALLOWED movements are movements in which the endoscope is located substantially in at least one of the n 3D spatial positions, and the RESTRICTED movements are movements in which the location of the endoscope is substantially different from the n 3D spatial positions. In other words, the preferred volume zone rule defines a volume of interest (a desired volume of interest), such that an ALLOWED movement, according to the preferred volume zone rule, is a movement in which the endoscope (or any surgical tool) is moved to a location within the defined preferred volume. A RESTRICTED movement, according to the preferred volume zone rule, is a movement in which the endoscope (or any surgical tool) is moved to a location outside the defined preferred volume.

According to another embodiment, the preferred tool rule comprises a communicable database, the database stores a preferred tool; the preferred tool rule is adapted to determine the ALLOWED movement of the endoscope according to the movement of the preferred tool. In other words, the preferred tool rule defines a preferred tool (i.e., a tool of interest) that the user of the system wishes to track. An ALLOWED movement, according to the preferred tool rule, is a movement in which the endoscope is moved to a location substantially the same as the location of the preferred tool. A RESTRICTED movement is a movement in which the endoscope is moved to a location substantially different from the location of the preferred tool. Thus, according to the preferred tool rule the endoscope constantly tracks the preferred tool, such that the field of view, as seen from the endoscope, is constantly the preferred tool. It should be noted that the user may define in said preferred tool rule to constantly track the tip of said preferred tool or alternatively, the user may define in said preferred tool rule to constantly track the body or any location on the preferred tool.

According to some embodiments, the no fly zone rule is adapted to define a RESTRICTED zone into which no tool (or alternatively no predefined tool) is permitted to enter. Thus, according to this embodiment, the no fly zone rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; the n 3D spatial positions define a predetermined volume within the surgical environment; the no fly zone rule is adapted to determine a RESTRICTED movement if the movement is within the no fly zone and an ALLOWED movement if the movement is outside the no fly zone, such that RESTRICTED movements are movements in which the at least one surgical tool is located substantially in at least one of the n 3D spatial positions, and the ALLOWED movements are movements in which the location of the at least one surgical tool is substantially different from the n 3D spatial positions.

According to another embodiment, the most used tool rule is adapted to define (either real-time, during the procedure or prior to the procedure) which tool is the most used tool (i.e., the tool which is moved the most during the procedure) and to instruct the maneuvering subsystem to constantly position the endoscope to track the movement of this tool. Thus, according to this embodiment, the most used tool rule comprises a communicable database counting the number of movements of each of the surgical tools; the most used tool rule is adapted to constantly position the endoscope to track the movement of the surgical tool with the largest number of movements. In another embodiment of the most used tool rule, the communicable database measures the amount of movement of each of the surgical tools; the most used tool rule is adapted to constantly position the endoscope to track the movement of the surgical tool with the largest amount of movement.

According to another embodiment, the system is adapted to alert the physician of a RESTRICTED movement of at least one surgical tool. The alert can be audio signaling, voice signaling, light signaling, flashing signaling and any combination thereof.

According to another embodiment, an ALLOWED movement is one permitted by the controller and a RESTRICTED movement is one denied by the controller.

According to another embodiment, the operator input rule is adapted to receive an input from the operator of the system regarding ALLOWED and RESTRICTED movements of the at least one surgical tool. In other words, the operator input rule receives instructions from the physician as to what can be regarded as ALLOWED movements and what are RESTRICTED movements. According to another embodiment, the operator input rule is adapted to convert an ALLOWED movement to a RESTRICTED movement and a RESTRICTED movement to an ALLOWED movement.

According to some embodiments, the history-based rule is adapted to determine the ALLOWED and RESTRICTED movements according to historical movements of the at least one surgical tool in at least one previous surgery. Thus, according to this embodiment, the history-based rule comprises a communicable database storing each 3D spatial position of each of the surgical tools, such that each movement of each surgical tool is stored; the history-based rule is adapted to determine ALLOWED and RESTRICTED movements according to historical movements of the at least one surgical tool, such that the ALLOWED movements are movements in which the at least one surgical tool is located substantially in at least one of the 3Ό spatial positions, and the RESTRICTED movements are movements in which the location of the at least one surgical tool is substantially different from the n 3D spatial positions.

According to some embodiments, the tool-dependent ALLOWED and RESTRICTED movements rule is adapted to determine ALLOWED and RESTRICTED movements according to predetermined characteristics of the surgical tool, where the predetermined characteristics of the surgical tool are selected from a group consisting of: physical dimensions, structure, weight, sharpness, and any combination thereof. Thus, according to this embodiment, the tool-dependent ALLOWED and RESTRICTED movements rule comprises a communicable database; the communicable database is adapted to store predetermined characteristics of at least one of the surgical tools; the tool-dependent ALLOWED and RESTRICTED movements rule is adapted to determine ALLOWED and RESTRICTED movements according to the predetermined characteristics of the surgical tool.

According to another embodiment, the predetermined characteristics of the surgical tool are selected from a group consisting of: physical dimensions, structure, weight, sharpness, and any combination thereof.

According to this embodiment, the user can define, e.g., the structure of the surgical tool he wishes the endoscope to track. Thus, according to the tool-dependent ALLOWED and RESTRICTED movements rule the endoscope constantly tracks the surgical tool having said predetermined characteristics as defined by the user.

According to another embodiment of the present invention, the movement detection rule comprises a communicable database comprising the real-time 3D spatial positions of each surgical tool; said movement detection rule is adapted to detect movement of at least one surgical tool. When a change in the 3D spatial position of that surgical tool is received, ALLOWED movements are movements in which the endoscope is re-directed to focus on the moving surgical tool.

According to another embodiment of the present invention, the tagged tool rule comprises means of tagging at least one surgical tool within the surgical environment such that, by maneuvering the endoscope, the endoscope is constantly directed to the tagged surgical tool. Thus, according to the tagged tool rule, the endoscope constantly tracks the preferred (i.e., tagged) tool, such that the field of view, as seen from the endoscope, is constantly maintained on the preferred (tagged) tool. It should be noted that the user can define the tagged tool rule to constantly track the tip of the preferred (tagged) tool, the body of the preferred (tagged) tool, or any other location on the preferred (tagged) tool.

According to another embodiment of the present invention, the system further comprises a maneuvering subsystem communicable with the controller. The maneuvering subsystem is adapted to spatially reposition the at least one surgical tool during a surgery according to the predetermined set of rules. According to some embodiments, the at least one location estimating means is at least one endoscope adapted to acquire real-time images of a surgical environment within the human body for the estimation of the location of at least one surgical tool.

According to another embodiment, the location estimating means comprise at least one selected from a group consisting of optical imaging means, radio frequency transmitting and receiving means, at least one mark on at least one surgical tool and any combination thereof.

According to another embodiment, the at least one location estimating means is an interface subsystem between a surgeon and at least one surgical tool, the interface subsystem comprising (a) at least one array comprising N regular light sources or N pattern light sources, where N is a positive integer; (b) at least one array comprising M cameras, where M is a positive integer; (c) optional optical markers and means for attaching the optical markers to at least one surgical tool; and (d) a computerized algorithm operable via the controller, the computerized algorithm adapted to record images received by each camera of each of the M cameras and to calculate therefrom the position of each of the tools, and further adapted to provide automatically the results of the calculation to the human operator of the interface.

It is well known that surgery is a highly dynamic procedure with a constantly changing environment which depends on many variables. A non-limiting list of these variables includes, for example: the type of the surgery, the working space (e.g., with foreign objects, dynamic uncorrelated movements, etc), the type of tools used during the surgery, changing background, relative movements, dynamic procedures, dynamic input from the operator and the history of the patient. Therefore, there is need for a system which is able to integrate all the variables by weighting their importance and deciding to which spatial position the endoscope should be relocated.

The present invention can be also utilized to improve the interface between the operators (e.g., the surgeon, the operating medical assistant, the surgeon's colleagues, etc.). Moreover, the present invention can be also utilized to control and/or direct an automated maneuvering subsystem to focus the endoscope on an instrument selected by the surgeon, or to any other region of interest. This may be performed in order to estimate the location of at least one surgical tool during a surgical procedure.

The present invention also discloses a surgical tracking system which is adapted to guide and relocate an endoscope to a predetermined region of interest in an automatic and/or a semi-automatic manner. This operation is assisted by an image processing algorithm(s) which is adapted to analyze the received data from the endoscope in real time, and to assess the surgical environment of the endoscope.

According to an embodiment, the system comprises a "smart" tracking subsystem, which receives instructions from a maneuvering function f(t) (t is the time) as to where to direct the endoscope and which instructs the maneuvering subsystem to relocate the endoscope to the required area.

The maneuvering function f(t) receives, as input, output from at least two instructing functions ¾(t), analyses their output and provides instruction to the "smart" tracking system (which eventually redirects the endoscope).

According to some embodiments, each instructing function g_z(t) is also given a weighting function, ¾(t).

The instructing functions g_z(t) of the present invention are functions which are configured to assess the environment of the endoscope and the surgery, and to output data which guides the tracking subsystem for controlling the spatial position of the maneuvering subsystem and the endoscope. The instructing functions g_z(t) may be selected from a group consisting of: a. a tool detection function g_/(t); b. a movement detection function g^t); c. an organ detection function gi(t); d. a collision detection function g^(t); e. an operator input function gj(t); f. a prediction function g_<j(t); g- a past statistical analysis function gz(t); h. a most used tool function g$(t); i. a right tool function g_?(t); j- a left tool function g_/o(t); k. a field of view function g//(t);

1. a preferred volume zone function gn(t); m. a no fly zone function g_/i(t); n. a proximity function g_/^(t); o. a tagged tool function g_/j(t); p. a preferred tool function g_/<j(t).

Thus, for example, the maneuvering function f(t) receives input from two instructing functions: the collision detection function g^(t) (the function providing information whether the distance between two elements is smaller than a predetermined distance) and from the most used tool function g§(t) (the function counts the number of times each tool is moved during a surgical procedure and provides information as to whether the most moved or most used tool is currently moving). The output given from the collision detection function g₄(t) is that a surgical tool is dangerously close to an organ in the surgical environment. The output given from the most used tool function g§(t) is that the tool identified statistically as the most moved tool is currently moving.

The maneuvering function f(t) then assigns each of the instructing functions with weighting functions a_z(t). For example, the most used tool function g₈(t) is assigned with a greater weight than the weight assigned to the collision detection function g^(t).

After the maneuvering function f(t) analyses the information received from the instructing functions g_/(t) and the weighting functions a_z(t) of each, the same outputs instructions to the maneuvering subsystem to re-direct the endoscope (either to focus on the moving tool or on the tool approaching dangerously close to the organ).

It should be emphasized that all of the above (and the following disclosure) is enabled by constantly monitoring and locating/identifying the 3D spatial location of each element/tool in the surgical environment.

According to some embodiments, the surgical tracking subsystem comprises: a. at least one endoscope adapted to acquire real-time images of a surgical environment within the human body; b. a maneuvering subsystem adapted to control the spatial position of the endoscope during the laparoscopic surgery; and, c. a tracking subsystem in communication with the maneuvering subsystem, adapted to control the maneuvering subsystem so as to direct and modify the spatial position of the endoscope to a region of interest. According to this embodiment, the tracking subsystem comprises a data processor. The data processor is adapted to perform real-time image processing of the surgical environment and to instruct the maneuvering subsystem to modify the spatial position of the endoscope according to input received from a maneuvering function f(t); the maneuvering function f(t) is adapted to (a) receive input from at least two instructing functions g_z(t), where z is Ι ,. , . ,η and n > 2 and where t is time; i and n are integers; and (b) to output instructions to the maneuvering subsystem based on the input from the at least two instructing functions g_z(t), so as to spatially position the endoscope to the region of interest.

According to one embodiment, the tool detection function g (t) is adapted to detect tools in the surgical environment. According to this embodiment, the tool detection function is adapted to detect surgical tools in the surgical environment and to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the detected surgical tools.

According to some embodiments, the functions g_z(t) may rank the different detected areas in the surgical environment according to a ranking scale (e.g., from 1 to 10) in which prohibited areas (i.e., areas which are defined as area to which the surgical tools are forbidden to 'enter) receive the lowest score (e.g., 1) and preferred areas (i.e., areas which are defined as area in which the surgical tools should be maintained) receive the highest score (e.g., 10).

According to a preferred embodiment, one function g (t) is adapted to detect tools in the surgical environment and inform the maneuvering function f(t) if they are in preferred areas or in prohibited areas.

According to some embodiments, the movement detection function g2(t) comprises a communicable database comprising the real-time 3D spatial positions of each of the surgical tools in the surgical environment; means to detect movement of the at least one surgical tool when a change in the 3D spatial positions is received, and means to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the moved surgical tool.

According to some embodiments, the organ detection function g^(t) is adapted to detect physiological organs in the surgical environment and to classify the detected organs as prohibited areas or preferred areas. For example, if the operator instructs the system that the specific surgery is kidney surgery, the organ detection function g^(t) will classify the kidneys (or one kidney, if the surgery is specified to be on a single kidney) as a preferred area and other organs will be classified as prohibited areas. According to another embodiment, the organ detection function is adapted to detect organs in the surgical environment and to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the detected organs. According to some embodiments, the right tool function is adapted to detect surgical tool positioned to right of the endoscope and to output instructions to the tracking subsystem to instruct the maneuvering system to constantly direct the endoscope on the right tool and to track the right tool.

According to another embodiment, the left tool function is adapted to detect surgical tool positioned to left of the endoscope and to output instructions to the tracking subsystem to instruct the maneuvering system to constantly direct the endoscope on the left tool and to track the left tool.

According to some embodiments, the collision detection function g₄(t) is adapted to detect prohibited areas within the surgical environment so as to prevent collisions between the endoscope and the prohibited areas. For example, if the endoscope is located in a narrow area in which a precise movement of the same is preferred, the collision detection function g^(t) will detect and classify different areas (e.g., nerves, veins, walls of organs) as prohibited areas. Thus, according to this embodiment, the collision prevention function is adapted to define a predetermined distance between the at least one surgical tool and an anatomical element within the surgical environment; and to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the surgical tool and the anatomical element within the surgical environment if the distance between the at least one surgical tool and an anatomical element is less than the predetermined distance. According to one embodiment of the present invention the anatomical element is selected from a group consisting of tissue, organ, another surgical tool and any combination thereof.

According to some embodiments, the operator input function gj(t) is adapted to receive an input from the operator. The input can be, for example: an input regarding prohibited areas in the surgical environment, an input regarding allowed areas in the surgical environment, or an input regarding the region of interest and any combination thereof. The operator input function gj(t) can receive instructions from the operator before or during the surgery, and respond accordingly. According to some embodiments, the operator input function may further comprise a selection algorithm for selection of areas selected from a group consisting of: prohibited areas, allowed areas, regions of interest, and any combination thereof. The selection may be performed via an input device (e.g., a touch screen).

According to some embodiments, the operator input function gj(t) comprises a communicable database; the communicable database is adapted to receive an input from the operator of the system; the input comprising n 3D spatial positions; n is an integer greater than or equal to 2; and to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the at least one 3D spatial position received.

According to some embodiments, the prediction function g₆(t) is adapted to provide data regarding a surgical environment at a time tf > to, wherein to is the present time and tf is a future time. The prediction function g<j(t) may communicate with a database which stores data regarding the environment of the surgery (e.g., the organs in the environment). This data may be used by the prediction function g^(t) for the prediction of expected or unexpected events or expected or unexpected objects during the operation. Thus, according to this embodiment, the prediction function g<j(t) comprises a communicable database storing each 3D spatial position of each of surgical tool within the surgical environment, such that each movement of each surgical tool is stored; the prediction function is adapted to (a) to predict the future 3D spatial position of each of the surgical tools (or each object); and, (b) to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the future 3D spatial position.

According to some embodiments, the past statistical analysis function g₇(t) is adapted to provide data regarding the surgical environment or the laparoscopic surgery based on past statistical data stored in a database. The data regarding the surgical environment may be for example: data regarding prohibited areas, data regarding allowed areas, data regarding the region of interest and any combination thereof. Thus, according to this embodiment, the past statistical analysis function g<$(t) comprises a communicable database storing each 3D spatial position of each of surgical tool within the surgical environment, such that each movement of each surgical tool is stored; the past statistical analysis function g^(t) is adapted to (a) perform statistical analysis on the 3D spatial positions of each of the surgical tools in the past; and, (b) to predict the future 3D spatial position of each of the surgical tools; and, (c) to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the future 3D spatial position. Thus, according to the past statistical analysis function gz(t), the past movements of each tool are analyzed and, according to this analysis, a prediction of the tool's next move is provided.

According to another embodiment, the most used tool function g§(t) comprises a communicable database counting the amount of movement of each surgical tool located within the surgical environment; the most used tool function is adapted to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to constantly position the endoscope to track the movement of the most moved surgical tool. The amount of movement of a tool can be defined as the total number of movements of that tool or the total distance the tool has moved.

According to some embodiments, the right tool function gp(t) is adapted to detect at least one surgical tool in a specified position in relation to the endoscope, preferably positioned to right of the endoscope and to output instructions to the tracking subsystem to instruct the maneuvering subsystem to constantly direct the endoscope to the right tool and to track the same. According to preferred embodiments, the right tool is defined as the tool positioned to the right of the endoscope; according to other embodiments, any tool can be defined as the right tool.

According to another embodiment, the left tool function g_/o(t) is adapted to detect at least one surgical tool in a specified position in relation to the endoscope, preferably positioned to left of the endoscope and to output instructions to the tracking subsystem to instruct the maneuvering subsystem to constantly direct the endoscope to the left tool and to track the same. According to preferred embodiments, the left tool is defined as the tool positioned to the left of the endoscope; according to other embodiments, any tool can be defined as the left tool. .

According to another embodiment, the field of view function g//(t) comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; the combination of all of the n 3D spatial positions provides a predetermined field of view; the field of view function is adapted to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to at least one 3D spatial position substantially within the n 3D spatial positions so as to maintain a constant field of view.

According to another embodiment, the preferred volume zone function gn( ) comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; the n 3D spatial positions provide the preferred volume zone; the preferred volume zone function g 2(t) is adapted to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to at least one 3D spatial position substantially within the preferred volume zone.

According to another embodiment, the no fly zone function g_/i(t) comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; the n 3D spatial positions define a predetermined volume within the surgical environment; the no fly zone function g/i(t) is adapted to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to at least one 3D spatial position substantially different from all the n 3D spatial positions.

According to some embodiments, the proximity function g_/^(t) is adapted to define a predetermined distance between at least two surgical tools; and to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the two surgical tools if the distance between the two surgical tools is less than or if it is greater than the predetermined distance.

According to another embodiment, the proximity function g (t) is adapted to define a predetermined angle between at least three surgical tools; and to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the three surgical tools if the angle between the two surgical tools is less than or if it is greater than the predetermined angle.

According to another embodiment, the preferred volume zone function comprises communicable database comprising n 3D spatial positions; n is an integer greater than or equals to 2; the n 3D spatial positions provides the preferred volume zone; the preferred volume zone function is adapted to output instructions to the tracking subsystem to instruct the maneuvering system to direct the endoscope to the preferred volume zone.

According to another embodiment, the field of view function comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equals to 2; the combination of all of the n 3D spatial positions provides a predetermined field of view; the field of view function is adapted to output instructions to the tracking subsystem to instruct the maneuvering system to direct the endoscope to at least one 3D spatial position substantially within the n 3D spatial positions so as to maintain a constant field of view.

According to another embodiment, the no fly zone function comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equals to 2; the n 3D spatial positions define a predetermined volume within the surgical environment; the no fly zone function is adapted to output instructions to the tracking subsystem to instruct the maneuvering system to direct the endoscope to at least one 3D spatial position substantially different from all the n 3D spatial positions.

According to another embodiment, the most used tool function comprises a communicable database counting the amount of movement of each surgical tool located within the surgical environment; the most used tool function is adapted to output instructions to the tracking subsystem to instruct the maneuvering system to direct the endoscope to constantly position the endoscope to track the movement of the most moved surgical tool.

According to some embodiments, the prediction function g₆(t) is adapted to provide data regarding a surgical environment in a time tf > t, wherein t is the present time and tf is the future time. The prediction function g_<j(t) may communicate with a database which stores data regarding the environment of the surgery (e.g., the organs in the environment). This data may be used by the prediction function g^(t) for the prediction of expected or unexpected events or object during the operation. Thus, according to this embodiment, the prediction function comprises a communicable database storing each 3D spatial position of each of surgical tool within the surgical environment, such that each movement of each surgical tool is stored; the prediction function is adapted to (a) to predict the future 3D spatial position of each of the surgical tools; and, (b) to output instructions to the tracking subsystem to instruct the maneuvering system to direct the endoscope to the future 3D spatial position.

According to some embodiments, the past statistical analysis function g₇(t) is adapted to provide data regarding the surgical environment or the laparoscopic surgery based on past statistical data stored in a database. The data regarding the surgical environment may be for example: data regarding prohibited areas, data regarding allowed areas, data regarding the region of interest. Thus, according to this embodiment, the past statistical analysis function comprises a communicable database storing each 3D spatial position of each of surgical tool within the surgical environment, such that each movement of each surgical tool is stored; the past statistical analysis function is adapted to (a) statistical analyze the 3D spatial positions of each of the surgical tools in the past; and, (b) to predict the future 3D spatial position of each of the surgical tools; and, (c) to output instructions to the tracking subsystem to instruct the maneuvering system to direct the endoscope to the future 3D spatial position. Thus, according to the past statistical analysis function gz(t), the past movements of each tool are analyzed and according to this analysis a future prediction of the tool's next move is provided.

According to some embodiments, preferred tool function comprises a communicable database, the database stores a preferred tool; the preferred tool function is adapted to output instructions to the tracking subsystem to instruct the maneuvering system to constantly direct the endoscope to the preferred tool, such that said endoscope constantly tracks said preferred tool.

Thus, according to the preferred tool function the endoscope constantly tracks the preferred tool, such that the field of view, as seen from the endoscope, is constantly maintained on said preferred tool. It should be noted that the user may define in said preferred tool function to constantly track the tip of said preferred tool or alternatively, the user may define in said preferred tool function to constantly track the body or any location on the preferred tool.

According to some embodiments, the tagged tool function g j(t) comprises means adapted to tag at least one surgical tool within the surgical environment and to output instructions to the tracking subsystem to instruct the maneuvering subsystem to constantly direct the endoscope to the tagged surgical tool. Thus, according to the tagged tool function, the endoscope constantly tracks the preferred (i.e., tagged) tool, such that the field of view, as seen from the endoscope, is constantly maintained on the preferred (tagged) tool. It should be noted that the user can define the tagged tool function to constantly track the tip of the preferred (tagged) tool, the body of the preferred (tagged) tool, or any other location on the preferred (tagged) tool.

According to some embodiments, the means are adapted to constantly tag at least one surgical tool within the surgical environment.

According to some embodiments, the preferred tool function g/_<j(t) comprises a communicable database. The database stores a preferred tool; and the preferred tool function is adapted to output instructions to the tracking subsystem to instruct the maneuvering subsystem to direct the endoscope to the preferred tool.

According to some embodiments, the system further comprises means adapted to re-tag the at least one of the surgical tools until a desired tool is selected.

According to some embodiments, the system further comprises means adapted to toggle the surgical tools. According to some embodiments, the toggling is performed manually or automatically.

According to different embodiments of the present invention, the weighting functions a_z(t) are time- varying functions (or constants), the value of which is determined by the operator or the output of the instructing functions g_z(t). For example, if a specific function g_z(t) detected an important event or object, its weighting functions a_z(t) may be adjusted in order to elevate the chances that the maneuvering function f(t) will instruct the maneuvering subsystem to move the endoscope towards this important event or object.

According to different embodiments of the present invention, the tracking subsystem may implement various image processing algorithms which may also be algorithms that are well known in the art. The image processing algorithms may be for example: image stabilization algorithms, image improvement algorithms, image compilation algorithms, image enhancement algorithms, image detection algorithms, image classification algorithms, image correlations with the cardiac cycle or the respiratory cycle of the human body, smoke reduction algorithms, vapor reduction algorithms, steam reduction algorithms and any combination thereof. Smoke, vapor and steam reduction algorithms may be needed as it is known that, under certain conditions, smoke, vapor or steam may be emitted by or from the endoscope. The image processing algorithm may also be implemented and used to analyze 2D or 3D representations which may be rendered from the realtime images of the surgical environment.

According to different embodiments, the endoscope may comprise an image acquisition device selected from a group consisting of: a camera, a video camera, an electromagnetic sensor, a computer tomography imaging device, a fluoroscopic imaging device, an ultrasound imaging device, and any combination thereof.

According to some embodiments, the system may also comprise a display adapted to provide input or output to the operator regarding the operation of the system. The display may be used to output the acquired real-time images of a surgical environment with augmented reality elements. The display may also be used for the definition of the region of interest by the operator.

According to some embodiments, the endoscope may be controlled be an endoscope controller for performing operations such as: acquiring the real-time images and zooming-in to a predetermined area. For example, the endoscope controller may cause the endoscope to acquire the real-time images in correlation with the cardiac cycle or the respiratory cycle of a human body.

According to different embodiments, the data processor of the present invention may operate a pattern recognition algorithm for assisting the operation of the instructing functions g_z(t). The pattern recognition algorithm may be used as part of the image processing algorithm.

Reference is made now to Fig. 2, which is a general schematic view of an embodiment of a surgical tracking system 100. In this figure are illustrated surgical instruments 17b and 17c and an endoscope 21 which may be maneuvered by means of maneuvering subsystem 19 according to the instructions received from a tracking subsystem operable by computer 15.

According to one embodiment of the present invention as defined in the above, the user may define the field of view function as constantly monitoring at least one of surgical instruments 17b and 17c.

According to this embodiment, the surgical tracking system 100 may also comprise one or more button operated wireless transmitters 12a, which transmit, upon activation, a single code wave 14 through aerial 13 to connected receiver 11 that produces a signal processed by computer 15, thereby directing and modifying the spatial position of endoscope 21 to the region of interest, as defined by the field of view function.

Alternatively, according to the proximity rule, if the distance between the surgical instruments 17b and 17c is smaller than a predetermined distance (as defined by the collision prevention rule), the system alerts the user that any movement of either one of the surgical instruments 17b and 17c that will reduce the distance is a RESTRICTED movement.

In preferred embodiments of the present system, the system comprises all the mechanisms required to control fully the movement of an articulated endoscope so that the position and angle of the tip of the endoscope are fully under control. Such control is preferably automatic, as described herein, but it can be manual and controlled by a joystick or other control under the command of a surgeon.

In some embodiments, a standard articulating endoscope, such as the Stryker™ articulating endoscope is used. In other embodiments, an integral articulating endoscope is used.

Figs. 3a-b show an embodiment wherein the fine control means is a control mechanism (1830) which attaches to the endoscope (1810). The fine control mechanism attaches to the manual controls (1820) for the articulating endoscope via a connector (1840). In a preferred embodiment, the connector can connect any endoscope control means with any articulating endoscope. Fig. 3a shows the fine control mechanism (1830) before it is attached to the articulating endoscope (1810), while Fig. 3b shows the control mechanism (1830) attached to the articulating endoscope (1810), with the endoscope manual control (1840) connected to the fine control mechanism via the connector (1830).

In some embodiments, such as that shown in Fig. 3, hardware control of the articulation is used, with the current system in effect replacing the surgeon by moving the controls of the articulating tool. In other embodiments, software control is used, with the current system in effect replacing the controls of the articulating tool so that the tool articulates based on commands coming directly from the current system rather than via the tool's manual controls. Fig. 4 shows an embodiment of the articulating endoscope (1810) in use. The endoscope (1810) is attached to the zoom mechanism of the coarse control system (1960), which is attached to the articulating arm (1970) of the coarse control system. The fine control mechanism (1830) is attached to the articulating endoscope (1810) and also enabled to be controlled (either in a wired manner or wirelessly) either automatically by the control system or manually by the endoscope operator. The fine control mechanism (1840) is also connected to the manual controls (1922, 1924) of the articulating endoscope. In this example, one control (1922) is forward and one (1924) is backward, turning the endoscope tip (1950) toward the right of the figure.

Fig. 5a-d shows articulation of an embodiment of the articulating endoscope. Fig. 5a illustrates the flexibility of the articulating tip, showing it in typical positions - bent forwards, out of the plane of the paper (1952), to the right (1954), downward (1956), and to the left and backward, into the plane of the paper (1958).

Figs. 5b-d illustrate the articulating tip (1950) in use, following the movements of the tip (2082) of a medical instrument (2080). In Fig. 5b, the endoscope tip (1950) is straight; it is not yet following the tip of the instrument (2082). In Fig. 5c, the instrument tip (2082) has moved to the right and the tip of the endoscope (1950) has turned right to follow the tip (2082) of the instrument. It can be seen from the angle of the endoscope (1950) that the pivoting point of the endoscope has not changed, although the field of view of the endoscope (1950) has changed significantly. In Fig. 5d, the instrument tip (2082) has moved towards the endocope and forward, out of the plane of the paper. The tip of the endoscope (1950) has rotated to follow the movement of the instrument tip (2082), but the pivoting point of the endoscope has not changed. It is clear from Figs. 5a-5d that use of the articulating endoscope allows the surgeon a much larger field of view than would be possible with only movement of an endoscope around the pivoting point. Use of an articulating endoscope also minimizes movement of the whole endoscope relative to the pivoting point, which has the possibility of causing unwanted movement of the pivoting point and, therefore, unwanted movement of the field of view.

EXAMPLES

Examples are given in order to prove the embodiments claimed in the present invention. The example, which is a clinical test, describes the manner and process of the present invention and set forth the best mode contemplated by the inventors for carrying out the invention, but are not to be construed as limiting the invention. In the examples below, similar numbers refer to similar parts in all of the figures.

In Figs. 6 - 19 in the examples below, for simplicity and clarity, a rigid tool has been illustrated although the rules are equally applicable to both rigid and articulating tools.

Example 1 - Tracking system with collision avoidance system

One embodiment of such a rule-based system will comprise the following set of commands:

Detection (denoted by Gd):

Gdl Tool location detection function

Gd2 Organ (e.g. Liver) detection function

Gd3 Movement (vector) calculation and estimation function

Gd4 Collision probability detection function

Tool Instructions (denoted Gt):

Gtl Move according to manual command

Gt2 Stop movement

The scenario - manual move command by the surgeon:

Locations Gdl(t) and Gd2(t) are calculated in real time at each time step (from an image or location marker).

Tool movement vector Gd3(t) is calculated from Gdl(t) as the difference between the current location and at least one previous location (probably also taking into account previous movement vectors).

The probability of collision - Gd4(t) - is calculated, for example, from the difference between location Gdl and location Gd2 (the smaller the distance, the closer the proximity and the higher the probability of collision), from movement vector Gd3(t) indicating a collision, etc.

Tool Instructions Gtl Weight function a (t) = 1 If Gtl(t) < a predetermined threshold and 0 otherwise

Tool Instructions Gt2 Weight function oi2(t) = 1 If Gt2(t) > a predetermined threshold and 0 otherwise

Tool Instructions = <x_/(t) * Gtl + a₂(t) * Gt2(t); In reference to Fig. 6, which shows, in a non-limiting manner, an embodiment of a tracking system and collision avoidance system. The system tracks a tool 310 and the liver 320, in order to determine whether a collision between the tool 310 and the liver 320 is possible within the next time step. Figs. 6a and 6b show how the behavior of the system depends on the distance 330 between the tool 310 and the liver 320, while Figs. 6c and 6d show how movement of the tool 310 affects the behavior. In Fig. 6a, the distance 330 between the tool 310 and the liver 320 is large enough that a collision is not possible in that time step. Since no collision is possible, no movement of the tool is commanded. In Fig. 6b, the distance 330 between the tool 310 and the liver 320 is small enough that a collision is likely. In the embodiment illustrated, a movement 340 is commanded to move the tool 310 away from the liver 320. In other embodiments, the system prevents movement 350, but does not command movement 340; in such embodiments, the tool 310 will remain close to the liver 320. In yet other embodiments, the system warns/signals the operator that the move is RESTRICTED, but does not restrict movement 350 or command movement 340 away from the liver. Such a warning/signaling can be visual or aural, using any of the methods known in the art.

Figs. 6c and 6d illustrate schematically the effect of the movement of tool 310 on the collision avoidance system. In Figs. 6c and 6d, the tool 310 is close enough to the liver 320 that a collision between the two is possible. If the system tracked only the positions of the tool 310 and the liver 320, then motion of the tool 310 away from the liver 320 would be commanded. Fig. 6c illustrates the effect of a movement 350 that would increase the distance between tool 310 and liver 320. Since the movement 350 is away from liver 320, no collision is possible in this time step and no movement of the tool 310 is commanded.

In Fig. 6d, tool 310 is the same distance from liver 320 as in Fig. 6c. However, in Fig. 6d, the movement 350 of the tool 310 is toward the liver 320, making a collision between tool 310 and liver 320 possible. In some embodiments, a movement 340 is commanded to move the tool 310 away from the liver 320. In other embodiments, the system prevents movement 350, but does not command movement 340; in this embodiment the tool 310 will remain close to the liver 320. In yet other embodiments, the system warns the operator that move is RESTRICTED, but does not restrict movement 350 or command movement 340 away from the liver. Such a warning can be visual or aural, using any of the methods known in the art.

As a non-limiting example, in an operation on the liver, the collision detection function can warn the operator that a collision between a tool and the liver is likely but not prevent the collision. In an operation on the gall bladder, the collision detection function can prevent a collision between the tool and the liver, either by preventing the movement or by commanding a movement redirecting the tool away from the liver,

Example 2 - Tracking system with soft control - fast movement when nothing is nearby, slow movement when something is close

One embodiment of such rule-based system comprises the following set of commands: Detection (denoted by Gd):

Main Tool location detection function (denoted by GdM);

Gd-tooll-K - Tool location detection function;

Gd-organ2-L - Organ (e.g. Liver) detection function;

Gd3 Main Tool Movement (vector) calculation and estimation function;

Gd4 Proximity probability detection function;

Tool Instructions (denoted Gt):

Gtl Movement vector (direction and speed) according to manual command The scenario - manual move command by the surgeon:

Locations GdM(t), Gd-tooll-K(t) and Gd-organ2-L(t) are calculated in real time at each time step (from image or location marker).

Main Tool Movement Vector Gd3(t) is calculated per GdM (t) as the difference between the current location and at least one previous location (probably also taking into account previous movement vectors)

The proximity of the main tool to other tools - Gd4(t) - is calculated, for example, as the smallest of the differences between the main tool location and the other tools' locations.

Tool Instructions Gtl Weight function a (t) is proportional to tool proximity function Gd4(t), the closer the tool the slower the movement so that, for example a₂(t) = Gd4 / maximum(Gd4) or α₂(ί) = log (Gd4 / maximum(Gd4)) where maximum(Gd4) is the maximum distance which is likely to result in a collision given the distances, the speed of the tool and the movement vector. Tool Instructions = a (t) * Gtl .

Example 3 - Tracking system with no-fly rule/function

In reference to Fig. 7a-d, which shows, in a non-limiting manner, an embodiment of a tracking system with no-fly rule. The system tracks a tool 310 with respect to a no-fly zone (460), in order to determine whether the tool will enter the no-fly zone (460) within the next time step. In this example, the no-fly zone 460 surrounds the liver.

Figs. 7a and 7b show how the behavior of the system depends on the location of the tool tip with respect to the no-fly zone, while Figs. 7c and 7d show how movement of the tool affects the behavior.

In Fig. 7a, the tool 310 is outside the no-fly zone rule/function 460 and no movement of the tool is commanded. In Fig. 7b, the tool 310 is inside the no-fly zone 460.

The no-fly zone rule/function performs as follows:

In the embodiment illustrated, a movement 350 is commanded to move the tool 310 away from the no-fly zone 460. In other embodiments, the system prevents movement further into the no-fly zone (refers as movement 340, see Fig. 7c), but does not command movement 340; in such embodiments, the tool 310 will remain close to the no-fly zone 460.

In yet other embodiments, the system warns/signals the operator that the move is RESTRICTED, but does not restrict movement further into the no-fly zone or command movement 340 away from the no-fly zone 460. Such a warning/signaling can be visual or aural, using any of the methods known in the art.

Figs. 7c and 7d illustrate schematically the effect of the tool's movement on operation of the no-fly zone rule/function. In Figs. 7c and 7d, the tool 310 is close enough to the no-fly zone 460 (distance 330 is small enough) that it is possible for the tool to enter the no-fly zone during the next time step. Fig. 7c illustrates the effect of a movement 340 that would increase the distance between tool 310 and no-fly zone 460. Since the movement 340 is away from no-fly zone 460, no collision is possible in this time step and no movement of the tool 310 is commanded.

In Fig. 7d, tool 310 is the same distance from no-fly zone 460 as in Fig. 7c. However, in Fig. 7d, the movement 340 of the tool is toward no-fly zone 460, making it possible for tool 310 to enter no- fly zone 460. In the embodiment illustrated, a movement 350 is commanded to move the tool 310 away from the no-fly zone 460. In other embodiments, the system prevents movement 340, but does not command movement 350; in such embodiments, the tool 310 will remain close to the no- fly zone 460. In yet other embodiments, the system warns/signals the operator that the move is RESTRICTED, but does not restrict movement 340 or command movement 350 away from the no- fly zone rule/function 460. Such a warning/signaling can be visual or aural, using any of the methods known in the art.

Example 4 - Tracking system with preferred volume zone rule/function

In reference to Fig. 8a-d, which shows, in a non-limiting manner, an embodiment of a tracking system with a preferred volume zone function/rule.

The system tracks a tool 310 with respect to a preferred volume zone (570), in order to determine whether the tool will leave the preferred volume (570) within the next time step.

In this example, the preferred volume zone 570 extends over the right lobe of the liver. Figs. 8a and 8b show how the behavior of the system depends on the location of the tool tip with respect to the preferred volume zone 570, while Figs. 8c and 8d show how movement of the tool affects the behavior (i.e., the preferred volume zone rule/function).

In Fig. 8a, the tool 310 is inside the preferred volume zone 570 and no movement of the tool is commanded. In Fig. 8b, the tool 310 is outside the preferred volume zone 570.

In the embodiment illustrated, a movement 340 is commanded to move the tool 310 away from the preferred volume zone 570. In other embodiments, the system prevents movement 340; in such embodiments, the tool 310 will remain close to the preferred volume zone 570. In yet other embodiments, the system warns/signals the operator that the move 340 is RESTRICTED. Such a warning/signaling can be visual or aural, using any of the methods known in the art.

Figs. 8c and 8d illustrate schematically the effect of the tool's movement on operation of the preferred volume rule/function. In Figs. 8c and 8d, the tool 310 is close enough to the edge of preferred volume zone 570 that it is possible for the tool to leave the preferred volume zone during the next time step. Fig. 8c illustrates the effect of a movement 350 that would take the tool 310 deeper into preferred volume zone 570. Since the movement 350 is into preferred volume 570, said movement is an allowed movement.

In Fig. 8d, the movement 350 of the tool is out of the preferred volume 570, making it possible for tool 310 to leave preferred volume 570.

According to one embodiment illustrated, a movement 340 is commanded to move the tool 310 into the preferred volume zone 570. In other embodiments, the system prevents movement 350, but does not command movement 340; in such embodiments, the tool 310 will remain close to the preferred volume zone 570. In yet other embodiments, the system warns/signals the operator that the move is RESTRICTED, but does not restrict movement 350 or command movement 340 away from the preferred volume zone 570. Such a warning/signaling can be visual or aural, using any of the methods known in the art.

Example 5 - Organ/tool Detection Function

In reference to Fig. 9, which shows, in a non-limiting manner, an embodiment of an organ detection system (however, it should be noted that the same is provided for detection of tools, instead of organs).

For each organ, the 3D spatial positions of the organs stored in a database. In Fig. 9, the perimeter of each organ is marked, to indicate the edge of the volume of 3D spatial locations stored in the database.

In Fig. 9, the liver 610 is labeled with a dashed line. The stomach 620 is labeled with a long- dashed line, the intestine 630 with a solid line and the gall bladder 640 is labeled with a dotted line.

In some embodiments, a label or tag visible to the operator is also presented. Any method of displaying identifying markers known in the art can be used. For non-limiting example, in an enhanced display, colored or patterned markers can indicate the locations of the organs, with the marker either indicating the perimeter of the organ or the area of the display in which it appears.

Example 6 - Tool Detection Function

In reference to Fig. 10, which shows, in a non-limiting manner, an embodiment of a tool detection function. For each tool, the 3Ό spatial positions of the tools stored in a database. In Fig. 10, the perimeter of each tool is marked, to indicate the edge of the volume of 3D spatial locations stored in the database. In Fig. 10, the left tool is labeled with a dashed line while the right tool is labeled with a dotted line.

In some embodiments, a label or tag visible to the operator is also presented. Any method of displaying identifying markers known in the art can be used. For non-limiting example, in an enhanced display, colored or patterned markers can indicate the locations of the tools, with the marker either indicating the perimeter of the tool or the area of the display in which it appears.

Example 7 - Movement Detection Function/rule

In reference to Fig. lla-b, which shows, in a non-limiting manner, an embodiment of a movement detection function/rule. Fig. 11a schematically illustrates a liver 810, a left tool 820 and a right tool 830 at a time t. Fig. lib schematically illustrates the liver 810, left tool 820 and right tool 830 at a later time t + At, where At is a small time interval. In this example, the left tool 820 has moved downward (towards the direction of liver 810) in the time interval At.

The system has detected movement of left tool 820 and labels it. This is illustrated schematically in Fig. lib by a dashed line around left tool 820.

Example 8 - Prediction Function

In reference to Fig. 12a-d, which shows, in a non-limiting manner, an embodiment of the above discussed prediction function.

Fig. 12a shows a left tool 920 and a right tool 930 at a time t.

Fig. 12b shows the same tools at a later time t + At, where At is a small time interval. Left tool 920 is moving to the right and downward, while right tool 930 is moving to the left and upward. If the motion continues (shown by the dashed line in Fig. 12c), then by the end of the next time interval, in other words, at some time between time t + At and time t + 2At, the tools will collide, as shown by tool tips within the dotted circle 950 in Fig. 12c.

In this embodiment, the system automatically prevents predicted collisions and, in this example, the system applies a motion 940 to redirect left tool 920 so as to prevent the collision.

In other embodiments, the system warns/signals the operator that a collision is likely to occur, but does not alter the movement of any tool. Such a warning/signaling can be visual or aural, using any of the methods known in the art. In other embodiments, the prediction function can be enabled to, for non-limiting example, alter the field of view to follow the predicted movement of a tool or of an organ, to warn of (or prevent) predicted motion into a no-fly zone, to warn of (or prevent) predicted motion out of a preferred zone.

Example 9 - Right Tool Function/rule

In reference to Fig. 13, which shows, in a non-limiting manner, an embodiment of a right tool function. Fig. 13 schematically illustrates a liver 1010, a left tool 1020 and a right tool 1030. The right tool, illustrated schematically by the dashed line 1040, is labeled and its 3D spacial location is constantly and real-time stored in a database. Now, according to the right tool function/rule the endoscope constantly tracks the right tool.

It should be pointed out that the same rule/function applies for the left tool (the left tool function/rule).

Example 10 - Field of View Function/rule

In reference to Fig. 14a-b, which shows, in a non-limiting manner, an embodiment of a field of view function/rule.

Fig. 14a schematically illustrates a field of view of the abdomen at a time t. In the field of view are the liver 1110, stomach 1120, intestines 1130 and gall bladder 1140.

The gall bladder is nearly completely visible at the left of the field of view. Two tools are also in the field of view, with their tips in proximity with the liver. These are left tool 1150 and right tool 1160. In this example, the field of view function/rule tracks left tool 1150. In this example, left tool 1150 is moving to the right, as indicated by arrow 1170.

Fig. 14b shows the field of view at time t + At. The field of view has moved to the right so that the tip of left tool 1150 is still nearly at the center of the field of view. It can be seen that much less of gall bladder 1140 is visible, while more of right tool 1160 has entered the field of view.

The field of view function/rule can be set to follow a selected tool, as in this example or to keep a selected organ in the center of the field of view. It can also be set to keep a particular set of tools in the field of view, zooming in or out as necessary to prevent any of the chosen tools from being outside the field of view. Alternatively, the field of view function/rule defines n 3D spatial positions; n is an integer greater than or equal to 2; the combination of all of said n 3D spatial positions provides a predetermined field of view.

Each movement of the endoscope or the surgical tool within said n 3D spatial positions is an allowed movement and any movement of the endoscope or the surgical tool outside said n 3D spatial positions is a restricted movement.

Alternatively, said the field of view function/rule defines n 3D spatial positions; n is an integer greater than or equal to 2; the combination of all of said n 3D spatial positions provides a predetermined field of view.

According to the field of view function/rule, the endoscope is relocated if movement has been detected by said detection means, such that said field of view is maintained.

Example 11 - Tagged Tool Function/rule (or alternatively the preferred tool rule)

In reference to Fig. 15, which shows, in a non-limiting manner, an embodiment of a tagged tool function/rule.

Fig. 15 shows three tools (1220, 1230 and 1240) in proximity to the organ of interest, in this example, the liver 1210.

The tool most of interest to the surgeon, at this point during the operation, is tool 1240. Tool 1240 has been tagged (dotted line 1250); the 3D spacial location of tool 1240 is constantly stored in a database and this spacial location has been labeled as one of interest.

The system can use this tagging for many purposes, including, but not limited to, keeping tool 1240 in the center of the field of view, predicting its future motion, keeping it from colliding with other tools or keeping other tools from colliding with it, instructing the endoscope to constantly monitor and track said tagged tool 1250 and so on.

It should be noted that in the preferred tool rule, the system tags one of the tools and performs as in the tagged tool rule/function. Example 12 - Proximity Function/rule

In reference to Fig. 16a-c, which shows, in a non-limiting manner, an embodiment of a proximity function/rule.

Fig. 16a schematically illustrates two tools (1310 and 1320) separated by a distance 1330 which is greater than a predefined proximity distance. Since tool 1310 is not within proximity of tool 1320, the field of view (1380) does not move.

Fig. 16b schematically illustrates two tools (1310 and 1320) separated by a distance 1330 which is less than a predefined proximity distance.

Since tool 1310 is within proximity of tool 1320, the field of view 1380 moves upward, illustrated schematically by arrow 1340, until the tips of tool 1310 and tool 1320 are in the center of field of view 1380 (Fig. 16c).

Alternatively the once the distance 1330 between the two tool 1320 and 1310 is smaller than a predetermined distance, the system alerts the user of said proximity (which might lead to a collision between the two tools). Alternatively, the system moves one of the tools away from the other one.

Example 13 - Operator Input Function/rule

In reference to Fig. 17a-b, which shows, in a non-limiting manner, an embodiment of an operator input function/rule. According to this embodiment, input is received from the operator.

In the following example, the input received from the operator is which tool to track.

Fig. 17a schematically illustrates an endoscope with field of view 1480 showing a liver 1410 and two tools 1420 and 1430. Operator 1450 first selects the tip of the left tool as the region of interest, preferably by touching the tool tip on the screen, causing the system to tag (1440) the tip of the left tool.

As illustrated in Fig. 17b, the system then directs and modifies the spatial position of the endoscope so that the tagged tool tip 1440 is in the center of the field of view 1480.

Another example of the operator input function/rule is the following:

If a tool has been moved closely to an organ in the surgical environment, according to the proximity rule or the collision prevention rule, the system will, according to one embodiment, prevent the movement of the surgical tool. According to one embodiment of the present invention, once the surgical tool has been stopped, any movement of said tool in the direction is interpreted as input from the operator to continue the movement of said surgical tool in said direction.

Thus, according to this embodiment, the operator input function/rule receives input from the operator (i.e., physician) to continue the move of said surgical tool (even though it is "against" the collision prevention rule). Said input is simply in the form of the continued movement of the surgical tool (after the alert of the system or after the movement prevention by the system).

Example 14 - constant field of view rule/function

In reference to Figs. 18a-d, which shows, in a non- limiting manner, an embodiment of a tracking system with a constant field of view rule/function.

In many endoscopic systems, the tip lens in the camera optics is not at a right angle to the sides of the endoscope. Conventionally, the tip lens angle is described relative to the right angle, so that a tip lens at right angles to the sides of the endoscope is described as having an angle of 0. Typically, angled endoscope tip lenses have an angle of 30° or 45°. This tip lens angle affects the image seen during zooming. Fig. 18 illustrates, in an out-of-scale manner, for a conventional system, the effect of zooming in the field of view in an endoscope with tip lens set straight in the end (Fig. 18a and 18b) vs. the effect of zooming in the field of view in an endoscope with angled tip lens (Fig. 18c and 18d).

Figs. 18a and 18c illustrate the endoscope (100), the object it is viewing (200) and the image seen by the endoscope camera (130) before the zoom. The solid arrows (160) show the limits of the FOV and the dashed arrow (170), the center of the field of view (FOV); since the object is in the center of the FOV, an image of the object (210) is in the center of the camera image (130). Figs. 18b and 18d illustrate the endoscope (100), the object it is viewing (200) and the image seen by the endoscope camera (130) after the zoom. The solid arrows (160) show the limits of the FOV and the dashed arrow (170), the center of the field of view.

If the tip lens is set straight in the end of the endoscope (Figs.l8a and 18b), an object (200) in the center of the field of view will be in the center of the field of view (FOV) (and the camera image) (130) both before (Fig. 18a) and after (Fig. 18b) the zoom. However, if the tip lens is set at an angle in the end of the endoscope (Figs. 18c and 18d), then an object that is in the center of the FOV (and the camera image) before the zoom (Fig. 18c) will not be in the center of the FOV (or the camera image) after the zoom (Fig. 18d) since the direction of motion of the endoscope is not the direction in which the center of the field of view (170) points.

In an embodiment of the system of the present invention, unlike in conventional systems, the controlling means maintains the center of the field of view (FOV) during zoom independent of the tip lens angle. An advantage of controlling the zoom of the endoscope via a data processing system is that the tip lens angle does not need to be input to the data processing system, obviating a possible source of error.

According to one embodiment of the present invention, the endoscope's movement will be adjusted in order to maintain a constant field of view.

Example 15 - misalignment rule/function

According to another embodiment of the present invention, the system can inform the user of any misalignment of the same system.

Misalignment of the system may cause parasitic movement of the endoscope tip, where the endoscope tip does not move exactly in the expected direction. According to one embodiment of the system, the system comprises sensors (e.g., gyroscopes, accelerometers and any combination thereof) that calculate/estimates the position of the pivot point in real time in order to (a) inform the user of misalignment; or (b) calculate the misalignment so that the system can adjust its movement to prevent parasitic movement.

Example 16 - change of speed rule/function

In reference to Fig. 19, which shows, in a non-limiting manner, an embodiment of a tracking system with a change of speed rule/function.

In conventional endoscopic control systems, motion of the endoscope occurs at a single speed. This speed is fairly fast so that the endoscope can be moved rapidly between locations that are well separated. However, this means that making fine adjustments so difficult that fine adjustments are normally not made. In an embodiment of the present invention, the speed of the tip of the endoscope is automatically varied such that, the closer the endoscope tip is to an object, be it a tool, an obstacle, or the object of interest, the more slowly it moves. In this embodiment, as shown in Fig. 19, measurements are made of the distance X (150) from the tip (195) of the endoscope (100) to the pivot point of the endoscope (190), where said pivot point is at or near the surface of the skin (1100) of a patient (1000). Measurements are also made of the distance Y (250) from the tip of the endoscope (195) to the object in the center of the scene of view (200). From a predetermined velocity V_p, the actual velocity of the tip of the endoscope at a given time, V_act, is calculated from

Y

V act , oc— j V p

Therefore, the closer to the object at the center of the scene of view, the more slowly the endoscope moves, making it possible to use automatic control of even fine adjustments, and reducing the probability that the endoscope will come in contact with tissue or instruments. .

Example 17 - articulation of tool

In reference to Fig. 20a-b, a non-limiting example of movement of an articulating tool (310), here an endoscope, is shown schematically.

In Fig. 20a-b, the endoscope (310) is moved so that, instead of viewing the outer side of the liver (320) from the right, it views the inner side of the liver (320) from the left.

Fig. 20a shows the endoscope (310) at the beginning of the movement. It is fully extended and its tip (318) is positioned about halfway up the outer side of the liver. The dashed line shows the movement of the base (312) of the endoscope, which will move in a straight line from its starting position (Fig. 20a) to its final position (Fig. 20b). The dotted line shows the movement of the endoscope tip (318) - the tip (318) moves upward, over the top of the liver (320), and then down the inner side of the liver (320), to allow imaging of the left (inner) side of the liver from between the liver (320) and the lungs (1790).

In Fig. 20b, the movement has been completed. The endoscope tip (318) now points rightward; the articulating section (316) being curved so that the endoscope (310) views the right side of the liver (310), with the endoscope tip (318) being between the liver (320) and the lungs (1790) while its base (312) remains on the right side of the body.

Example 18 - articulation of tool

In reference to Fig. 21, a non- limiting example of flexion of an articulating tool (310), here an endoscope, is shown schematically.

In Fig. 21, portions of the small intestine (1795) are shown schematically. The endoscope enters the body from the body's right side (body not shown), and views a portion of the small intestine (1795F) from the left and below. The articulating section of the endoscope (316) bypasses a loop of small intestine (1795A), passes between two portions of small intestine (1795B, 1795C), and over other portions of small intestine (1795D, 1795E) so that the endoscope's tip (318) views the desired portion of the small intestine (1795F) from the desired direction.

In the foregoing description, embodiments of the invention, including preferred embodiments, have been presented for the purpose of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obvious modifications or variations are possible in light of the above teachings. The embodiments were chosen and described to provide the best illustration of the principals of the invention and its practical application, and to enable one of ordinary skill in the art to utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated. All such modifications and variations are within the scope of the invention as determined by the appended claims when interpreted in accordance with the breadth they are fairly, legally, and equitably entitled.

Claims

CLAIMS:

1. A surgical controlling system, comprising: a. at least one surgical tool adapted to be inserted into a surgical environment of a human body for assisting a surgical procedure, at least one said surgical tool being an articulating tool; b. at least one location estimating means adapted to real-time locate the 3D spatial position of said at least one surgical tool at any given time t; c. at least one movement detection means communicable with a movement's database and with said location estimating means; said movement's database is adapted to store said 3D spatial position of said at least one surgical tool at time t and at time to_,- where tf > to,' said movement detection means is adapted to detect movement of said at least one surgical tool if the 3D spatial position of said at least one surgical tool at time tf is different than said 3D spatial position of said at least one surgical tool at time to,' and, d. a controller having a processing means communicable with a controller's database, said controller adapted to control the spatial position of said at least one surgical tool; said controller's database is in communication with said movement detection means; said controller adapted to provide instructions for moving said at least one surgical tool; wherein said controller is adapted to change the articulation of said articulating tool during said direction of said surgical tool to said location via said instructions provided by said controller.

2. The surgical controlling system of claim 1 , additionally comprising at least one endoscope adapted to provide a real time image of said surgical environment.

3. The surgical controlling system of claim 2, wherein said endoscope is an articulating endoscope.

4. The surgical controlling system of claim 1 , wherein said tool is an articulating endoscope.

5. The surgical controlling system of claim 1 , wherein said tool comprises at least one proximity sensor positioned on the outer circumference of the same.

6. The surgical controlling system of claim 1 , additionally comprising a touchscreen.

7. The surgical controlling system of claim 6, wherein said location within said surgical environment of said human body is determinable from pressure on a portion of said touchscreen.

8. The surgical controlling system of claim 6, wherein said portion of said touchscreen is that which displays the image of said location.

9. The surgical controlling system of claim 6, wherein said portion of said touchscreen displays a direction indicator, said direction indicator selected from a group consisting of: an arrow pointing in a predefined direction, a line pointing in a predefined direction, a pointer pointing in a predefined direction, the word "left", the word "right" the word "up", the word "down", the word "forward", the word "back", the word "zoom", the word "in", the word "out", and any combination thereof.

10. The surgical controlling system of claim 1, wherein said instructions comprise a predetermined set of rules selected from a group consisting of: most used tool rule, right tool rule, left tool rule, field of view rule, no fly zone rule, a route rule, environmental rule, operator input rule, proximity rule; collision prevention rule, history-based rule, tool- dependent ALLOWED and RESTRICTED movements rule, preferred volume zone rule, preferred tool rule, movement detection rule, tagged tool rule, change of speed rule and any combination thereof.

11. The surgical controlling system of claim 7, wherein said route rule comprises a communicable database storing predefined route in which said at least one surgical tool is adapted to move within said surgical environment; said predefined route comprises n 3D spatial positions of said at least one surgical tool; n is an integer greater than or equal to 2; said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said n 3D spatial positions of said predefined route, and said RESTRICTED movements are movements in which said location of said at least one surgical tool is substantially different from said n 3D spatial positions of said predefined route.

12. The surgical controlling system of claim 7, wherein said environmental rule comprises a comprises a communicable database; said communicable database adapted to receive at least one real-time image of said surgical environment and is adapted to perform real-time image processing of the same and to determine the 3D spatial position of hazards or obstacles in said surgical environment; said environmental rule is adapted to determine said ALLOWED and RESTRICTED movements according to said hazards or obstacles in said surgical environment, such that said RESTRICTED movements are movements in which said at least one surgical tool is located substantially in at least one of said 3D spatial positions, and said ALLOWED movements are movements in which the location of said at least one surgical tool is substantially different from said 3D spatial positions.

13. The surgical controlling system of claim 9, wherein said hazards or obstacles in said surgical environment are selected from a group consisting of tissue, a surgical tool, an organ, an endoscope and any combination thereof.

14. The surgical controlling system of claim 7, wherein said operator input rule comprises a communicable database; said communicable database is adapted to receive an input from the operator of said system regarding said ALLOWED and RESTRICTED movements of said at least one surgical tool.

15. The surgical controlling system of claim 11, wherein said input comprises n 3D spatial positions; n is an integer greater than or equal to 2; wherein at least one of which is defined as ALLOWED location and at least one of which is defined as RESTRICTED location, such that said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said at least one surgical tool is substantially different from said n 3D spatial positions.

16. The surgical controlling system of claim 11, wherein said input comprises at least one rule according to which ALLOWED and RESTRICTED movements of said at least one surgical tool are determined, such that the spatial position of said at least one surgical tool is controlled by said controller according to said ALLOWED and RESTRICTED movements.

17. The surgical controlling system of claim 13, wherein said predetermined set of rules comprises at least one rule selected from a group consisting of: most used tool, right tool rule, left tool rule, field of view rule, no fly zone rule, route rule, environmental rule, operator input rule, proximity rule, collision prevention rule, preferred volume zone rule, preferred tool rule, movement detection rule, history-based rule, tool-dependent ALLOWED and RESTRICTED movements rule, and any combination thereof.

18. The surgical controlling system of claims 11-14, wherein said operator input rule converts an ALLOWED movement to a RESTRICTED movement and a RESTRICTED movement to an ALLOWED movement.

19. The surgical controlling system of claim 7, wherein said proximity rule is adapted to define a predetermined distance between at least two surgical tools; said ALLOWED movements are movements which are within the range or out of the range of said predetermined distance, and said RESTRICTED movements are movements which are out of the range or within the range of said predetermined distance.

20. The surgical controlling system of claim 7, wherein said proximity rule is adapted to define a predetermined angle between at least three surgical tools; said ALLOWED movements are movements which are within the range or out of the range of said predetermined angle, and said RESTRICTED movements which are out of the range or within the range of said predetermined angle.

21. The surgical controlling system of claim 7, wherein said collision prevention rule is adapted to define a predetermined distance between said at least one surgical tool and an anatomical element within said surgical environment; said ALLOWED movements are movements which are in a range that is larger than said predetermined distance, and said RESTRICTED movements are movements which is in a range that is smaller than said predetermined distance.

22. The surgical controlling system of claim 18, wherein said anatomical element is selected from a group consisting of tissue, organ, another surgical tool and any combination thereof.

23. The surgical controlling system of claim 7, wherein at least one of the following is being held true (a) said system additionally comprises an endoscope; said endoscope is adapted to provide real-time image of said surgical environment; (b) at least one of said surgical tools is an endoscope adapted to provide at least one real-time image of said surgical environment.

24. The surgical controlling system of claim 20, wherein said right tool rule is adapted to determine said ALLOWED movement of said endoscope according to the movement of the surgical tool positioned to right of said endoscope; further wherein said left tool rule is adapted to determine said ALLOWED movement of said endoscope according to the movement of the surgical tool positioned to left of said endoscope.

25. The surgical controlling system of claim 20, wherein said tagged tool rule comprises means adapted to tag at least one surgical tool within said surgical environment and to determine said ALLOWED movement of said endoscope so as to constantly track the movement of said tagged surgical tool.

26. The surgical controlling system of claim 20, wherein said field of view rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; the combination of all of said n 3D spatial positions provides a predetermined field of view; said field of view rule is adapted to determine said ALLOWED movement of said endoscope within said n 3D spatial positions so as to maintain a constant field of view, such that said ALLOWED movements are movements in which said endoscope is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said endoscope is substantially different from said n 3D spatial positions.

27. The surgical controlling system of claim 20, wherein said preferred volume zone rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; said n 3D spatial positions provides said preferred volume zone; said preferred volume zone rule is adapted to determine said ALLOWED movement of said endoscope within said n 3D spatial positions and RESTRICTED movement of said endoscope outside said n 3D spatial positions, such that said ALLOWED movements are movements in which said endoscope is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said endoscope is substantially different from said n 3D spatial positions.

28. The surgical controlling system of claim 20, wherein said preferred tool rule comprises a communicable database, said database stores a preferred tool; said preferred tool rule is adapted to determine said ALLOWED movement of said endoscope to constantly track the movement of said preferred tool.

29. The surgical controlling system of claim 20, wherein said no fly zone rule comprises a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; said n 3D spatial positions define a predetermined volume within said surgical environment; said no fly zone rule is adapted to determine said RESTRICTED movement if said movement is within said no fly zone and ALLOWED movement if said movement is outside said no fly zone, such that said RESTRICTED movements are movements in which said at least one of said surgical tool is located substantially in at least one of said n 3D spatial positions, and said ALLOWED movements are movements in which the location of said at least one endoscope is substantially different from said n 3D spatial positions.

30. The surgical controlling system of claim 20, wherein said most used tool rule comprises a communicable database counting the amount of movement of each said surgical tool; said most used tool rule is adapted to constantly position said endoscope to track the movement of the most moved surgical tool.

31. The surgical controlling system of either one of claims 7 or 20, wherein said system further comprises a maneuvering subsystem communicable with said controller, said maneuvering subsystem is adapted to spatially reposition said at least one surgical tool during a surgery according to said predetermined set of rules; further wherein said system is adapted to alert the physician of said RESTRICTED movement of said at least one surgical tool.

32. The surgical controlling system of claim 28, wherein said alert is selected from a group consisting of audio signaling, voice signaling, light signaling, flashing signaling and any combination thereof.

33. The surgical controlling system of either one of claims 7 or 20, wherein said ALLOWED movement is permitted by said controller and said RESTRICTED movement is denied by said controller.

34. The surgical controlling system of claim 7, wherein said history-based rule comprises a communicable database storing each 3D spatial position of each said surgical tool, such that each movement of each surgical tool is stored; said history-based rule is adapted to determine said ALLOWED and RESTRICTED movements according to historical movements of said at least one surgical tool, such that said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said 3Ό spatial positions, and said RESTRICTED movements are movements in which the location of said at least one surgical tool is substantially different from said n 3D spatial positions.

35. The surgical controlling system of claim 20, wherein said tool-dependent ALLOWED and RESTRICTED movements rule comprises a communicable database; said communicable database is adapted to store predetermined characteristics of at least one of said surgical tool; said tool-dependent ALLOWED and RESTRICTED movements rule is adapted to determine said ALLOWED and RESTRICTED movements according to said predetermined characteristics of said surgical tool; such that ALLOWED movements are movements of said endoscope which track said surgical tool having said predetermined characteristics.

36. The surgical controlling system of claim 32, wherein said predetermined characteristics of said surgical tool are selected from a group consisting of: physical dimensions, structure, weight, sharpness, and any combination thereof.

37. The surgical tracking system of claim 20, wherein said movement detection rule comprises a communicable database comprising the real-time 3D spatial positions of each said surgical tool; said movement detection rule is adapted to detect movement of said at least one surgical tool when a change in said 3D spatial positions is received, such that said ALLOWED movements are movements in which said endoscope is re-directed to focus on said moving surgical tool.

38. The surgical controlling system of claim 7, further comprising a maneuvering subsystem communicable with said controller, said maneuvering subsystem is adapted to spatially reposition said at least one surgical tool during a surgery according to said predetermined set of rules, such that if said movement of said at least one surgical tool is a RESTRICTED movement, said maneuvering subsystem prevents said movement.

39. The surgical controlling system of claim 1, wherein said at least one location estimating means comprises at least one endoscope adapted to acquire real-time images of said surgical environment within said human body; and at least one surgical instrument spatial location software adapted to receive said real-time images of said surgical environment and to estimate said 3D spatial position of said at least one surgical tool.

40. The surgical tracking system of claim 1, wherein said at least one location estimating means comprises (a) at least one element selected from a group consisting of optical imaging means, radio frequency transmitting and receiving means, at least one mark on said at least one surgical tool and any combination thereof; and, (b) at least one surgical instrument spatial location software adapted to estimate said 3D spatial position of said at least one surgical tool by means of said element.

41. The surgical tracking system of claim 1, wherein said at least one location estimating means is an interface subsystem between a surgeon and said at least one surgical tool, the interface subsystem comprising: a. at least one array comprising N regular or pattern light sources, where N is a positive integer; b. at least one array comprising M cameras, each of the M cameras, where M is a positive integer; c. optional optical markers and means for attaching the optical marker to the at least one surgical tool; and; d. a computerized algorithm operable via the controller, the computerized algorithm adapted to record images received by each camera of each of the M cameras and to calculate therefrom the position of each of the tools, and further adapted to provide automatically the results of the calculation to the human operator of the interface.

A method of using a surgical controlling system, comprising steps of: a. providing a surgical controlling system comprising: i. at least one surgical tool adapted to be inserted into a surgical environment of a human body for assisting a surgical procedure; ii. at least one location estimating means adapted to real-time locate the 3D spatial position of said at least one surgical tool at any given time t; iii. at least one movement detection means communicable with a movement's database and with said location estimating means; said movement's database is adapted to store said 3D spatial position of said at least one surgical tool at time tf and at time to; where tf > to; said movement detection means is adapted to detect movement of said at least one surgical tool if the 3D spatial position of said at least one surgical tool at time tf is different than said 3D spatial position of said at least one surgical tool at time to and, iv. a controller having a processing means communicable with a controller's database, said controller adapted to control the spatial position of said at least one surgical tool; said controller's database is in communication with said movement detection means; and v. at least one touchscreen adapted to display an image of at least a portion of said surgical environment of said human body and to receive input of at least one location within said surgical environment of said human body; b. inserting at least one said surgical tool into said surgical environment; c. displaying said image of at least a portion of said surgical environment via said touchscreen; d. determining said location within said surgical environment of said human body from pressure on a portion of said touchscreen; e. estimating the 3D spatial position of at least one said surgical tool; and f. directing and moving said surgical tool to said location via instructions provided by said controller.

43. The method of claim 39, additionally comprising steps of providing a real time image of said surgical environment using at least one endoscope.

44. The method of claim 39, additionally comprising steps of selecting said tool to be an endoscope.

45. The method of claim 39, additionally comprising steps of positioning at least one proximity sensor on the outer circumference of said tool.

46. The method of claim 39, additionally comprising steps of selecting said portion of said touchscreen to be that which displays the image of said location.

47. The method of claim 39, additionally comprising steps of displaying a direction indicator on said portion of said touchscreen, said direction indicator selected from a group consisting of: an arrow pointing in a predefined direction, a line pointing in a predefined direction, a pointer pointing in a predefined direction, the word "left", the word "right" the word "up", the word "down", the word "forward", the word "back", the word "zoom", the word "in", the word "out", and any combination thereof.

48. The method of claim 39, additionally comprising steps of selecting said instructions from a predetermined set of rules selected from a group consisting of: most used tool rule, right tool rule, left tool rule, field of view rule, no fly zone rule, a route rule, environmental rule, operator input rule, proximity rule; collision prevention rule, history-based rule, tool- dependent ALLOWED and RESTRICTED movements rule, preferred volume zone rule, preferred tool rule, movement detection rule, tagged tool rule, change of speed rule and any combination thereof.

49. The method of claim 45, wherein said route rule comprises steps of: providing a communicable database; storing a predefined route in which said at least one surgical tool is adapted to move within said surgical environment; comprising said predefined route of n 3D spatial positions of said at least one surgical tool, n is an integer greater than or equal to 2; said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said n 3D spatial positions of said predefined route, and said RESTRICTED movements are movements in which said location of said at least one surgical tool is substantially different from said n 3D spatial positions of said predefined route.

50. The method of claim 45, wherein said environmental rule comprises steps of: providing a communicable database; receiving at least one real-time image of said surgical environment in said communicable database; performing real-time image processing of the same and determining the 3D spatial position of hazards or obstacles in said surgical environment; determining said ALLOWED and RESTRICTED movements according to said hazards or obstacles in said surgical environment, such that said RESTRICTED movements are movements in which said at least one surgical tool is located substantially in at least one of said 3D spatial positions, and said ALLOWED movements are movements in which the location of said at least one surgical tool is substantially different from said 3D spatial positions.

51. The method of claim 47, additionally comprising steps of selecting said hazards or obstacles in said surgical environment from a group consisting of tissue, a surgical tool, an organ, an endoscope and any combination thereof.

52. The method of claim 45, wherein said operator input rule comprises steps of: providing a communicable database; and receiving input from an operator of said system regarding said ALLOWED and RESTRICTED movements of said at least one surgical tool.

53. The method of claim 49, additionally comprising steps of: comprising said input of n 3D spatial positions, n is an integer greater than or equal to 2; defining at least one of said spatial positions as an ALLOWED location; defining at least one of said spatial positions as a RESTRICTED location; such that said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said at least one surgical tool is substantially different from said n 3D spatial positions.

54. The method of claim 49, additionally comprising steps of: comprising said input of at least one rule according to which ALLOWED and RESTRICTED movements of said at least one surgical tool are determined, such that the spatial position of said at least one surgical tool is controlled by said controller according to said ALLOWED and RESTRICTED movements.

55. The method of claim 51, additionally comprising steps of selecting said predetermined set of rules from a group consisting of: most used tool, right tool rule, left tool rule, field of view rule, no fly zone rule, route rule, environmental rule, operator input rule, proximity rule, collision prevention rule, preferred volume zone rule, preferred tool rule, movement detection rule, history-based rule, tool-dependent ALLOWED and RESTRICTED movements rule, and any combination thereof.

56. The method of claims 49-52, wherein said operator input rule comprises steps of: converting an ALLOWED movement to a RESTRICTED movement and converting a RESTRICTED movement to an ALLOWED movement.

57. The method of claim 45, wherein said proximity rule comprises steps of: defining a predetermined distance between at least two surgical tools; said ALLOWED movements are movements which are within the range or out of the range of said predetermined distance, and said RESTRICTED movements are movements which are out of the range or within the range of said predetermined distance.

58. The method of claim 45, wherein said proximity rule comprises steps of: defining a predetermined angle between at least three surgical tools; said ALLOWED movements are movements which are within the range or out of the range of said predetermined angle, and said RESTRICTED movements are movements which are out of the range or within the range of said predetermined angle.

59. The method of claim 45, wherein said collision prevention rule comprises steps of: defining a predetermined distance between said at least one surgical tool and an anatomical element within said surgical environment; said ALLOWED movements are movements which are in a range that is larger than said predetermined distance, and said RESTRICTED movements are movements which is in a range that is smaller than said predetermined distance.

60. The method of claim 56, additionally comprising steps of selecting said anatomical element from a group consisting of tissue, organ, another surgical tool and any combination thereof.

61. The method of claim 45, wherein at least one of the following is being held true (a) additionally providing an endoscope for said system; and provide at least one real-time image of said surgical environment by means of said endoscope; (b) selecting at least one of said surgical tools to be an endoscope and providing at least one real-time image of said surgical environment by means of said endoscope.

62. The method of claim 58, wherein said right tool rule comprises steps of: determining said ALLOWED movement of said endoscope according to the movement of the surgical tool positioned to right of said endoscope; further wherein said left tool rule comprises steps of: determining said ALLOWED movement of said endoscope according to the movement of the surgical tool positioned to left of said endoscope.

63. The method of claim 58, wherein said tagged tool rule comprises steps of: tagging at least one surgical tool within said surgical environment and determining said ALLOWED movements of said endoscope to be movements that constantly track the movement of said tagged surgical tool.

64. The method of claim 58, wherein said field of view rule comprises steps of: providing a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; generating a field of view from the combination of all of said n 3D spatial positions; maintaining a constant field of view by determining said ALLOWED movement of said endoscope to be within said n 3D spatial positions, such that said ALLOWED movements are movements in which said endoscope is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said endoscope is substantially different from said n 3D spatial positions.

65. The method of claim 58, wherein said preferred volume zone rule comprises steps of: providing a communicable database comprising n 3D spatial positions; n is an integer greater than or equal to 2; generating said preferred volume zone from said n 3D spatial positions; determining said ALLOWED movement of said endoscope to be within said n 3D spatial positions and said RESTRICTED movement of said endoscope to be outside said n 3D spatial positions, such that said ALLOWED movements are movements in which said endoscope is located substantially in at least one of said n 3D spatial positions, and said RESTRICTED movements are movements in which the location of said endoscope is substantially different from said n 3D spatial positions.

66. The method of claim 58, wherein said preferred tool rule comprises steps of: providing a communicable database, storing a preferred tool in said database; determining said ALLOWED movement of said endoscope so as to constantly track the movement of said preferred tool.

67. The method of claim 58, wherein said no fly zone rule comprises steps of: providing a communicable database comprising n 3D spatial positions, n is an integer greater than or equal to 2; defining a predetermined volume within said surgical environment from said n 3D spatial positions; determining said RESTRICTED movement to be said movement within said no fly zone; determining said ALLOWED movement to be said movement outside said no fly zone, such that said RESTRICTED movements are movements in which said at least one of said surgical tool is located substantially in at least one of said n 3D spatial positions, and said ALLOWED movements are movements in which the location of said at least one endoscope is substantially different from said n 3D spatial positions.

68. The method of claim 58, wherein said most used tool rule comprises steps of: providing a communicable database; counting the amount of movement of each said surgical tool; constantly positioning said endoscope to track movement of the most moved surgical tool.

69. The method of either one of claims 45 or 58, additionally comprising steps of providing a maneuvering subsystem communicable with said controller, spatially repositioning said at least one surgical tool during a surgery according to said predetermined set of rules; and alerting the physician of said RESTRICTED movement of said at least one surgical tool.

70. The method of claim 66, additionally comprising steps of selecting said alert from a group consisting of: audio signaling, voice signaling, light signaling, flashing signaling and any combination thereof.

71. The method of either one of claims 45 or 58, additionally comprising steps of defining said ALLOWED movement as a movement permitted by said controller and defining said RESTRICTED movement as a movement denied by said controller.

72. The method of claim 45, wherein said history-based rule comprises steps of: providing a communicable database storing each 3D spatial position of each said surgical tool, such that each movement of each surgical tool is stored; determining said ALLOWED and RESTRICTED movements according to historical movements of said at least one surgical tool, such that said ALLOWED movements are movements in which said at least one surgical tool is located substantially in at least one of said 3Ό spatial positions, and said RESTRICTED movements are movements in which the location of said at least one surgical tool is substantially different from said n 3D spatial positions.

73. The method of claim 58, wherein said tool-dependent ALLOWED and RESTRICTED movements rule comprises steps of: providing a communicable database; storing predetermined characteristics of at least one said surgical tool; determining said ALLOWED and RESTRICTED movements according to said predetermined characteristics of said surgical tool; such that ALLOWED movements are movements of said endoscope which track said surgical tool having said predetermined characteristics.

74. The method of claim 70, additionally comprising steps of selecting said predetermined characteristics of said surgical tool from a group consisting of: physical dimensions, structure, weight, sharpness, and any combination thereof.

75. The surgical tracking system of claim 58, wherein said movement detection rule comprises steps of: providing a communicable database comprising the real-time 3D spatial positions of each said surgical tool; detecting movement of said at least one surgical tool when a change in said 3D spatial positions is received, such that said ALLOWED movements are movements in which said endoscope is re-directed to focus on said moving surgical tool.

76. The method of claim 45, additionally comprising steps of providing a maneuvering subsystem communicable with said controller, spatially repositioning said at least one surgical tool during a surgery according to said predetermined set of rules, such that if said movement of said at least one surgical tool is a RESTRICTED movement, said maneuvering subsystem prevents said movement.

77. The method of claim 39, additionally comprising steps of comprising said at least one location estimating means of at least one endoscope adapted to acquire real-time images of said surgical environment within said human body; providing at least one surgical instrument spatial location software; receiving said real-time images of said surgical environment from said endoscope and estimating said 3D spatial position of said at least one surgical tool using said spatial location software.

78. The method of claim 39, additionally comprising steps of providing said at least one location estimating means comprising (a) at least one element selected from a group consisting of optical imaging means, radio frequency transmitting and receiving means, at least one mark on said at least one surgical tool and any combination thereof; and, (b) at least one surgical instrument spatial location software adapted to estimate said 3D spatial position of said at least one surgical tool by means of said element.

79. The method of claim 39, additionally comprising steps of selecting said at least one location estimating means to be an interface subsystem between a surgeon and said at least one surgical tool, the interface subsystem comprising: a. at least one array comprising N regular or pattern light sources, where N is a positive integer; b. at least one array comprising M cameras, each of the M cameras, where M is a positive integer; c. optional optical markers and means for attaching the optical marker to the at least one surgical tool; and; d. a computerized algorithm operable via the controller, the computerized algorithm adapted to record images received by each camera of each of the M cameras and to calculate therefrom the position of each of the tools, and further adapted to provide automatically the results of the calculation to the human operator of the interface.

80. The surgical controlling system of claim 1, wherein said articulating tool has articulations substantially at the tip of said tool, substantially along the body of said too, and any combination thereof.

81. The surgical controlling system of claim 1, wherein control of articulation is selected from a group consisting of hardware control, software control and any combination thereof.

82. The surgical controlling system of claim 1, wherein said tool has articulation in a regions selected from a group consisting of near the tip of said tool, on the body of said tool, and any combination thereof.

83. The method of claim 42, additionally comprising steps of providing said tool with articulations substantially at the tip of said tool, substantially along the body of said too, and any combination thereof.

84. The method of claim 42, additionally comprising steps of controlling articulation by means of a method selected from a group consisting of hardware control, software control and any combination thereof.

85. The method of claim 42, additionally comprising steps of providing a toll articulated at a region selected from a group consisting of near the tip of said tool, on the body of said tool, and any combination thereof.