KR20110047929A - Surgical robot system and motion restriction control method thereof - Google Patents

Abstract

PURPOSE: A surgical robot system and a method for limiting the operation of the same are provided to secure the safety of a patient in an operation by preventing the mis-manipulation of robot arm and/or instruments. CONSTITUTION: A limiting section setting part(350) receives limiting section setting information with respect to the limiting section of the manipulation with respect to a target, and the coordinate information with respect to the limiting section is generated. An arm manipulating part(360) receives manipulating information for manipulating the target. A manipulation verifying part(370) verifies the contact of the coordinate information and the external surface of the target referring to the displacement information of the target.

Description

Surgical robot system and motion restriction control method

The present invention relates to surgery, and more particularly to a surgical robot system and a method of limiting its operation.

A surgical robot refers to a robot having a function that can replace a surgical operation performed by a doctor. Such a surgical robot has the advantage of being capable of accurate and precise operation and remote surgery compared to humans.

Surgical robots currently being developed worldwide include bone surgery robots, laparoscopic surgery robots, and stereotactic surgery robots. The laparoscopic surgical robot is a robot that performs minimally invasive surgery using a laparoscope and a small surgical tool.

Laparoscopic surgery is an advanced surgical technique that involves surgery after inserting a laparoscope, which is an endoscope for looking into the belly with a hole about 1 cm in the navel area, and is expected to be developed in the future.

Recent laparoscopic cameras are equipped with a computer chip to provide a clearer and enlarged image than can be seen with the naked eye, and by using a specially designed laparoscopic surgical instruments while viewing the screen through the monitor has been developed enough to be able to perform any operation.

Moreover, laparoscopic surgery has the same range of surgery as laparotomy, but has fewer complications than laparotomy, and can start treatment much faster after the procedure, and it has the advantage of maintaining the stamina or immune function of the patient. have. As a result, laparoscopic surgery is increasingly recognized as a standard surgery for treating colorectal cancer in the US and Europe.

The surgical robot system generally consists of a master robot and a slave robot. When the operator manipulates a manipulator (for example, a handle) provided in the master robot, a surgical tool coupled to the robot arm of the slave robot or held by the robot arm is operated to perform surgery.

That is, the surgical robot system is equipped with a plurality of surgical instruments, each instrument performs the operation necessary for the operation in accordance with the operator's operation on the surgical robot system. Instruments operated by the operator during the operation are controlled by the operator, but are not operated by the operator, especially those that are not visible on the user's operation screen (i.e., located in an out-of-visible area). However, there is a possibility that unnecessary operations may be performed by operations other than the intention of the operator.

As such, by operating the instrument in an unintended manner, the operator may collide with the robot body or an adjacent instrument, and the possibility of causing damage or injury to blood vessels or tissues of the patient under surgery cannot be excluded.

It is an object of the present invention to provide a surgical robot system and an operation limiting method thereof in which an instrument is controlled only in a manner in which the operator intends to perform normal surgery.

It is another object of the present invention to provide a surgical robot system and a method of limiting its operation, which can promote safety of a patient under surgery by fundamentally preventing misoperation of a robot arm and / or an instrument.

Technical problems other than the present invention will be easily understood through the following description.

According to an aspect of the present invention, as a surgical robot, a restricted area setting unit for receiving the restricted area setting information for the area to which the operation of the controlled object is restricted and generating and storing the restricted area coordinate information, An arm operation unit for receiving operation information for the operation and an operation determination unit for determining whether the outer surface of the control object is in contact with the coordinate information of the restricted area by referring to the displacement information of the control object according to the operation information, wherein Surgical robot is provided, characterized in that the controlled object is at least one of a robot arm, an instrument and an endoscope.

When the outer surface of the object to be controlled is in contact with the coordinate information of the restricted area, the controller may further control to control the object to be controlled, and control to output the response information accordingly. Here, the reaction information may be one or more of tactile information, visual information, and auditory information.

The controller may determine whether to control the control object when the outer surface of the control object is in contact with the coordinate information of the restricted area, and then control the control object to be restricted only when the control object is determined to be restricted. Can be.

In addition, when the input unit for receiving the command to ignore the restricted area setting is further included, the controller may process to allow the manipulation of the object to be controlled when the restricted area setting ignore command is input.

The restricted area coordinate information may be coordinate range information corresponding to a closed curve input using a touch-sensitive screen on which a video image is displayed.

The image image may be one or more of an image image obtained by an endoscope, a CT image image, an MRI image image, and a human modeling image image.

The restricted area coordinate information may be coordinate range information of an area that is not displayed in the video image obtained by the endoscope.

The endoscope may be one or more of laparoscopic, thoracoscopic, arthroscopic, parenteral, bladder, rectal, duodenum, mediastinal, cardiac.

The restricted area coordinate information may be coordinate range information of a figure formed by positions designated as restricted area setting information among positions at which the object to be controlled is manipulated by the operation information of the arm operating unit.

The object to be controlled is manipulated in three-dimensional space, and the designated positions may be points designated to form an outline or an outer surface of a figure in three-dimensional space, respectively.

The surgical robot may further include an arm manipulation setting unit configured to generate and store manipulation setting information on whether to allow manipulation of each control object by manipulation of the arm manipulation unit. Here, the manipulation determination unit may further determine whether the manipulation information for any manipulation object is valid manipulation information.

According to another aspect of the present invention, a method for limiting the operation of a surgical robot, the method comprising: receiving and receiving restricted zone setting information for a zone in which manipulation of a controlled object is restricted, generating and storing restricted zone coordinate information, and a controlled object; Receiving operation information for the operation of the controller; determining whether the outer surface of the control object is in contact with the coordinate information of the restricted area by referring to the displacement information of the control object according to the operation information; And controlling the controlled object to be restricted in operation when the outer surface contacts the limited area coordinate information, wherein the controlled object is one or more of a robot arm, an instrument, and an endoscope. Is provided.

The controlling may include determining whether to control the control object when the external surface of the control object is in contact with the restricted area coordinate information, and when the control object is determined to be restricted to control. The control may include a control step.

In the determining step, it may be determined whether a command to ignore the restricted area setting is input or not to control the controlled object.

The controlling may include controlling the response information to be output when the outer surface of the object to be controlled is in contact with the coordinate information of the restricted area.

The operation limiting method of the surgical robot includes generating and storing operation setting information on whether to allow operation of each control target object by operation of the arm operating unit, and whether the operation information for any operation target object is valid operation information. The method may further include determining whether or not.

If the manipulation information is invalid manipulation information, the method may further include controlling the reaction information to be output.

According to another aspect of the invention, as a surgical robot, a display unit for displaying an image image obtained and provided by the endoscope, an arm operation unit for receiving the operation information for the operation of the controlled object, the control object is moved It includes a control unit for controlling to display the area image corresponding to the area including the plurality of points of the point input command inputted in the outline to the image image is overlaid on the image image, the object to be controlled includes: robot arm, instrument and endoscope There is provided a surgical robot, characterized in that at least one.

By inputting a point designation command and a segmentation command for two or more positions defining a boundary line connected to segment the region image, the region image can be divided into two or more separate regions.

By inputting a point designation command and a folding command to two or more locations that define a boundary line that is connected to segment the area image, the area image is rotated and overlapped with respect to the other one or more individual areas by the input of a point designation command and a folding command. Can be displayed.

The region image may be displayed by rotating and shifting the overlapping individual regions returned to their original positions by input of the restoration command.

One or more individual areas may be erased by inputting a point designation command and a delete command for two or more locations that define a boundary line connected to divide the area image.

The outline and the outer surface of the area image are recognized as a series of points, and by the point designation and movement command for any one point, the area image can be transformed.

The coordinate range extracted to correspond to the area image may be set as a restricted area or allowed area.

If the coordinate range extracted to correspond to the area image is set as the restricted area, an operation determination unit for determining whether the outer surface of the control object is in contact with the coordinate range by referring to the displacement information of the control object according to the operation information is further included. Can be.

When the external surface of the object to be controlled is in contact with the coordinate range, the controller may control the object to be controlled to be restricted, and may control the response information to be output accordingly.

The arm operation setting unit may further include an arm operation setting unit for generating and storing operation setting information on whether to allow operation of each control target object by an operation of the arm operation unit, and the operation determination unit may determine whether the operation information for any control target object is valid operation information. It can be further judged.

According to still another aspect of the present invention, there is provided a method for setting a region of a surgical robot, the method comprising: displaying an image image obtained and provided by an endoscope; receiving operation information for manipulation of a controlled object; And controlling a region image corresponding to a region including a plurality of positions where a point designation command is input among the positions to which the object is moved is overlaid on the video image to be displayed, wherein the object to be controlled includes a robot arm and an instrument. And at least one of endoscopes, and a coordinate range extracted to correspond to an area image is set to a restricted area or an allowable area.

According to another aspect of the present invention, a surgical robot, a display unit for displaying an image image obtained and provided by the endoscope, a storage unit for storing the human body modeling information and the corresponding modeling data, and obtained one or more organs And a control unit for extracting modeling data corresponding to organ selection information for and controlling an image of an area corresponding to the modeling data to be overlaid on the image image, wherein the modeling data includes at least one of color, shape, and size of the organ. A surgical robot is provided that includes information.

Human body modeling information and modeling data may be generated using a reference image that is one or more of a CT image and an MRI image.

The organ selection information may be selectively designated as an organ corresponding to one or more of the shape and color of the organ recognized by the image recognition of the image image among the modeling data.

When the arm operation unit further receives operation information for manipulation of the controlled object including one or more of the robot arm, the instrument, and the endoscope, when one or more of the position change or shape change of the organ is generated by the operation information, The region image may be deformed to correspond to the outline of the organ.

According to still another aspect of the present invention, there is provided a method for setting a region of a surgical robot, the method comprising: storing human modeling information and corresponding modeling data, displaying an image image obtained and provided by an endoscope, and Extracting modeling data corresponding to organ selection information for one or more organs, and controlling the area image corresponding to the modeling data to be overlaid and displayed on the image image, wherein the modeling data includes the color, shape, and size of the organs. There is provided a method for setting a region of a surgical robot comprising one or more information.

Other aspects, features, and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to an embodiment of the present invention, there is an effect that the instrument is controlled only in the manner in which the operator intends to perform normal surgery.

In addition, there is an effect to prevent the operation of the robot arm and / or instrument fundamentally to ensure the safety of the patient under surgery.

The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated and described in the drawings. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. In the following description of the present invention, if it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

In addition, the terms "... unit", "... group", "module" and the like that can be described in the specification means a unit for processing at least one function or operation, which means hardware or software or hardware and It can be implemented in a combination of software.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and in the following description with reference to the accompanying drawings, the same or corresponding components are given the same reference numerals and redundant description thereof will be omitted. Shall be.

In addition, in describing various embodiments of the present invention, each embodiment should not be interpreted or implemented independently, and the feature elements and / or technical ideas described in each embodiment may be combined with other embodiments separately described. It is to be understood that it may be interpreted or practiced.

1 is a plan view showing the overall structure of a surgical robot according to an embodiment of the present invention, Figure 2 is a conceptual diagram showing a master interface of a surgical robot according to an embodiment of the present invention.

1 and 2, a laparoscopic surgical robot system includes a slave robot 2 that actually performs surgery on a patient lying on an operating table, and a master robot 1 that remotely controls the slave robot 2. Include. 1 illustrates a case in which the master robot 1 and the slave robot 2 are separately implemented, but the master robot 1 and the slave robot 2 are not necessarily separated into physically independent devices. It can be integrated into one and configured in one piece, in which case the master interface 4 can correspond to, for example, the interface part of the one-piece robot.

The master interface 4 of the master robot 1 comprises a monitor 6 and a master manipulator, and the slave robot 2 comprises a robot arm 3 and a laparoscope 5. The illustrated laparoscopic 5 is one embodiment of a surgical endoscope, and may be replaced with one or more of thoracoscopic, arthroscopic, parenteral, bladder, rectal, duodenum, mediastinal, cardiac, etc.

The master manipulator of the master interface 4 is embodied so that the operator can grip and manipulate each hand. As illustrated in FIGS. 1 and 2, the master controller may be implemented with two handles 10 or more handles 10, and an operation signal according to the operator's manipulation of the handle 10 may be controlled by the slave robot 2. Robot arm 3 and / or instruments (not shown) are controlled. By operation of the operator's handle 10, for example, a surgical operation such as position movement, rotation, and cutting of the robot arm 3 may be performed. The master controller is not limited to the shape of the handle 10 and may be applied without any limitation as long as it can control the operation of the robot arm 3 through a network.

For example, the handle 10 may be configured to include a main handle and a sub handle. The operator may operate the slave robot arm 3 or the laparoscope 5 or the like only by the main handle, or may operate the sub handles to simultaneously operate a plurality of surgical equipments in real time. The main handle and the sub handle may have various mechanical configurations depending on the operation method thereof. For example, the robot arm 3 and / or other surgery of the slave robot 2, such as a joystick type, a keypad, a trackball, and a touch screen, may be used. Various input means for operating the equipment can be used.

An image input by the laparoscope 5 may be displayed as an image image on the monitor unit 6 of the master interface 4. Of course, the manner in which the image input by the laparoscope 5 is displayed may be various in addition to the manner in which the image is displayed through the monitor 6, which is omitted from the description because it is somewhat distance from the gist of the present invention.

The monitor unit 6 may be composed of one or more monitors, and may display information necessary for surgery on each monitor separately. 1 and 2 illustrate a case in which the monitor unit 6 includes three monitors, the quantity of the monitors may be variously determined according to the type or type of information requiring display.

The monitor unit 6 may further output one or more biometric information about the patient. In this case, one or more indicators indicating a patient's condition, for example, body temperature, pulse rate, respiration, and blood pressure, may be output through one or more monitors of the monitor unit 6. It may be output. In order to provide such biometric information to the master robot 1, the slave robot 2 measures biometric information including at least one of a body temperature measuring module, a pulse measuring module, a respiratory measuring module, a blood pressure measuring module, an electrocardiogram measuring module, and the like. It may include a unit. The biometric information measured by each module may be transmitted from the slave robot 2 to the master robot 1 in the form of an analog signal or a digital signal, and the master robot 1 monitors the received biometric information. Can be displayed via

The slave robot 2 and the master robot 1 may be coupled to each other through a wired communication network or a wireless communication network so that an operation signal and a laparoscope image input through the laparoscope 5 may be transmitted to the counterpart. If two operation signals by the two handles 10 provided in the master interface 4 and / or operation signals for adjusting the position of the laparoscope 5 need to be transmitted at the same time and / or at a similar point in time. Each operation signal may be independently transmitted to the slave robot 2. Herein, when each operation signal is 'independently' transmitted, it means that the operation signals do not interfere with each other and one operation signal does not affect the other signal. As described above, in order to transmit the plurality of operation signals independently of each other, in the generation step of each operation signal, header information for each operation signal is added and transmitted, or each operation signal is transmitted in the generation order thereof, or Various methods may be used such as prioritizing each operation signal in advance and transmitting the operation signal accordingly. In this case, the transmission path through which each operation signal is transmitted may be provided independently so that interference between each operation signal may be fundamentally prevented.

The robot arm 3 of the slave robot 2 can be implemented to be driven with multiple degrees of freedom. The robot arm 3 includes, for example, a surgical instrument inserted into a surgical site of a patient, a rocking drive unit for rotating the surgical instrument in the yaw direction according to the surgical position, and a pitch direction perpendicular to the rotational drive of the rocking drive unit. It comprises a pitch drive unit for rotating the surgical instruments, a transfer drive for moving the surgical instruments in the longitudinal direction, a rotation drive for rotating the surgical instruments, a surgical instrument drive unit installed on the end of the surgical instruments to cut or cut the surgical lesion Can be. However, the configuration of the robot arm 3 is not limited thereto, and it should be understood that this example does not limit the scope of the present invention.

One or more slave robots 2 may be used to operate the patient, and the laparoscope 5 for displaying the surgical site as an image image through the monitor unit 6 is implemented as an independent slave robot 2. May be In addition, as described above, embodiments of the present invention may be used universally in operations in which various surgical endoscopes (eg, thoracoscopic, arthroscopy, parenteral, etc.) other than laparoscopic are used.

3 is a block diagram schematically showing the configuration of a master robot and a slave robot according to an embodiment of the present invention, Figures 4a to 4l is a view showing a region designation method according to embodiments of the present invention. As described above, the master robot 1 and the slave robot 2 may be integrally implemented.

Referring to FIG. 3, the master robot 1 includes an image input unit 310, a screen display unit 320, an arm operation setting unit 330, a storage unit 340, a restricted area setting unit 350, and an arm operation unit 360. ), An operation determination unit 370, a reaction information processing unit 380, and a control unit 390. Although not shown, the master robot 1 includes an input unit for the operator to input a control command, a speaker unit for outputting the reaction information as auditory information (for example, a warning sound or a warning voice message), and visualizes the response information. It may further include one or more of the LED unit for outputting the information.

The slave robot 2 may comprise a robot arm 3 and a laparoscope 5. In addition, the robot arm 3 may be interpreted as a concept including an instrument unless explicitly limited herein. Although not shown, the slave robot 2 further includes a location information providing unit for providing location information of the robot arm 3 and the laparoscope 5, a biometric information measuring unit for measuring and providing biometric information about the patient. It may include. The position information providing unit masters, for example, information about how much the robot arm 3 or the laparoscope 5 has moved at an angle from the basic position (for example, the rotation angle of a driving motor such as a robot arm). It may be configured to provide to the robot 1.

The image input unit 310 receives an image input through a camera provided in the laparoscope 5 of the slave robot 2 through a wired or wireless communication network.

The screen display unit 320 outputs an image image corresponding to the image received through the image input unit 310 as visual information. In addition, the screen display unit 320 may output the response information as visual information (for example, screen flickering), or may further output corresponding information when biometric information is input from the slave robot 2. The screen display unit 320 may be implemented in the form of, for example, the monitor unit 6.

The arm manipulation setting unit 330 receives the arm manipulation setting from the operator or the setter and generates manipulation setting information on whether the robot arm 3 and / or the instrument is to be manipulated according to the manipulation instruction of the operator. . The arm manipulation setting may be, for example, a setting of whether or not the robot arm and / or the instrument located in the out-of-visible region not visible in the video image acquired by the laparoscope 5 is to be manipulated.

The manner in which the operator performs cancer manipulation settings can vary. Only a few of these examples are presented below.

As a first embodiment, the operator sets only the robot arm and / or instrument shown on the screen with reference to the image image input through the laparoscope 5 to be operable using the identification information, and other robots. Arms and instruments can be set inoperable.

As a second embodiment, after the master robot 1 receives the position information of each robot arm and / or instrument, the position of the laparoscope 5 and the image input angle from the slave robot 2, After calculating whether or not the image image of the coordinate range is input, it can be set to be operable only the robot arm and the instrument positioned to include some or all within the coordinate range. The laparoscope 5 may further include a distance sensing sensor for sensing a distance to the surface of the surgical site to calculate a coordinate range of the surgical site represented by the image image obtained by the laparoscope 5. Of course, the robot arm outside the coordinate range for the replacement of the robot arm or instrument to be used during the operation can be set to be manipulated.

The storage unit 340 stores the operation setting information and the restricted area coordinate information set by the restricted area setting unit 350 in the robot arm 3 and / or the instrument generated by the arm operating setting unit 330. In addition, when the response information processing method is designated by the operator, information on the response information processing type (eg, one or more of tactile information output method, auditory information output method, visual information output method, etc.) may be further stored.

The control unit 390, when the operation information for any robot arm (3) or the instrument is input by the operator, when the operation using the operator's arm operation unit 360 corresponds to the specified restricted area for the response information processing type With reference to the information, the reaction information processor 380 may control to perform a corresponding process. The controller 390 may include an operation determination unit 370 described below.

The restricted area setting unit 350 may approach the robot arm 3 or the instrument in order to prevent damage to a body part (eg, a blood vessel, an organ, etc.) of the patient under surgery due to a misoperation during operation. Create and store coordinate information for restricted areas that cannot be manipulated. Here, the restricted area may be designated or changed by the operator before or during the operation.

There may be a variety of ways for the operator to designate the restricted area and to generate corresponding restricted area coordinate information. In the present specification, a description will be given of a method of setting a restricted area by a method such as designating an arbitrary area by an operator, but only an area designated by an operator is set as an allowable area, and other areas are set as restricted areas. Naturally, the method can be applied.

The following is only a few examples of various embodiments in which an operator designates a restricted area. The area set as follows may be displayed overlaid on the image image obtained by the laparoscope 5.

As a first embodiment, when the operator wants to perform surgery on a limited organ (eg, liver, etc.), when the operator selects the organ using a menu item displayed on the monitor 6, the master robot 1 ) Analyzes the image image acquired by the laparoscope 5 (for example, the presence or absence of the selected organ by the color analysis of the surgical site included in the image image and the interpretation of its coordinate region, etc.). Coordinate information (eg, restricted area coordinate information or allowed area coordinate information) may be generated.

As a second embodiment, the allowable area may be set only for the coordinate range corresponding to the image image obtained and displayed by the laparoscope 5, and all other areas may be set as the restricted area. At this time, only the robot arm 3 or the instrument, which is partially or entirely located in the allowable area, will be operated by the operator's arm operation or the like.

In a third embodiment, 3 corresponding to an area (ie, a restricted area or an allowable area) set using a master controller and a controlled object moving according to its manipulation (eg, at least one of a robot arm, an instrument, and an endoscope). Dimensional coordinate values can be set. Hereinafter, a case in which the corresponding area is set as the restricted area will be described in detail with reference to the accompanying drawings.

For example, as shown in FIG. 4A, the end of the instrument is recognized as a three-dimensional mouse cursor, and the operator performs an operation of taking a point in space (that is, specifying a point) while viewing an image image acquired by the laparoscope 5. This allows the dots to be connected to be set as a restricted area.

That is, the operator sets an area to include S1 in the area on the image image obtained and displayed by the laparoscope 5 (in the image image, organs, blood vessels such as S1, S2, and S3 are included). After the end is positioned at the first vertex, after inputting a point designation command, the user moves to the second vertex and additionally inputs a point designation command. After repeating the vertex input operation such that an area to be set is defined, and inputting an area partition command, an area to which vertices previously set by the point designation command are connected is set. Each vertex may be connected in a straight line, but may be connected in a curved form, and the corresponding area may be set as a planar region (for example, a triangle or a circle) or a three-dimensional region (for example, a polyhedron shape). The aforementioned point designation command, area section command, and the like may be input using a master controller provided in the master interface 4 or the like.

As shown in FIG. 4B, the restricted area setting method may be extended by designating a vertex on a three-dimensional space to three-dimensionally set the restricted area.

By the above-described method, not only the three-dimensional restriction zone can be set in a manner of designating a vertex or the like on the three-dimensional space, but also a planar restriction zone such as a wall can be set.

In addition, the area set by the operator may be deformed by positioning the end of the instrument at one point of the outline of the area and moving the point out of the area or inward. Here, the outline and the outer surface of the region formed by the designated vertices may be interpreted and stored in advance as being the connection of the points.

That is, in order for the operator to further include S2 as well as S1, the operator performs the point movement after positioning the end of the instrument at the vertex P4 as shown in FIG. 4C. In addition, the end of the instrument is located on the line connecting the vertices P3 and P4 and then the point movement is performed. By such point movements, an area set to include S2 in the corresponding area may be extended.

The above-described deformation of the set area is not limited to be made only in a plane, and as shown in FIG. 4D, it is natural that the same can be performed even when the three-dimensional area is set. In addition, although the case in which the region is deformed by a point movement method using vertices is illustrated in FIG. 4, it is natural that the region may be deformed by pulling surfaces, edges, and the like in three-dimensional space.

In addition, the area set by the operator may be divided into a plurality of areas and separated into individual areas. That is, as shown in FIGS. 4E and 4F, the operator may divide a region set to include S1 and S2 into a first individual region including only S1 and a second individual region including only S2. To this end, the operator may designate two or more points (eg, Q1 and Q2) at a point to be separated, and then input a division command to separate the corresponding area into two or more separate areas.

In addition, after the area set by the operator is divided into a plurality of areas, unnecessary areas may be removed. That is, as shown in FIGS. 4G and 4H, the operator may allow the region set to include S1 and S2 to be reduced to the region including only S2. To do this, the operator assigns two or more points (e.g., Q1 and Q2) to the boundary point to be removed, allows the instrument end to be located in the area to be removed, and then enters a delete command. In this case, the separated area can be deleted.

In addition, the area set by the operator may be manipulated to be folded around the boundary point partitioning into a plurality of areas or to be restored to its original position. That is, as shown in FIGS. 4I and 4J, the operator divides a region set to include S1 and S2 into a first individual region including only S1 and a second individual region including only S2, and then each individual region is Folding operations can be performed to act as a restricted or permitted area. For reference, FIG. 4I is assumed to deform the restricted area in a planar manner by folding or restoring a planarly set area in plan view, and FIG. 4J is assumed to deform a three-dimensional restricted area to fold in a planar area. .

Referring to FIG. 4I, when an operator specifies two or more points (eg, Q1 and Q2) to function as boundary points and enters a folding command, an individual area including the point where the instrument end is located is centered on the boundary point. Rotation is performed. As a result, when the corresponding area is set as the restricted area, the S2 is able to operate the instrument in a state in which it is released from the restricted area. Thereafter, when the areas folded by the restore command are restored to their original state, S2 is also reset to the restricted area.

Similarly, as shown in FIG. 4J, the operator specifies a plurality of points (e.g., Q1-Q2 and Q3-Q4) that serve as boundary points, and enters a fold command, whereby one direction (e.g., The rotation and folding commands may be executed in the direction corresponding to the position so that the restricted area may be set in the shape of a three-dimensional figure. Of course, it may be set in advance as to what angle the individual regions at which positions are to be rotated around the boundary point. In addition, the three-dimensional shape of the area image shown in plan by the folding command may be various.

The method of setting and modifying the above-described region is one of arbitrary shapes (for example, circles, triangles, squares, spheres, hexahedrons, tetrahedrons, etc.) designated as a template, in addition to the method in which the operator precedes the region setting using an instrument. After the shape of the organ or the organ is shown in space, it is natural that the three-dimensional mouse (or the end of the instrument) can be deformed or separated. For example, as shown in Figure 4k, if you want to set the cardiac region as a restricted area, by selecting a template corresponding to the heart to be shown, such as by specifying a scale or by pulling the outline or outline The template can be adapted to suit the characteristics of the surgical patient. In this case, the control may be controlled to track the change of the actual organ (for example, position shift, shape change, etc.) that is deformed by the operator's manipulation of the object to be controlled so that the set restricted area is deformed correspondingly. To this end, for example, an image analysis technique such as an edge detection may be applied to an image of an actual organ.

As a fourth embodiment, a reference image (for example, an image image obtained by the laparoscope 5, an MRI image of a patient, etc.) of a surgical patient displayed through a monitor implemented as a touch sensitive input device is generated and generalized. In the human body modeling image, when the operator shows a certain range in the shape of a closed curve with a finger, the corresponding region may be set as a restricted area. In this case, the coordinates of the range shown by the operator are coordinated to correspond to the position of organs or blood vessels inside the body of the surgical patient based on an arbitrary reference point (for example, the position of the characteristic organs, the position of the laparoscope 5, etc.). The information is mapped and the coordinate information mapping may be processed by a position recognition method by conventional image analysis, and thus description thereof will be omitted.

As a fifth embodiment, modeling of each organ or blood vessel of a human model image generated using a reference image (eg, CT image, MRI image, etc.) of a surgical patient or a human model image generated of a general human body You can also set the area automatically using the data.

Referring to FIG. 4L, a human modeling image is generated using a reference image (eg, a CT image, an MRI image, etc.) of a surgical patient in step 450, and modeling data corresponding to the human modeling image (for example, in step 455). For example, the color, shape, size, etc. of each organ and blood vessel) are generated and stored. When the human body modeling image and the modeling data generated for the general human body are used, steps 450 and 455 may be omitted.

In step 460, long-term selection information is obtained. The organ selection information is obtained by using an image analysis technique (for example, pixel-by-pixel color analysis, organ outline analysis, etc.) to determine whether organs or blood vessels in the image image obtained and displayed by the laparoscope 5 are displayed, for example. After analysis, it can be automatically recognized against pre-stored modeling data. Of course, a method in which an operator selects an arbitrary organ from an organ list including a drop down menu or a human modeling image displayed through the monitor unit 6 may be used.

In operation 465, the corresponding area is partitioned with reference to the modeling data and displayed as an area image (see FIGS. 4A to 4K) for the operator to recognize. The master robot 1 recognizes the position of the organ corresponding to the organ selection information in consideration of the height of the patient, the lying down shape and the general position of the organ in the human body, and then the area of the shape defined by the modeling data. This is set so as to be overlaid and displayed in the image image obtained by the monitor unit 6 and / or the laparoscope 5. For example, when the heart is selected as organ selection information, modeling data is extracted from the storage unit 340 or the like to display an area image corresponding to the shape of the heart, and the area image corresponding to the extracted modeling data is the actual organ. The image may be processed to be overlaid and displayed on the video image.

In operation 470, the operator checks whether the area image overlaid on the image image corresponds to the area of the actual organ, and if it does not match, deforms the area image overlaid (see FIGS. 4D to 4K). The region image deformation in step 470 may be processed by the operator as described above, but if the organ is present in the image image acquired and displayed by the laparoscope 5, the shape (size) of the organ is determined using an image analysis technique. And so on, the image may be automatically transformed to match the shape of the organ.

In addition, a method of setting and using a restricted area in advance may also be applied. For example, a method of using a three-dimensional image obtained by reconstructing a CT, MRI image, etc., in which a restriction zone is set in advance in a three-dimensional image, and the image image obtained by the laparoscope 5 (for example, Endoscope images). Of course, the matching of the 3D image and the real-time image may be omitted, and a method of matching only a few points required may also be applied.

The arm manipulation unit 360 is a means for allowing the operator to manipulate the position and function of the robot arm 3 of the slave robot 2. The arm manipulation unit 330 may be formed in the shape of the handle 10 as illustrated in FIG. 2, but the shape is not limited thereto and may be modified in various shapes for achieving the same purpose. Further, for example, some may be formed in the shape of a handle, others may be formed in a different shape, such as a clutch button, finger insertion tube or insertion to enable the operator's fingers can be inserted and fixed to facilitate the operation of the surgical tool More rings may be formed. In addition, if the laparoscope 5 for receiving an image is not fixed at a specific position, and the position and / or the image input angle can be moved or changed by the operator's adjustment, the clutch button 14 or the like is used as the laparoscope 5. It may be set to function for the adjustment of the position and / or the image input angle of.

The operation determination unit 370 determines whether the operation information by the operator's operation of the arm operation unit 360 is valid operation information by referring to at least one of the operation setting information and the restricted area coordinate information stored in the storage unit 340.

For example, the manipulation determination unit 370 may determine that the manipulation information is not valid when the manipulation information according to the manipulation of the arm manipulation unit 360 is a robot arm set to be inoperable with reference to the manipulation setting information.

In addition, the manipulation determination unit 370 may determine that the manipulation information according to the manipulation of the arm manipulation unit 360 is invalid when the robot arm 3 contacts the restricted region with reference to the restricted region coordinate information. . To this end, the operation determining unit 370 is displacement information (for example, the robot arm 3) for which the robot arm 3 or the instrument received from the position information providing unit has moved at which angle and at what angle from the basic position. Rotation angle of a driving motor, etc.).

The response information processing unit 380 performs the processing of the response information specified by the operator or / or the setter when the operation determination unit 370 determines that the operator's operation information is invalid operation information. The processing of the response information includes visual information such as tactile information output for the operator to recognize force by applying force feedback to the handle 10 and outputting a warning message corresponding to the image image obtained by the blinking LED or the laparoscope 5. May be processed in one or more ways, such as output of auditory information, such as output of warning sounds.

The controller 390 controls the operation of each component so that the above-described function can be performed. The controller 390 may perform a function of converting an image input through the image input unit 310 into an image image to be displayed through the screen display unit 320.

5 is a flowchart illustrating a method for setting a restricted area according to an embodiment of the present invention.

In the present embodiment described below, the generation and storage of the arm operation setting information and the generation and storage of the restricted area setting information are illustrated as sequential steps, but the order of each step may be changed or performed simultaneously.

Referring to FIG. 5, in operation 510, the master robot 1 receives an arm manipulation setting on whether to allow the robot arm 3 and / or the instrument located in the out-of-visible region to be manipulated by the operator or the setter.

In operation 520, the master robot 1 generates and stores arm operation setting information corresponding to the input arm operation setting.

In step 530, the master robot 1 inputs a restricted area setting for designating an area restricted from the operator or setter so that the robot arm 3 and / or the instruments located in the visible and non-visible areas are restricted from entering in the process of operating. Receive.

In step 540, the master robot 1 generates and stores the restricted zone setting information corresponding to the entered restricted zone setting.

As such, the operator or / and setter may be displayed by pre-acquired image information (e.g., one or more of patient images such as CT, MRI, etc., virtual images modeled to be generalized, etc.) or / and clicked on a drop down menu. Long-term items can be used to establish restricted areas before or during surgery. In the area set as the restricted area, the robot arm 3 and / or the instrument is not operated in spite of the user's operation, and the access or contact to the restricted area is fed back to the operator as reaction information, thereby injuring the surgical procedure. The occurrence of medical accidents due to malfunctions can be prevented for important areas that should not be. The response information may be output to the operator in one or more of a tactile information type, a visual information type, and an auditory information type.

For example, when a surgeon designates a specific area using an external input device such as a touch screen or a mouse device in an image image obtained and viewed by the laparoscope 5, the designated area is set as a restricted area. can do.

Alternatively, the patient image information such as CT, MRI, or the like taken in advance or a reconstructed image from the patient image information is displayed on the operator's screen, and if the operator designates a specific region in the image displayed on the screen, the designated region becomes a restricted area. It can also be set.

In this case, as described above, a process of recognizing the coordinate range of the designated specific region may be performed by image analysis or the like.

As such, when the restricted area is designated, the operator may designate the restricted area by referring to the screen on which the corresponding image is displayed, may be specified in advance in the photographed image, and in addition, the restricted area may be specified in various ways.

6 and 7 are flowcharts illustrating an operation limiting method of a surgical robot system according to an exemplary embodiment of the present invention, and FIG. 8 is an exemplary view of a screen display for explaining an operation limiting method according to an embodiment of the present invention. .

Referring to FIG. 6, in operation 610, the master robot 1 receives arm manipulation information according to manipulation of the arm manipulation unit 360 from an operator.

The master robot 1 determines whether the arm operation information input in step 520 is for the robot arm 3 and / or the instrument located in the out-of-visible region. For example, the robot arm positioned in the non-visible region may be manipulated by a misoperation of an operator or when the robot arm 3 positioned in the visible region is to be replaced with the robot arm 3 positioned in the non-visible region. (3) The operation may be commanded. As mentioned above, the robot arm 3 may be interpreted as a concept including an instrument, unless expressly limited herein.

As a result of the determination in step 620, if it is not an operation command for the robot arm 3 or the like located in the out-of-visible region (i.e., the visible region), the process proceeds to step 630, whereby the master robot 1 generates an arm manipulation instruction. Output Step 710 shown in FIG. 7 is followed by execution of step 630. Referring to FIG. 8, robot arms or instruments positioned in the visible region may be referred to as 830a and 830b, and robot arms or instruments positioned in the non-visible region may be referred to as 850.

However, as a result of the determination in step 620, if it is an operation command for the robot arm 3 or the like located in the non-visible region, the process proceeds to step 640, where the master robot 1 is located in the robot arm 3 or the like positioned in the non-visible region. It is determined whether the operation setting information for allowing the operation to be permitted is stored.

As a result of the determination in step 640, if operation setting information for allowing the operation of the robot arm 3 and the like located in the out-of-visible region is stored, the process proceeds to step 630 and the master robot 1 generates and outputs an arm operation command. . In this case, the robot arm 3 or the like located in the visible region or the non-visible region may be moved or manipulated in the same manner by the arm manipulation command. An arm operation instruction may be generated to allow the robot arm 3 or the like positioned in the visible area to be processed slower than the operation speed or the operation speed.

However, as a result of the determination in step 640, if operation setting information is stored so that the operation on the robot arm 3, etc. located in the out-of-visible region is not allowed, the process proceeds to step 650, whereby the master robot 1 performs the output processing of the reaction information. To allow the operator to recognize it.

Referring to FIG. 7, in operation 710, the master robot 1 receives arm operation state information from the slave robot 2. The arm operation state information may include, for example, information about how much the robot arm 3 or the instrument has moved at an angle from the basic position (for example, the rotation angle of a driving motor such as the robot arm), The corresponding information may be provided from the location information providing unit.

In operation 720, the master robot 1 determines whether the robot arm 3 or the instrument is in contact with a preset restricted area with reference to the arm operation state information.

As described above, the restricted area may be designated by the operator with reference to the screen on which the corresponding image is displayed, may be specified in advance in the captured image, or may be set by various methods.

In the restricted area set by the above-described process, the augmented reality technique is applied to the real image acquired by the laparoscope 5, etc. (see 840 of FIG. 8), a line treatment method of a specific color, and an opacity process. It can be handled variously in one or more ways. As a result, the operator may continuously inform that the area is a restricted area. Naturally, the restricted zone display method may be processed in conjunction with the movement or enlargement of the display screen.

As a result of the determination of step 720, if the robot arm 3 or the instrument did not contact the restricted area, the process proceeds to step 610 again.

However, as a result of the determination in step 720, if the robot arm 3 or the instrument is in contact with the restricted area, the process proceeds to step 730 where the master robot 1 performs control of the operation limitation of the robot arm and also performs response information output processing. This allows the operator to recognize it.

Operation restriction control of the robot arm 3 in the restricted zone can, for example, allow the set restricted zone to be recognized as if it were a virtual wall. In other words, when the end of the instrument or part of the intermediate shaft tries to enter the restricted area during the robotic operation, the operator restricts the operation of the instrument as if the instrument is not operated by the virtual wall and recognizes it through the output of the response information. Do it.

Of course, the contact with the restricted area is not necessarily limited to not operate the instrument, it is also possible to determine whether the operation by the operator's choice. For example, if bleeding occurs in a region (eg, a blood vessel) set as a restricted area irrespective of an operator's intention, rapid bleeding may be possible if the instrument is set to operate. Here, the command for ignoring the restricted area setting for allowing an instrument or the like to be operated in the restricted area when the contacted with the restricted area may be input by, for example, an operator operating a predetermined button or pedal. will be.

Further, the restricted area is not necessarily set only within the area currently displayed on the screen (ie, the visible area 810), and is not displayed on the screen (ie, the non-visible area 820) as illustrated in FIG. 8. Of course, the restricted area 840 may be designated and set. That is, even when the restricted area 840 is set in the out-of-visible region 820, if the instrument is in contact with the restricted area may cause a serious risk to the safety of the surgical patient, the movement of the instrument is limited to prevent this Can be controlled.

As a control method for limiting the operation of the instrument, for example, a method of locking the instrument so as not to operate may be used, and further, by allowing reaction force to be fed back to the operation handle of the user who operates the locked instrument, The user may be informed that the operation is limited. In addition, the manner in which the reaction information is output may vary as described above.

The operation limiting method of the above-described surgical robot system may be implemented by a software program or the like. Codes and code segments constituting a program can be easily inferred by a computer programmer in the art. The program is also stored in a computer readable media, and read and executed by a computer to implement the method. The information storage medium includes a magnetic recording medium, an optical recording medium and a carrier wave medium.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

1 is a plan view showing the overall structure of a surgical robot according to an embodiment of the present invention.

Figure 2 is a conceptual diagram showing a master interface of the surgical robot according to an embodiment of the present invention.

Figure 3 is a block diagram schematically showing the configuration of a master robot and a slave robot according to an embodiment of the present invention.

4A to 4L are diagrams illustrating a region designation method according to embodiments of the present invention.

5 is a flow chart showing a method for setting a restricted area according to an embodiment of the present invention.

6 and 7 is a flow chart illustrating a method of limiting the operation of the surgical robot system according to an embodiment of the present invention.

8 is an exemplary view of a screen display for explaining an operation restriction method according to an embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

1: master robot 2: slave robot

3: robot arm 4: master interface

5: Laparoscope 6: Monitor

10: handle 310: video input unit

320: screen display unit 330: arm operation setting unit

340: storage unit 350: restricted area setting unit

360: arm operation unit 370: operation determination unit

380: reaction information processor 390: control unit

Claims (67)

As a surgical robot, A restricted area setting unit configured to receive restricted area setting information about a zone in which manipulation of a control object is restricted and generate and store restricted area coordinate information; An arm manipulation unit which receives manipulation information for manipulation of the control object; And And an operation determination unit determining whether an outer surface of the control object is in contact with the coordinate information of the restricted area by referring to the displacement information of the control object according to the manipulation information. The control object is a surgical robot, characterized in that at least one of the robot arm, instrument and endoscope. The method of claim 1, And a controller configured to control the control target object to be restricted in operation when the outer surface of the control target object contacts the limited area coordinate information, and to output response information corresponding thereto. The method of claim 2, The control unit, When the external surface of the control object is in contact with the coordinate information of the restricted area, after determining whether to control the control object, the control object is limited to the operation only when the control object is determined to be restricted operation. Surgical robot, characterized in that for controlling. The method of claim 3, Further including an input unit for receiving a command to ignore the restricted area setting, The control unit is a surgical robot, characterized in that the processing to allow the manipulation of the object to be controlled, if the command to ignore the restricted zone setting is input. The method of claim 2, The reaction information is a surgical robot, characterized in that at least one of tactile information, visual information and auditory information. The method of claim 1, The limited area coordinate information is a surgical robot, characterized in that the coordinate range information corresponding to the closed curve input using the touch-sensitive screen on which the image image is displayed. The method of claim 6, The image image is a surgical robot, characterized in that at least one of the image image obtained by the endoscope, CT image, MRI image and human modeling image. The method of claim 1, The limited area coordinate information is a surgical robot, characterized in that the coordinate range information of the area that is not displayed in the image image obtained by the endoscope. The method of claim 1, The endoscope is a surgical robot, characterized in that at least one of the laparoscope, thoracoscopic, arthroscopy, parenteral, bladder, rectal, duodenum, mediastinoscope, cardiac. The method of claim 1, Wherein the restricted area coordinate information is coordinate range information of a figure formed by positions designated as the restricted area setting information among positions at which the control object is manipulated by operation information of the arm manipulation unit; robot. The method of claim 10, And the controlled object is manipulated in three-dimensional space, and the designated positions are points designated to form an outline or an outer surface of a figure in three-dimensional space, respectively. The method of claim 1, The arm manipulation unit may further include an arm manipulation setting unit configured to generate and store manipulation setting information on whether to allow manipulation of each control target object by manipulation of the arm manipulation unit. And the manipulation determining unit further determines whether the manipulation information for any control object is valid manipulation information. As a motion limiting method of a surgical robot, Receiving restricted zone setting information on a zone in which manipulation of a control object is restricted, generating and storing restricted zone coordinate information; Receiving manipulation information for manipulation of the control object; Determining whether an outer surface of the controlled object is in contact with the limited area coordinate information by referring to the displacement information of the controlled object according to the manipulation information; And And controlling the controlled object to be restricted in operation when the outer surface of the controlled object is in contact with the limited area coordinate information. The control object is a method of limiting the operation of the surgical robot, characterized in that at least one of the robot arm, instrument and endoscope. The method of claim 13, The controlling step, Determining whether the controlled object is controlled to be restricted when the outer surface of the controlled object is in contact with the restricted area coordinate information; And And controlling the controlled object to be restricted in operation when the control target object is determined to be limited in operation. The method of claim 14, The determining may include determining whether or not to control the object to be controlled by judging whether a restricted area setting ignore command has been input or not. The method of claim 13, The controlling step, And controlling the reaction information to be output when the outer surface of the control object is in contact with the restriction area coordinate information. The method of claim 16, The reaction information is a motion limiting method of the surgical robot, characterized in that at least one of the tactile information, visual information and auditory information. The method of claim 13, The limited area coordinate information is a motion limiting method of a surgical robot, characterized in that the coordinate range information corresponding to the closed curve input using the touch-sensitive screen on which the image image is displayed. The method of claim 18, The image image is a method of limiting the operation of the surgical robot, characterized in that at least one of the image image obtained by the endoscope, CT image, MRI image and human modeling image. The method of claim 13, The endoscope is a method of limiting the operation of the surgical robot, characterized in that at least one of the laparoscope, thoracoscopic, arthroscopy, parenteral, bladder, rectal, duodenum, mediastinoscope, cardiac. The method of claim 13, The restriction area coordinate information is a motion limiting method of a surgical robot, characterized in that the coordinate range information of the area that is not displayed in the image image obtained by the endoscope. The method of claim 13, Wherein the restricted area coordinate information is coordinate range information of a figure formed by positions designated as the restricted area setting information among positions at which the control object is manipulated by operation information of the arm manipulation unit; How to limit robot's motion. The method of claim 22, And the control object is manipulated in three-dimensional space, and the designated positions are points designated to form an outline or an outer surface of a figure in three-dimensional space, respectively. The method of claim 13, Generating and storing manipulation setting information on whether to allow manipulation of each control object by manipulation of the arm manipulation unit; And And determining whether the manipulation information for any control object is valid manipulation information. The method of claim 24, If the manipulation information is invalid manipulation information, further comprising controlling the reaction information to be output. A program of instructions that can be executed by a digital processing device is tangibly embodied and executed by the digital processing device in order to perform the motion limiting method of the surgical robot according to any one of claims 13 to 25. Recording medium. As a surgical robot, A display unit for displaying a video image obtained and provided by an endoscope; An arm manipulation unit which receives manipulation information for manipulation of the object to be controlled; And And a controller configured to control a region image corresponding to a region including a plurality of positions of which a point designation command is input from among the positions to which the control object is moved is overlaid on the image image and displayed. The control object is a surgical robot, characterized in that at least one of the robot arm, the instrument and the endoscope. The method of claim 27, And by input of a point designation command and a segmentation command for two or more positions defining a boundary line connected to segment the region image, the region image is divided into two or more separate regions. The method of claim 27, Area image so that, by input of a point designation command and a folding command for two or more positions defining a boundary line connected to segment the area image, one or more individual areas are rotated relative to the other one or more individual areas based on the boundary line. Surgical robot, characterized in that the deformation. 30. The method of claim 29, And a region image in which the individual regions which are rotated and moved on the basis of the boundary line and returned to their original positions by the restoration command are displayed. The method of claim 27, And one or more individual regions are erased by inputting a point designation command and a deletion command for two or more positions defining a boundary line connected to segment the region image. The method of claim 27, The outline and the outer surface of the area image are recognized as a series of points, and by the point designation and movement command for any one point, the surgical robot, characterized in that the deformation process. The method of claim 27, And a coordinate range extracted to correspond to the area image is set as a restricted area or an allowable area. 34. The method of claim 33, When the coordinate range extracted to correspond to the area image is set as the restricted area, it is determined whether the outer surface of the controlled object is in contact with the coordinate range by referring to the displacement information of the controlled object according to the manipulation information. Surgical robot further comprising an operation determination unit. The method of claim 34, wherein And the control unit controls the control target object to be restricted in operation when the outer surface of the control target object is in contact with the coordinate range, and controls to output response information corresponding thereto. 36. The method of claim 35 wherein The arm manipulation unit may further include an arm manipulation setting unit configured to generate and store manipulation setting information on whether to allow manipulation of each control target object by manipulation of the arm manipulation unit. And the manipulation determining unit further determines whether the manipulation information for any control object is valid manipulation information. The method of claim 27, The endoscope is a surgical robot, characterized in that at least one of the laparoscope, thoracoscopic, arthroscopy, parenteral, bladder, rectal, duodenum, mediastinoscope, cardiac. As an area setting method of a surgical robot, Displaying a video image obtained and provided by the endoscope; Receiving manipulation information for manipulation of the object to be controlled; And And controlling a region image corresponding to a region including a plurality of positions of which a point designation command is input among the positions to which the control object is moved is overlaid on the image image and displayed. The object to be controlled is at least one of a robot arm, an instrument, and the endoscope, and a coordinate range extracted to correspond to the area image is set to a restricted area or an allowable area. 39. The method of claim 38, The region image is divided into two or more separate regions by input of a point designation command and a segmentation command for two or more positions defining a boundary line connected to segment the region image; How to set up. 39. The method of claim 38, By input of a point designation command and a folding command for two or more positions defining a boundary line connected to segment the region image, one or more individual regions are rotated and overlapped with respect to the one or more individual regions with respect to the boundary line. An area setting method of a surgical robot, characterized in that the area image is displayed. The method of claim 40, The area setting method of the surgical robot, characterized in that for displaying the area image is rotated relative to the boundary line by the input of the restore command to return to the original position of the individual areas displayed superimposed. 39. The method of claim 38, And at least one individual area is deleted by inputting a point designating command and a deleting command for two or more locations defining a boundary line connected to divide the area image. 39. The method of claim 38, When the coordinate range extracted to correspond to the area image is set as the restricted area, it is determined whether the outer surface of the controlled object is in contact with the coordinate range by referring to the displacement information of the controlled object according to the manipulation information. Area setting method of the surgical robot further comprising the step. The method of claim 43, In the controlling step, And when the external surface of the object to be controlled is in contact with the coordinate range, the object to be controlled is controlled to be controlled to be controlled, and the response information is controlled to output response information corresponding thereto. The method of claim 44, Generating and storing manipulation setting information regarding whether to allow manipulation of each control object; And And determining whether the manipulation information for any manipulation target object is valid manipulation information. 39. The method of claim 38, The endoscope is a laparoscopic, thoracoscopic, arthroscopic, parenteral, bladder, rectal, duodenum, mediastinum, cardiac, the area setting method of the surgical robot, characterized in that one or more. As a surgical robot, A display unit for displaying a video image obtained and provided by an endoscope; A storage unit storing human modeling information and corresponding modeling data; And a controller configured to extract modeling data corresponding to the acquired organ selection information about one or more organs, and to control a region image corresponding to the modeling data to be overlaid and displayed on the image image. The modeling data is a surgical robot, characterized in that it comprises one or more of the color, shape, size of the organ. 49. The method of claim 47, The human body modeling information and the modeling data is a surgical robot, characterized in that generated using a reference image of at least one of the CT image, MRI image. The method of claim 48, And the organ selection information is selected and designated as an organ corresponding to one or more of the shape and color of the organ recognized by the image recognition of the image image among the modeling data. The method of claim 49, And the area image is modified to have an outline corresponding to an outline of an organ recognized by edge detection of the image. 51. The method of claim 50, Further comprising a arm operation unit for receiving operation information for the operation of the controlled object including at least one of the robot arm, the instrument and the endoscope, And one or more of position change and shape change of the organ by the manipulation information, wherein the region image is deformed to correspond to the outline of the organ. 49. The method of claim 47, The outline and the outer surface of the area image are recognized as a series of points, and by the point designation and movement command for any one point, the surgical robot, characterized in that the deformation process. 49. The method of claim 47, And by input of a point designation command and a segmentation command for two or more positions defining a boundary line connected to segment the region image, the region image is divided into two or more separate regions. 49. The method of claim 47, By input of a point designation command and a folding command for two or more positions defining a boundary line connected to segment the region image, one or more individual regions are rotated and overlapped with respect to the one or more individual regions with respect to the boundary line. Surgical robot, characterized in that the area image is displayed. 55. The method of claim 54, And a region image in which the individual regions which are rotated and moved on the basis of the boundary line and returned to their original positions by the restoration command are displayed. 49. The method of claim 47, And one or more individual regions are deleted by inputting a point designation command and a deletion command for two or more positions defining a boundary line connected to segment the region image. As an area setting method of a surgical robot, Storing human modeling information and corresponding modeling data; Displaying a video image obtained and provided by the endoscope; And Extracting modeling data corresponding to the acquired organ selection information for one or more organs, and controlling an area image corresponding to the modeling data to be overlaid and displayed on the image image, The modeling data, the area setting method of the surgical robot, characterized in that it comprises one or more of the color, shape, size of the organ. The method of claim 57, The human modeling information and the modeling data is a region setting method of a surgical robot, characterized in that it is generated using a reference image that is one or more of the CT image, MRI image. 59. The method of claim 58, And the organ selection information is selected and designated as an organ corresponding to at least one of the shape and color of the organ recognized by the image recognition of the image image among the modeling data. The method of claim 59, And the area image is deformed to have an outline corresponding to an outline of an organ recognized by edge detection of the image image. The method of claim 60, When one or more of position change or shape change of the organ is generated by operation information for manipulation of a controlled object including at least one of a robot arm, an instrument, and an endoscope, the region image is modified to correspond to the outline of the organ. Method for setting the area of the surgical robot, characterized in that the process. The method of claim 57, The outline of the area image is recognized as a series of points, and the area image is deformed by a point designation and movement command for any one point. The method of claim 57, The region image is divided into two or more separate regions by input of a point designation command and a segmentation command for two or more positions defining a boundary line connected to segment the region image; How to set up. The method of claim 57, By input of a point designation command and a folding command for two or more positions defining a boundary line connected to segment the region image, one or more individual regions are rotated and overlapped with respect to the one or more individual regions with respect to the boundary line. An area setting method of a surgical robot, characterized in that the area image is displayed. 65. The method of claim 64, The area setting method of the surgical robot, characterized in that for displaying the area image is rotated relative to the boundary line by the input of the restore command to return to the original position of the individual areas displayed superimposed. The method of claim 57, And at least one individual area is deleted by inputting a point designating command and a deleting command for two or more locations defining a boundary line connected to divide the area image. 67. A program of instructions that can be executed by a digital processing apparatus is tangibly implemented and digitally processed to perform the method for setting a region of a surgical robot according to any one of claims 38 to 46 and 57 to 66. Recording medium recording a program that can be read by the device.

Images