KR101740286B1 - Surgical robot system and adaptive control method thereof - Google Patents

Abstract

A surgical robot system and an adaptive control method are disclosed. A master interface for controlling a position of the control object included in a surgical robot system to which a control object, which is at least one of a surgical endoscope and a surgical tool, is mounted, includes a manipulator for inputting a user command for manipulating a control object, A storage unit for storing a moving image scale factor corresponding to at least one of the image information provided from the endoscope and the reference image; and a determination unit for determining whether the current image information currently provided from the surgical endoscope is reduced or enlarged, , A scale factor arithmetic unit for adjusting and resetting a moving scale factor so as to be proportional to a degree of reduction or enlargement of the control signal, and an operation signal generating unit for generating an operation signal for the control object by converting the user command using the operation unit to a reset motion scale factor Section. The present invention is capable of shortening the operation time and alleviating the fatigue of the operator during the operation progress, and intuitively recognizing the movement distance of the surgical tool based on the displayed image information.

Description

[0001] The present invention relates to a surgical robot system and an adaptive control method,

The present invention relates to a surgical robot system and an adaptive control method.

Medically, surgery refers to the act of treating a skin, mucous membrane, or other tissue using a medical device, cutting, manipulating, or manipulating the disease. Particularly, due to problems such as hemorrhage, side effects, patient's pain, scarring, etc., the incision is made by cutting the skin of the surgical site and opening, treating, And is attracting attention as an alternative.

The surgical robot system is generally composed of a master robot and a slave robot, and the master robot and the slave robot may be implemented independently or integrally.

When the operator manipulates a manipulator (e.g., a handle) provided in the master robot, the surgical tool (i.e., an instrument) coupled to the robot arm of the slave robot or held by the robot arm is operated to perform the operation.

In order to allow the operation according to manipulation of the manipulator of the operator to proceed, the surgical robot system performs the specified operation in various operation modes. The master repositioning mode, the laparoscopic position control mode, and the hold mode, which are some of the various operation modes, will be described as follows.

The master repositioning mode is an operation mode of the surgical robot system when the provided master repositioning button is depressed and is an operation mode in which an operation command according to the manipulation of the manipulator is not transmitted to the slave robot even if the operator moves the manipulator. The master repositioning button for entry into the master repositioning mode may be implemented with a clutch button, as will be described below.

The laparoscopic position control mode is an operation mode in which, when the operator moves the manipulator, the laparoscope is operated instead of the surgical tool mounted on the slave robot. The operator can manipulate the laparoscope in a laparoscopic position control mode or the like in a laparoscopic position control mode by moving the manipulator while pressing a camera button (e.g., a pedal) provided in advance in the master robot.

The hold mode is an operation mode that fixes the current operation state as it is, and is an operation mode in which both the master robot and the slave robot are stopped as they are. The surgical robot system can enter the hold mode by an operation such as pressing the preset operator or removing the face from the console.

Hereinafter, operations of the master repositioning mode and the laparoscopic position control mode of the surgical robot system according to the prior art will be briefly described with reference to the related drawings.

FIG. 1 is a plan view showing the entire structure of a surgical robot system according to the related art, FIG. 2 is a conceptual view showing a master interface of a surgical robot system according to the related art, FIGS. 3a and 3b are cross- FIG. 2 is a conceptual diagram illustrating a movement control process of an instrument provided in the robot system. FIG.

1 and 2, the surgical robot system includes a slave robot 2 for directly performing surgery on a patient lying on a surgical table and a master robot 1 for remotely controlling the slave robot 2 . When the master robot 1 and the slave robot 2 are integrally formed, the master interface 4 may correspond to the interface part of the integrated surgical robot.

The slave robot 2 includes a robot arm 3 and a laparoscope 5. [ The master interface 4 of the master robot 1 includes a monitor 6, a handle 10, a master repositioning button and a camera button. Each of the master repositioning button and the camera button may be implemented in the form of a clutch button 14 or a pedal 30 or the like.

The master robot 1 and the slave robot 2 are coupled to each other via a wired communication network or a wireless communication network, and operation signals and the like can be transmitted to the other party. If two operation signals by the two handles 10 provided in the master interface 4 and / or operation signals for adjusting the position of the laparoscope 5 need to be transmitted at the same time and / , The respective operation signals can be transmitted to the slave robot 2 independently of each other.

The robot arm 3 of the slave robot 2 can be implemented to be driven with multiple degrees of freedom. The robot arm 3 includes, for example, a surgical tool inserted into a surgical site of a patient, a swinging drive unit for rotating the surgical tool in a yaw direction according to a surgical position, a pitch direction , A pitch driving part for rotating the surgical tool, a feed driving part for moving the surgical tool in the longitudinal direction, a rotation driving part for rotating the surgical tool, and a surgical tool driving part for cutting or cutting the surgical lesion, .

The two handles 10 provided on the master interface 4 are formed so that the operator can grasp the hands and grasp the hands so that an operation signal corresponding to the operation of the operator's handle 10 is transmitted to the slave robot 2, The arm 3 is controlled.

In the monitor 6 of the master interface 4, an image input by the laparoscope 5 is displayed as an image image. Further, the monitor 6 may additionally display a patient electrocardiogram graph or the like as illustrated.

The master repositioning button provided in the master interface 4 is used to set a work area for the operation progression (for example, an area photographed by the laparoscope 5 for operation of the surgical tool) (For example, the center position of the current working area is moved to the basic position of the handle 10) This is a control button that can be used when you want to return to the menu.

When the operator operates the steering wheel 10 in any direction in which the master repositioning button is pressed (for example, the pedal 30 is stepped on), a manipulation command for manipulating the manipulation tool or a manipulation command for the laparoscope 5, (2), the operator can return the handle (10) to the original state for the convenience of operation.

That is, when the operator wishes to move the work area from the first position to the second position, the operator first inputs a camera button input for moving the laparoscope 5 (that is, enters the laparoscopic position control mode) To move the laparoscope 5 to a desired position and then to prevent the laparoscope 5 from moving when the handle 10, which has been manipulated in an arbitrary direction for movement of the laparoscope 5, is returned to its original position After the handle 10 is returned to its original position in the state where the master master repositioning button is depressed (master repositioning mode), the pressing of the master repositioning button is terminated, The behavior area can be readjusted so that the handle 10 can be operated.

However, by the operation of the handle 10 in the laparoscopic position control mode, the degree to which the laparoscope 5 is moved in the corresponding direction is limited to the size of the work area 310 of a predetermined size, If the user wants to move the area 310 from the area (a) to the area (b), it causes a great inconvenience to the operator.

That is, when the operator wants to move the work area 310, which is the area (a), to the area (b), the operator inserts the work area 310 into (a), (p1), the operator is required to input a camera button for moving the laparoscope 5, move the laparoscope 5 according to the operation of the handle 10, move the handle 5, If the path for moving the work area 310 is changed, the number of times of inputting the camera button and the master repositioning button is more Can be increased. Here, c_a, c_b, and the like in FIG. 3B indicate the moving position of the laparoscope 5 in accordance with the operation of the handle 10.

Further, the problem of the surgical tool moving method in the surgical robot system according to the related art is maintained regardless of the magnification of the image information inputted through the laparoscope 5. [

This is because the MSF (Motion Scale Factor) defining the movement interval of the surgical tool is fixed to be constant in the preset work area 310 so as to have an optimal movement, and is input through the laparoscope 5 It is not considered at all which area and which area of the image information is included.

In addition, the laparoscopic (5) movement mechanism in the work area unit results in prolongation of the operation time, and also causes serious fatigue to the physician who performs the operation under the high tension condition.

The above-described background technology is technical information that the inventor holds for the derivation of the present invention or acquired in the process of deriving the present invention, and can not necessarily be a known technology disclosed to the general public prior to the filing of the present invention.

An object of the present invention is to provide a surgical robot system and an adaptive control method capable of adaptively varying a moving scale factor of a surgical tool according to which area and a range of subjects the image information photographed through the laparoscope is related to .

An object of the present invention is to provide a surgical robot system and an adaptive control method capable of shortening an operation time and alleviating fatigue of an operator during a surgical procedure and intuitively recognizing the moving distance of the surgical tool based on the displayed image information .

The present invention is to provide a surgical robot system and an adaptive control method capable of maximizing the operational convenience of laparoscopic and / or surgical instruments.

An object of the present invention is to provide a surgical robot system and an adaptive control method capable of realizing a new operation by improving a camera button used for laparoscopic operation and adding a separate position control / speed control button.

According to an aspect of the present invention, there is provided a master interface for controlling a position of a control object included in a surgical robot system to which a control object, which is at least one of a surgical endoscope and a surgical tool, A storage unit for storing a reference image which is at least one of image information provided from the surgical endoscope and a motion scale factor corresponding to the reference image, A scale factor arithmetic unit for determining whether the information is scaled or enlarged relative to the reference image and adjusting the scaling factor so as to be linear or nonlinearly proportional to the scaling or non-linearity of the scaling factor; The scale factor is converted to correspond to the control person And an operation signal generating section for generating an operation signal for the sieve.

The scale factor operation unit can detect the current image information in the reference image in accordance with a preset image analysis technique to determine whether the current image information is reduced or enlarged relative to the reference image.

The scale factor operation unit can adjust the movement scale factor using the proportional relationship between the conversion information that is at least one of the horizontal size, the vertical size, the diagonal line length, and the width of the set region and the information corresponding to the conversion information in the reference image.

The moving distance of the control object according to the user command can be adjusted depending on the moving scale factor.

Surgical endoscopy may be one or more of laparoscopic, thoracoscopic, arthroscopic, non-operative, cystoscopic, rectal, duodenal, mediastinal, and cardiac.

According to another aspect of the present invention, there is provided a master interface for controlling a position of a control object included in a surgical robot system to which a control object, which is at least one of a surgical endoscope and a surgical tool, A storage unit for storing at least one moving scale factor according to a photographing condition of the endoscope for surgery; and a search unit for searching the storage unit for a corresponding moving scale factor by receiving information on photographing conditions of the endoscope for surgery, And a manipulation signal generator for generating an manipulation signal for the control object by converting the user command using the manipulation unit to correspond to the set scaling factor.

The photographing conditions may be at least one of the image capturing zoom magnification, the photographing position, the distance from the surgical site surface, and the actual size of the surgical site to be photographed.

The moving scale factor in the case where the image capturing zoom magnification factor is high can be a value relatively smaller than the moving scale factor in the case where the image capturing zoom factor is low.

The moving scale factor when the surgical endoscope is relatively inserted into the human body may be smaller than the moving scale factor when the surgical endoscope is inserted relatively less.

The mobile scale factor may be set to a relatively small value as the distance from the surface of the surgical site is closer to the sensed value from a sensor provided to sense the distance to the surgical site surface.

The smaller the actual size of the surgical site to be photographed, the smaller the moving scale factor can be set to a relatively small value.

The moving distance of the surgical tool according to the user command can be adjusted depending on the moving scale factor.

If the user command is a command for a movement operation of the surgical endoscope, the movement distance of the surgical endoscope may be adjusted depending on the movement scale factor.

The moving scale factor can be computed in real time and stored in the storage unit according to the variation of the photographing condition.

The scale factor operating section can set a larger moving scale factor than when the operating section is operated with a relatively small manipulated variable when the operating section is operated with a relatively large manipulated variable for the moving operation of the controlled object.

When the surgical endoscope is a three-dimensional image acquisition device, the information about the photographing condition may be depth information calculated through analysis of image information taken by the endoscope for surgery.

Surgical endoscopy may be one or more of laparoscopic, thoracoscopic, arthroscopic, non-operative, cystoscopic, rectal, duodenal, mediastinal, and cardiac.

According to another aspect of the present invention, there is provided a method of adaptively controlling a control object, the control object being at least one of a surgical endoscope and a surgical tool performed in a master interface, The method comprising the steps of: storing a corresponding motion scale factor; determining whether the current image information currently provided from the surgical endoscope has been reduced or enlarged relative to the reference image; Or adjusting the moving scale factor so as to be non-linearly proportional to the moving scale factor of the control object, and converting the user command inputted for the moving operation of the control object to correspond to the reset moving scale factor to generate an operation signal of the control object A method of controlling an adaptive control object is provided.

In the determining step, it is possible to detect and match the area of the current image information that matches the current image information according to a predetermined image analyzing technique in order to determine whether the current image information is reduced or enlarged compared to the reference image.

The adjusting and resetting step may adjust the moving scale factor using the proportional relationship between the converted information, which is at least one of the horizontal size, the vertical size, the diagonal length and the width of the set area, and the information corresponding to the converted information in the reference image.

The moving distance of the control object according to the user command can be adjusted depending on the moving scale factor.

Surgical endoscopy may be one or more of laparoscopic, thoracoscopic, arthroscopic, non-operative, cystoscopic, rectal, duodenal, mediastinal, and cardiac.

According to another aspect of the present invention, there is provided a method of adaptively controlling a control object, which is at least one of a surgical endoscope and a surgical tool performed at a master interface, comprising: storing one or more moving scale factors according to an imaging condition of a surgical endoscope; A step of searching for a corresponding moving scale factor in a storage unit and setting the corresponding moving scale factor as a moving scale factor to be applied, and a step of receiving a user command inputted for a moving operation of the control object, And generating a manipulation signal for the control object by transforming the manipulation signal corresponding to the scale factor to provide an adaptive control method of the surgical tool.

The photographing conditions may be at least one of the image capturing zoom magnification, the photographing position, the distance from the surgical site surface, and the actual size of the surgical site to be photographed.

The moving scale factor in the case where the image capturing zoom magnification factor is high can be a value relatively smaller than the moving scale factor in the case where the image capturing zoom factor is low.

The moving scale factor when the surgical endoscope is relatively inserted into the human body may be smaller than the moving scale factor when the surgical endoscope is inserted relatively less.

The method further includes receiving distance information from the surface of the surgical site detected by the sensor, wherein the moving scale factor may be set to a relatively small value as the distance from the surface of the surgical site is closer.

The smaller the actual size of the surgical site to be photographed, the smaller the moving scale factor can be set to a relatively small value.

The moving distance of the control object according to the user command can be adjusted depending on the moving scale factor.

The moving scale factor can be computed in real time and stored in the storage unit according to the variation of the photographing condition.

The setting step can be set to a larger moving scale factor than when the operating section is operated with a relatively small manipulated variable when the user command is for a relatively large manipulated variable of the manipulating section for the manipulating manipulation of the controlled object.

When the surgical endoscope is a three-dimensional image acquisition device, the information about the photographing condition may be depth information calculated through analysis of image information taken by the endoscope for surgery.

Surgical endoscopy may be one or more of laparoscopic, thoracoscopic, arthroscopic, non-operative, cystoscopic, rectal, duodenal, mediastinal, and cardiac.

According to another aspect of the present invention, there is provided a master interface for controlling a position of a control object included in a surgical robot system to which a control object, which is at least one of a surgical endoscope and a surgical tool, And an operation signal generation unit for generating an operation signal for controlling the position of the corresponding control object in accordance with the selection information of the tool / vision selection input means in accordance with the manipulated variable of the manipulation unit Interface is provided.

The operation signal generating unit may determine whether or not the position control mode is selected and generate an operation signal for the position control only when the position control mode is selected.

According to the selection information of the tool / vision selection input means, the control object may be designated as at least one of a surgical endoscope and a surgical tool.

The tool / vision selection input means may be constituted by one or more buttons for specifying the control object.

When the control object is designated as the surgical endoscope and the surgical tool according to the selection information of the tool / vision selection input means, the operation signal generating unit maintains the alignment state of the surgical endoscope and the surgical tool in the direction corresponding to the operation direction of the operation unit, Can be generated.

When the control object is designated as a surgical endoscope and a surgical tool according to the selection information of the tool / vision selection input means, the surgical tool holding the arbitrary portion of the surgical site, the surgery in which the end is not taken by the surgical endoscope, When a certain fixed state button among the fixed state maintaining buttons provided on the operation tool, the surgical tool in a state where the jaws are closed, the surgical tool designated to keep the fixed state by the operator, and the plurality of operation parts, One or more of the correspondingly set surgical tools can maintain their position without moving regardless of the operation of the operating portion.

The amount of movement of the control object controlled to correspond to the manipulation amount of the manipulation portion can be determined depending on the motion scale factor.

A storage unit for storing a moving image scale factor corresponding to a reference image and at least one of image information provided from a surgical endoscope; and a storage unit for storing current image information currently provided from the surgical endoscope, Further comprising a scale factor arithmetic unit for adjusting and resetting the motion scale factor so as to linearly or nonlinearly proportional to a degree of reduction or enlargement of the motion scale factor, wherein the manipulation signal generator converts the manipulated variable of the manipulation unit into a corresponding motion scale factor So that an operation signal can be generated.

The scale factor operation unit can detect the current image information in the reference image in accordance with a preset image analysis technique to determine whether the current image information is reduced or enlarged relative to the reference image.

The scale factor operation unit can adjust the movement scale factor using the proportional relationship between the conversion information that is at least one of the horizontal size, the vertical size, the diagonal line length, and the width of the set region and the information corresponding to the conversion information in the reference image.

A storage unit for storing at least one moving scale factor according to a photographing condition of the endoscope for surgery; a storage unit for receiving information on photographing conditions of the endoscope for operation and searching for a corresponding moving scale factor in a storage unit, The operation signal generating unit may generate an operation signal by converting the operation amount of the operation unit to a value corresponding to the set moving scale factor.

The photographing conditions may be at least one of the image capturing zoom magnification, the photographing position, the distance from the surgical site surface, and the actual size of the surgical site to be photographed.

The moving scale factor in the case where the image capturing zoom magnification factor is high can be a value relatively smaller than the moving scale factor in the case where the image capturing zoom factor is low.

The moving scale factor when the surgical endoscope is relatively inserted into the human body may be smaller than the moving scale factor when the surgical endoscope is inserted relatively less.

The mobile scale factor may be set to a relatively small value as the distance from the surface of the surgical site is closer to the sensed value from a sensor provided to sense the distance to the surgical site surface.

The smaller the actual size of the surgical site to be photographed, the smaller the moving scale factor can be set to a relatively small value.

The moving scale factor can be computed in real time and stored in the storage unit according to the variation of the photographing condition.

The scale factor operating section can set a larger moving scale factor than when the operating section is operated with a relatively small manipulated variable when the operating section is operated with a relatively large manipulated variable for the moving operation of the surgical tool.

When the surgical endoscope is a three-dimensional image acquisition device, the information about the photographing condition may be depth information calculated through analysis of image information taken by the endoscope for surgery.

Surgical endoscopy may be one or more of laparoscopic, thoracoscopic, arthroscopic, non-operative, cystoscopic, rectal, duodenal, mediastinal, and cardiac.

According to another aspect of the present invention, there is provided a position control method for a control object, the control object including at least one of a surgical endoscope and a surgical tool performed at a master interface, the method comprising: receiving a user command for moving a control object using an operation unit; And generating an operation signal for controlling the position of the corresponding control object in accordance with the selection information of the tool / vision selection input means in accordance with the manipulation amount of the manipulation portion. / RTI >

According to the selection information of the tool / vision selection input means, the control object may be designated as at least one of a surgical endoscope and a surgical tool.

When the control object is designated as the surgical endoscope and the surgical tool according to the selection information of the tool / vision selection input means, the generating step maintains the alignment state of the surgical endoscope and the surgical tool in the direction corresponding to the operation direction of the operation portion, Can be generated.

When the control object is designated as a surgical endoscope and a surgical tool according to the selection information of the tool / vision selection input means, the surgical tool holding the arbitrary portion of the surgical site, the surgery in which the end is not taken by the surgical endoscope, When a certain fixed state button among the fixed state maintaining buttons provided on the operation tool, the surgical tool in a state where the jaws are closed, the surgical tool designated to keep the fixed state by the operator, and the plurality of operation parts, One or more of the correspondingly set surgical tools can maintain their position without moving regardless of the operation of the operating portion.

The amount of movement of the control object controlled to correspond to the manipulation amount of the manipulation portion can be determined depending on the motion scale factor.

Wherein the generating step includes the steps of: determining whether the current image information currently provided from the surgical endoscope has been reduced or enlarged relative to the reference image that is one of the image information provided by the surgical endoscope; Adjusting the moving scale factor so as to linearly or nonlinearly proportional to the degree of magnification or the degree of magnification, and generating an operation signal corresponding to the manipulated variable converted to match the reset moving scale factor A position control method for a control object is provided.

The current image information may be detected and regioned according to a predetermined image analysis technique by matching an area of the reference image that corresponds to the current image information.

Wherein the generating step includes the steps of: receiving information on an imaging condition of a surgical endoscope; extracting a moving scale factor corresponding to information provided from one or more moving scale factors according to an imaging condition stored in advance in a storage unit; And converting the manipulated variable of the manipulating portion into an equivalent to the extracted moving scale factor to generate an manipulation signal.

The photographing conditions may be at least one of the image capturing zoom magnification of the surgical endoscope, the photographing position, the distance from the surface of the surgical site, and the actual size of the surgical site photographed by the surgical endoscope.

The moving scale factor in the case where the image capturing zoom magnification factor is high can be a value relatively smaller than the moving scale factor in the case where the image capturing zoom factor is low.

The moving scale factor when the surgical endoscope is relatively inserted into the human body may be smaller than the moving scale factor when the surgical endoscope is inserted relatively less.

Receiving a sensing value from a sensor provided to sense a distance to a surgical site surface, wherein a moving scale factor may be set to a relatively small value as the distance from the surface of the surgical site is closer.

The smaller the actual size of the surgical site to be photographed, the smaller the moving scale factor can be set to a relatively small value.

The moving scale factor can be computed in real time and stored in the storage unit according to the variation of the photographing condition.

A larger moving scale factor can be set when the operating portion is operated with a relatively large manipulated variable for controlling the position of the control object, when the operating portion is operated with a relatively small manipulated variable.

When the surgical endoscope is a three-dimensional image acquisition device, the information about the photographing condition may be depth information calculated through analysis of image information taken by the endoscope for surgery.

According to another aspect of the present invention, there is provided a master interface for controlling a movement speed of a control object included in a surgical robot system to which a control object, which is at least one of a surgical endoscope and a surgical tool, An operation signal generation unit for generating an operation signal for controlling the movement speed of the corresponding control object in accordance with the selection information of the tool / vision selection input means in accordance with the operation amount of the operation unit; A master interface is provided.

The operation signal generation unit may determine whether or not the speed control mode is selected and generate an operation signal for controlling the movement speed only when the speed control mode is selected.

The movement speed of the control object can be set to a relatively large value as the operation amount of the operation portion is relatively large.

According to the selection information of the tool / vision selection input means, the control object may be designated as at least one of a surgical endoscope and a surgical tool.

The tool / vision selection input means may be constituted by one or more buttons for specifying the control object.

When the control object is designated as the surgical endoscope and the surgical tool according to the selection information of the tool / vision selection input means, the operation signal generating unit maintains the alignment state of the surgical endoscope and the surgical tool in the direction corresponding to the operation direction of the operation unit, Can be generated.

When the control object is designated as a surgical endoscope and a surgical tool according to the selection information of the tool / vision selection input means, the surgical tool holding the arbitrary portion of the surgical site, the surgery in which the end is not taken by the surgical endoscope, When a certain fixed state button among the fixed state maintaining buttons provided on the operation tool, the surgical tool in a state where the jaws are closed, the surgical tool designated to keep the fixed state by the operator, and the plurality of operation parts, One or more of the correspondingly set surgical tools can maintain their position without moving regardless of the operation of the operating portion.

The moving speed of the control object controlled to correspond to the manipulated variable of the manipulation portion can be determined depending on the motion scale factor.

A storage unit for storing a moving image scale factor corresponding to a reference image and at least one of image information provided from a surgical endoscope; and a storage unit for storing current image information currently provided from the surgical endoscope, Further comprising a scale factor arithmetic unit for adjusting and resetting the motion scale factor so as to linearly or nonlinearly proportional to a degree of reduction or enlargement of the motion scale factor, wherein the manipulation signal generator converts the manipulated variable of the manipulation unit into a corresponding motion scale factor So that an operation signal can be generated.

The scale factor operation unit can detect the current image information in the reference image in accordance with a preset image analysis technique to determine whether the current image information is reduced or enlarged relative to the reference image.

The scale factor operation unit can adjust the movement scale factor using the proportional relationship between the conversion information that is at least one of the horizontal size, the vertical size, the diagonal line length, and the width of the set region and the information corresponding to the conversion information in the reference image.

A storage unit for storing at least one moving scale factor according to a photographing condition of the endoscope for surgery; a storage unit for receiving information on photographing conditions of the endoscope for operation and searching for a corresponding moving scale factor in a storage unit, The operation signal generating unit may generate an operation signal by converting the operation amount of the operation unit to a value corresponding to the set moving scale factor.

The photographing conditions may be at least one of the image capturing zoom magnification, the photographing position, the distance from the surgical site surface, and the actual size of the surgical site to be photographed.

The moving scale factor in the case where the image capturing zoom magnification factor is high can be a value relatively smaller than the moving scale factor in the case where the image capturing zoom factor is low.

The moving scale factor when the surgical endoscope is relatively inserted into the human body may be smaller than the moving scale factor when the surgical endoscope is inserted relatively less.

The mobile scale factor may be set to a relatively small value as the distance from the surface of the surgical site is closer to the sensed value from a sensor provided to sense the distance to the surgical site surface.

The smaller the actual size of the surgical site to be photographed, the smaller the moving scale factor can be set to a relatively small value.

The moving scale factor can be computed in real time and stored in the storage unit according to the variation of the photographing condition.

The scale factor operating section can set a larger moving scale factor than when the operating section is operated with a relatively small manipulated variable when the operating section is operated with a relatively large manipulated variable for the moving operation of the surgical tool.

When the surgical endoscope is a three-dimensional image acquisition device, the information about the photographing condition may be depth information calculated through analysis of image information taken by the endoscope for surgery.

Surgical endoscopy may be one or more of laparoscopic, thoracoscopic, arthroscopic, non-operative, cystoscopic, rectal, duodenal, mediastinal, and cardiac.

According to another aspect of the present invention, there is provided a method for controlling a movement speed of a control object performed in a surgical robot system, the control object being mounted on at least one of a surgical endoscope and a surgical tool, A step of receiving a user command and a step of generating an operation signal for controlling the movement speed of the corresponding control object according to the selection information of the instrument / A control method for controlling the moving speed of the control object is provided.

In the generating step, the movement speed of the control object can be set to a relatively large value as the manipulation amount of the manipulation portion is relatively large.

According to the selection information of the tool / vision selection input means, the control object may be designated as at least one of a surgical endoscope and a surgical tool.

 When the control object is designated as the surgical endoscope and the surgical tool according to the selection information of the tool / vision selection input means, the generating step maintains the alignment state of the surgical endoscope and the surgical tool in the direction corresponding to the operation direction of the operation unit It is possible to generate an operation signal to be moved.

When the control object is designated as a surgical endoscope and a surgical tool according to the selection information of the tool / vision selection input means, the surgical tool holding the arbitrary portion of the surgical site, the surgery in which the end is not taken by the surgical endoscope, When a certain fixed state button among the fixed state maintaining buttons provided on the operation tool, the surgical tool in a state where the jaws are closed, the surgical tool designated to keep the fixed state by the operator, and the plurality of operation parts, One or more of the correspondingly set surgical tools can maintain their position without moving regardless of the operation of the operating portion.

The moving speed of the control object controlled to correspond to the manipulated variable of the manipulation portion can be determined depending on the motion scale factor.

Wherein the generating step includes the steps of: determining whether the current image information currently provided from the surgical endoscope has been reduced or enlarged relative to the reference image that is one of the image information provided by the surgical endoscope; Adjusting the moving scale factor so as to linearly or nonlinearly proportional to the degree of magnification or the degree of magnification, and generating an operation signal corresponding to the manipulated variable converted to match the reset moving scale factor A position control method for a control object is provided.

The current image information may be detected and regioned according to a predetermined image analysis technique by matching an area of the reference image that corresponds to the current image information.

Wherein the generating step includes the steps of: receiving information on an imaging condition of a surgical endoscope; extracting a moving scale factor corresponding to information provided from one or more moving scale factors according to an imaging condition stored in advance in a storage unit; And converting the manipulated variable of the manipulating portion into an equivalent to the extracted moving scale factor to generate an manipulation signal.

The photographing conditions may be at least one of the image capturing zoom magnification of the surgical endoscope, the photographing position, the distance from the surface of the surgical site, and the actual size of the surgical site photographed by the surgical endoscope.

The moving scale factor in the case where the image capturing zoom magnification factor is high can be a value relatively smaller than the moving scale factor in the case where the image capturing zoom factor is low.

The moving scale factor when the surgical endoscope is relatively inserted into the human body may be smaller than the moving scale factor when the surgical endoscope is inserted relatively less.

Receiving a sensing value from a sensor provided to sense a distance to a surgical site surface, wherein a moving scale factor may be set to a relatively small value as the distance from the surface of the surgical site is closer.

The smaller the actual size of the surgical site to be photographed, the smaller the moving scale factor can be set to a relatively small value.

The moving scale factor can be computed in real time and stored in the storage unit according to the variation of the photographing condition.

A larger moving scale factor can be set when the operating portion is operated with a relatively large manipulated variable for controlling the position of the control object, when the operating portion is operated with a relatively small manipulated variable.

When the surgical endoscope is a three-dimensional image acquisition device, the information about the photographing condition may be depth information calculated through analysis of image information taken by the endoscope for surgery.

Other aspects, features, and advantages will become apparent from the following drawings, claims, and detailed description of the invention.

According to the embodiment of the present invention, there is an effect that the moving scale factor of the surgical tool can be adaptively changed according to the area and the range of the object of the image information photographed through the laparoscope.

Further, it is possible to shorten the operation time and reduce the fatigue of the operator during the operation progress, and also to intuitively recognize the moving distance of the surgical tool based on the displayed image information.

In addition, there is also an effect of maximizing the operation convenience of the laparoscope and / or the surgical tool.

Further, there is an effect that a new operation can be realized by improving the camera button used for laparoscopic operation and adding a separate position control / speed control button.

1 is a plan view showing the entire structure of a surgical robot system according to the prior art;
2 is a conceptual diagram showing a master interface of a surgical robot system according to the prior art;
FIG. 3A and FIG. 3B are conceptual views illustrating a movement control process of an instrument included in the surgical robot system according to the related art.
FIG. 4A is a block diagram schematically showing a configuration of a surgical robot system according to an embodiment of the present invention; FIG.
4B and 4C illustrate a concept of applying a fixed MSF (Motion Scale Factor) according to an embodiment of the present invention.
Figures 4D and 4E illustrate an adaptive MSF application concept in accordance with an embodiment of the present invention.
5A and 5B illustrate the concept of adaptive adjustment of a moving scale factor according to an embodiment of the present invention.
6 is a flowchart illustrating a method of adaptively adjusting a moving scale factor according to an embodiment of the present invention.
FIG. 7 is a flowchart illustrating a method of determining a position control mode of a surgical robot system according to an embodiment of the present invention. FIG.
8 is a flowchart illustrating a method for determining a speed control mode of a surgical robot system according to another embodiment of the present invention.

The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated and described in the drawings. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Referring to the accompanying drawings, the same or corresponding components are denoted by the same reference numerals, .

FIG. 4A is a block diagram schematically showing the configuration of a surgical robot system according to an embodiment of the present invention. FIG. 4B and FIG. 4C are views showing a fixed MSF (Motion Scale Factor) applied according to an embodiment of the present invention FIGS. 4D and 4E are views showing an adaptive MSF application concept according to an embodiment of the present invention, and FIGS. 5A and 5B are views showing adaptation of a mobile scale factor according to an embodiment of the present invention FIG.

4A, the surgical robot system may include a master robot 1 and a slave robot 2. The master robot 1 and the slave robot 2 may be implemented independently or integrally, .

The master robot 1 includes an image input unit 410, a screen display unit 420, a dark operation unit 430, an operation signal generation unit 450, a scale factor operation unit 440 and a control unit 460. Although not shown, the master robot 1 may further include a storage unit. The slave robot 2 includes a robot arm 3 and a laparoscope 5. [

The image input unit 410 receives an image input through a camera provided in the laparoscope 5 of the slave robot 2 via a wired or wireless communication network.

The screen display unit 420 outputs visual images corresponding to images received through the image input unit 410 as visual information. The screen display unit 420 may be realized in the form of a monitor 6 or the like and an image processing process for causing the received image to be output as an image image through the screen display unit 420 may be performed by the control unit 460 or the image processing unit ). ≪ / RTI >

The arm control section 430 is a means for enabling an operator to operate the position and function of the robot arm 3 and / or the laparoscope 5 of the slave robot 2. [ The arm control unit 430 may be formed in the shape of a handle as illustrated in FIG. 2. However, the shape is not limited thereto, and may be modified into various shapes to achieve the same purpose. Further, for example, a part may be formed in a handle shape, and the other part may be formed in another shape such as a clutch button or the like. For example, when the operator manipulates the arm control part 430 in an arbitrary direction while holding the arm control part 430, the operator can manipulate the surgical tool to move in that direction. The dark operation unit 430 may be tilted in three-dimensional space, for example, but the operation mode may vary depending on the implementation of the dark operation unit 430. [ That is, in the present specification, the arm control unit 430 having a structure in which one end is fixed and the other end is tilted in an arbitrary direction and an operation command is inputted for the operation of the surgical tool and / or the laparoscope 5 And the arm operation unit 430 for receiving an operation command of the operator may be various, and the manner of inputting the operation command according to the adopted structure may be various.

As described above, the master interface 4 of the master robot 1 may include one or more function buttons, and each function button may be implemented as a mechanical button / switch having a physical entity, ) And can be implemented as software-like buttons / switches that are operated by the operator's touch.

The function buttons included in the master interface 4 may include at least one of a master repositioning button, a tool / vision selection button, and a position control / speed control mode button, for example.

The Master Reposition (MR) button is a button for allowing the surgical robot system to operate in the master repositioning mode in order to prevent an operation command by the operator using the master robot 1 from being transmitted to the slave robot 2 to be. The operator can newly set the starting point for the position / speed control in the position control / speed control mode, which will be described later, by returning the arm control portion 430 to the original position by using the master repositioning button.

The Instrument / Vision Selection (IVS) button is used to select one or more of the laparoscope 5 and the surgical instrument according to the operation result when the operator performs an operation using the arm control unit 430 or the like of the master robot 1 Is operated. This is an extension of the use and function of the camera button applied to the surgical robot system according to the prior art. The operator uses a tool / vision selection button to select a tool control mode in which the surgical tool moves according to the operation of the arm control unit 430, An operation mode that is one or more of a vision control mode in which the laparoscope 5 moves according to an operation of the operation unit 430 and an integrated control mode in which both the surgical tool and the laparoscope 5 are moved in accordance with the operation of the arm operation unit 430 can be designated.

As described above, for example, the IVS button may be composed of a vision control button and an integrated control button, in order to enable an arbitrary operation mode selection among a plurality of operation modes using the IVS button. In this case, it is possible to drive in the tool control mode in the state where no button is pressed, in the vision control mode in the state in which only the vision control button is pressed, and in the integrated control mode in the state in which only the integrated control button is pressed . At this time, when both of the two buttons are pressed, it is possible to drive in the hold mode described above.

Of course, the IVS button may also be implemented as a single button. For example, the IVS button can be configured to be rotated so as to be able to select any operation mode among a plurality of operation modes. As another example, the IVS button may be formed in a pushable structure. When the button is not pressed, the IVS button is driven in the tool control mode. When the pushing state is maintained, the IVS button is driven in thevision control mode. It may be implemented so that the integrated control mode is terminated when it is further pressed one more time.

It is a matter of course that information on the operation state of the IVS button, that is, the operation mode, can be displayed by the operator through the screen display unit 420.

As described above, the implementation mode of the button and the operation mode of the button may be various for specifying the operation mode of the surgical robot system, and the same concept can be applied to the position control / speed control mode button described below.

The Position / Velocity Mode (PVM) button is used to determine whether the operator uses the arm control unit 430 of the master robot 1 to operate in the position control mode or in the speed control mode Button. Here, the position control mode is an operation mode for allowing the position of the surgical tool or the laparoscope to be moved (for example, operation of a surgical tool or the like) according to the operation of the arm control unit 430 or the like, ) To change the speed at which the surgical tool or the laparoscope moves (which may further include the direction).

When the PVM button is selected, the surgical robot system operates in the position control mode or the speed control mode, and Table 1 summarizes the operation target in each operation mode.

Controlled object Position control mode
( PVM  Button: select position control) Speed control mode
( PVM  Button: select speed control) Surgical tool
Control - Position control mode of surgical instruments
- IVS button: Select tool control - Movement speed control mode of surgical tool
- IVS button: Select tool control Laparoscopic control - Laparoscopic position control mode
- IVS button: Vision control selection - Laparoscopic movement speed control mode
- IVS button: Vision control selection Integrated control - Surgical tools and laparoscopic position control mode
- IVS button: select integrated control - Surgical tool and laparoscopic movement speed control mode
- IVS button: select integrated control

Hereinafter, the concept of adaptive adjustment of the surgical robot system and the moving scale factor will be described first with reference to FIGS. 4A to 5B, and thereafter, the position control mode and the speed control mode will be specifically described with reference to FIGS. 7 and 8 .

The operation signal generating unit 450 may control the operating mode of the surgical robot system and the operation mode of the surgical robot system when the operator operates the arm manipulating unit 430 for positional movement of the robot arm 3 and / An operation signal corresponding to the operation state of the operation unit 430 is generated and transmitted to the slave robot 2. [ At this time, the operation signal generating unit 450 can generate an operation signal corresponding to the operation mode (for example, the position control mode, the speed control mode, and the like) selected by the operator under the control of the controller 460. [ The generated operation signal can be transmitted and received through a wired or wireless communication network as described above.

When the scale factor computing unit 440 updates and sets the moving scale factor as described later, the operation signal generating unit 450 generates an operation signal corresponding to the moving scale factor and transmits the operation signal to the slave robot 2.

The scale factor calculator 440 recognizes the size and / or the area of the subject photographed by the laparoscope 5 and recognizes the size of the laparoscope 5 and / And the motion scale factor (MSF) so that the surgical tool can be manipulated. For example, the operation relationship between the movement of the surgical tool and the arm operation unit 430 can be defined as "movement amount of the surgical tool = k x MSF x operation amount of the arm operation unit ". Here, k is a proportional constant. This manipulation relationship can be similarly applied to the manipulated variable of the arm control section 430 and the amount of motion of the laparoscope 5. [

The MSF adjusted by the scale factor calculator 440 may be arbitrarily selected by the operator from fixed values, but can be adaptively adjusted according to a surgical situation.

As illustrated in FIGS. 4B and 4C, if one of the preset and fixed modes (i.e., the normal mode, the fine mode, and the ultra-fine mode) is selected, a predetermined MSF may be applied according to the selected mode. For example, in the normal mode, when the MSF, which is the ratio of the master to the slave, is set to 1: 1, when the slave (i.e., the surgical tool and / or the laparoscope) is moved by 1 using the arm control unit 430, 1 < / RTI > Referring to the illustrated example, in the fine mode, the master must be moved by three to operate the slave by one, and in the ultra-fine mode, the master must move by five to operate the slave by one.

The MSF may be set in advance for each mode as described above by applying the MSF to a fixed value. However, the MSF may be preset for a plurality of selection values (for example, 1: 1, 3: 1, 5: 1, 7: 1, 10: 1, etc.).

As described above, the MSF can be fixed and applied by the master or slave (for example, the manipulated amount of the female manipulation part, the insertion degree of the laparoscope, the distance to the affected part, and the like) (see FIG.

Also, as described above, the MSF can be adapted to adaptive MSF according to the surgical situation (e.g., the current situation of the master and / or the slave, etc.).

The MSF may be adaptively adjusted according to the characteristics of the image output through the screen display unit 420 (for example, the actual size of the image displayed on the screen) when the MSF is adaptively adjusted. For example, if the laparoscope 5 is positioned far away from the affected area (i.e., the surgical site) or zoomed out, the image on the screen display 420 will be relatively less detailed but wider , In which case the MSF can be adaptively adjusted to have a relatively large value. In contrast, when the laparoscope 5 is positioned close to the affected area (i.e., the surgical site) or zoomed in, the image on the relatively small but narrow area will be displayed on the screen display unit 420, The MSF can be adaptively adjusted to have a relatively small value.

Alternatively, as illustrated in FIG. 4D, a slave-dependent adaptive scaling control (SASC) scheme may be applied as a method for determining an MSF according to a situation where a slave is located. In this case, the MSF may be expressed as a function of a slave state . For example, it is possible to determine whether the image is enlarged or reduced by the image analysis, change the MSF accordingly, change the MSF according to how much the laparoscope 5 is inserted, Dimensional laparoscope 5 can measure the distance to the affected part by the image processing and change the MSF according to the measured depth. , And changing the MSF according to the actual size of the area displayed through the screen display unit 420 (see FIGS. 5A and 5B). Each case will be described later in detail with reference to Figs. 5A and 5B.

As described above, the MSF may be adaptively adjusted without being fixed by the master or slave situation (for example, the manipulated amount of the female manipulation part, the insertion degree of the laparoscope, and the distance to the affected part).

Similarly, as illustrated in FIG. 4E, a master-dependent adaptive scaling control (MASC) scheme may be applied as a method for determining an MSF according to a situation of a master, in which case the MSF may be expressed as a function of the master state .

For example, if the arm control portion 430 moves a little from the movement start position, the MSF is relatively large (that is, the slave moves slightly slowly), but moves away from the movement start position , So that the slave moves a lot faster). In this case, the movement start position can be reset when the above-described master repositioning button is pressed, and the MSF can be adjusted according to the movement amount as described above in the operation from the reset movement start position.

At this time, it is natural that the MSF may change linearly or nonlinearly. Also, the MSF may use a value calculated in real time, or a pre-stored value may be used.

Hereinafter, an adaptive adjustment method of the moving scale factor by the scale factor operator 440 will be briefly described with reference to FIGS. 5A and 5B. 5A and 5B show the current photographing region currently photographed by the laparoscope 5, and the reference numeral 520 designates a reference photographing region which can be photographed by the laparoscope 5. FIG. The actual size of the imaging region photographed by the laparoscope 5 may vary depending on factors such as the degree of proximity of the laparoscope 5 to the affected area, the degree of zoom in / out, and the like. The shooting area may match.

If the imaging magnification of the laparoscope 5 is set to a high magnification or the laparoscope 5 is moved so as to approach the surgical site, as shown in 510 and 5B in FIG. 5A, The region 510 may be smaller than the reference imaging region 520 in which the laparoscope 5 has taken the surgical site at a preset position and magnification.

However, if the imaging magnification of the laparoscope 5 is set at a low magnification or the laparoscope 5 is moved away from the surgical site, the current imaging area 510, which is taken by the laparoscope 5, Or greater.

5A and 5B, when the current photographing area 510 is smaller than the reference photographing area 520, it is common that the operator performs a detailed operation in a narrow area. Therefore, The movement scale of the surgical tool according to the present invention needs to be finely adjusted. On the contrary, when the current photographing area 510 is coincident with or larger than the reference photographing area 520, it is common that the operator observes the entire operation part or moves the operation area to be operated to another part. The manipulation scale of the surgical instrument according to the manipulation of the manipulator should be adjusted to a relatively large extent so that the operator can manipulate the surgical tool intuitively.

There may be a variety of methods for adaptively adjusting the moving scale factor MSF according to the size of the subject photographed by the laparoscope 5 and the content of the image information displayed through the screen display unit 420 correspondingly.

Some of the methods of adaptively adjusting the mobile scale factor will be described as follows. Of course, it should be noted that the adaptive adjustment method of the moving scale factor is not limited to only those exemplified below, and the scope of the present invention is not limited by the following exemplary explanations.

5A and 5B, the scale factor calculator 440 calculates the scale factor using the reference image stored in the storage unit, and calculates the scale factor calculator 440 based on the size of the reference photographing area 520 and the size difference between the current photographing area 510 The moving scale factor can be adjusted to be proportional.

5A, assuming that the image information photographed by the reference photographing region 520 and displayed on the display device is for a lesion having an actual size of 12 cm in height, the window size at this time is defined as 1, The mobile scale factor can also be defined as 1. In this situation, when the operator operates the arm control unit 430 in an arbitrary direction, the surgical tool is moved in the direction in which the arm control unit 430 is operated from the center point of the reference photographing area 520.

However, as illustrated in Figs. 5A and 5B, when the laparoscope 5 is moved to the actual area of the current photographing area 510 as a partial area of the reference photographing area 520 by an approach to or zoom- In the case where the affected part having a length of 4 cm is photographed, the window size at this time is interpreted as 0.33 (i.e., 4 cm / 12 cm), and the moving scale factor can be lowered (for example, 0.33). In this case, in the display area of the screen display part 420, the current photographing area 510 is displayed in its entirety (that is, a part of the entire surgical area is enlarged and displayed finely), but the moving scale factor is lowered, have.

For example, the image information photographed in the reference photographing area 520 is stored as a reference image in the storage unit, and then the magnification The image information of the current photographing area 510 in accordance with the adjustment and / or the movement of the laparoscope 5 in the lengthwise direction (the axial direction of the shaft) is determined by the image analysis technique (for example, An image analysis technique for detecting and zoning the same area in two images).

In the above-described example, it is assumed that the reference for adjusting the moving scale factor is a vertical length. However, the adjustment standard of the moving scale factor may vary depending on, for example, the diagonal length, the width, and the width.

As another example, the scale factor calculator 440 may adjust the moving scale factor using the imaging magnification, the positional shift, the distance from the surface of the surgical site, etc. of the laparoscope 5.

That is, if the image capturing magnification of the laparoscope 5 is set to a high magnification for capturing a narrow range, the scale factor calculator 440 lowers the moving scale factor so that the operator can finely manipulate the surgical tool, The moving scale factor can be adjusted in such a manner that the moving scale factor is increased. The moving scale factor for each magnification value that can be set in the laparoscope 5 can be calculated experimentally and statistically in advance and stored in advance in the storage unit.

The scale factor calculator 440 may calculate the scale factor calculator 440 in a manner that the laparoscope 5 is relatively long in the longitudinal direction (for example, inserted or drawn into the human body) relative to the reference position (for example, You can adjust the scale factor depending on whether you are moving. Information about the position of the laparoscope 5 can be directly calculated by the master robot 1 or can be provided from the slave robot 2 and the reference position can be obtained, for example, after the laparoscope 5 is inserted into the body, Or may be arbitrarily designated by the operator.

If the laparoscope 5 is further inserted into the human body (i.e., closer to the surgical site), the mobile scale factor is lowered so that the surgical tool can be finely manipulated, and the laparoscope 5 is moved in the direction , Away from the surgical site), the mobile scale factor can be increased. The moving scale factor according to the moving distance of the laparoscope 5 in the longitudinal direction from the reference position can be previously calculated statistically and stored in advance in the storage unit.

The scale factor calculator 440 may adjust the scale factor using the distance value from the distance sensor provided in the laparoscope 5 to the surface of the surgical site. If the distance from the laparoscope (5) to the surface of the surgical site is close, lower the moving scale factor to allow more precise manipulation of the surgical instrument. If the distance from the laparoscope (5) . The moving scale factor according to the distance from the laparoscope 5 to the surface of the surgical site can be previously calculated statistically and stored in advance in the storage unit.

When the image displayed through the screen display unit 420 is the image provided by the laparoscope 5, which is a three-dimensional image acquiring apparatus, the depth can be calculated through image processing. Using this value, May be calculated.

Referring again to FIG. 4A, the controller 460 controls the operation of each component so that the above-described functions can be performed. The control unit 460 may convert the image input through the image input unit 410 into an image to be displayed through the screen display unit 420. [ In addition, the controller 460 may perform processing for enabling the return operation of the handle 10 without moving the position of the laparoscope 5 and the surgical tool when the master repositioning button is input by the operator.

Hereinafter, an embodiment of the adaptive adjustment method of the moving scale factor (MSF) will be described with reference to the related drawings, and the position control mode and the speed control mode will be described with reference to the related drawings, respectively.

Although not separately described in each description process, it is of course possible to apply the concept of the moving scale factor (MSF) described above.

In the fixed MSF position control mode, if the master (for example, the arm control unit) moves 1 cm, the slave moves 1 cm (for example, in the normal mode) or the slave moves 1 cm if the master moves 3 cm For example, Fine mode).

On the other hand, in the adaptive MSF position control mode, the MSF may be linear or non-linear depending on the degree of enlargement, reduction, insertion of the endoscope, distance between the end of the endoscope and the affected part, The movement of the slave is determined accordingly.

Likewise, in the fixed MSF speed control mode, the slave can be controlled to move at 1 cm / sec when the master moves 1 cm from the starting point and the slave moves at 5 cm / sec when the master moves 5 cm from the starting point, regardless of the status of the master or slave.

In contrast, in the adaptive MSF speed control mode, the slave moves at a rate of 0.5 cm / sec when the master moves 1 cm from the starting point, when the screen is enlarged, when the endoscope is inserted deeply, The slave can move at 1.5cm / sec when moving, when the screen is reduced, when the endoscope is inserted shallowly, or when the distance from the affected part is long. That is, the MSF varies linearly or nonlinearly depending on the state of the master or the slave, and the amount of change in which the speed changes according to the motion of the master may adaptively change according to the MSF.

6 is a flowchart illustrating a method of adaptively adjusting a moving scale factor according to an embodiment of the present invention.

Referring to FIG. 6, in step 610, the master robot 1 stores the image information photographed by the laparoscope 5 as a reference image for adaptive adjustment of the moving scale factor.

In step 620, the master robot 1 determines whether a reference image reset request is input.

If a reference image reset request has been input, the flow advances to step 630 to reset the reference image for the surgical site.

If the reference image reset request is not inputted by the judgment of the step 620 or if the reference image is reset by the step 630, the process goes to the step 640.

In step 640, the master robot 1 determines whether a reduction request for the imaging area photographed by the laparoscope 5 is input by operating the operator's manipulation part 430. For example, the laparoscope 5 approaches the affected area, or instructs to take a picture of a narrower area through a zoom-in process or the like. Through the screen display unit 420, An image (i.e., an enlarged image) for the area is displayed.

If the shooting area reduction request is not input, the process proceeds to step 660.

However, if a shooting area reduction request has been input, the master robot 1 proceeds to step 650 to reset the moving scale factor.

That is, since the image taken by the laparoscope 5 is an enlarged image of a certain region of the entire operation region, the movement scale factor becomes smaller in order to perform detailed manipulation of the surgical tool. At this time, the master robot 1 determines which region and which size of the image information of the current photographing region 510 of the laparoscope 5 according to the request for reduction of the photographing region within the reference image set in Step 610 or Step 630, (For example, an image analysis technique for detecting and zoning the same area in two images), thereby determining to what level the moving scale factor is to be adjusted.

In step 660, the master robot 1 determines whether an enlargement request for the image capturing area of the image taken by the laparoscope 5 is input by operating the operator's manipulation part 430. For example, the enlargement request for the imaging area is for instructing the laparoscope 5 to move away from the affected area, or to take a picture of a wider area through a zoom-out process or the like. An image for a large area (i.e., a reduced image) is displayed.

If the photographing area enlargement request is not input, the process returns to step 620. [

However, if it is determined in step 660 that an enlargement request for the imaging area has been input, the master robot 1 proceeds to step 670. In step 670, the master robot 1 determines a moving scale for the current imaging area 510, Reset the factor.

6, an adaptive adjustment method of the moving scale factor using the reference image has been described. However, as described above, in order to adaptively adjust the moving scale factor, the imaging magnification, And the like can be used independently or additionally.

7 is a flowchart illustrating a method for determining a position control mode of a surgical robot system according to an embodiment of the present invention.

As described above, the operator uses the position control / speed control mode (PVM) button provided in the master robot 1 to determine whether the surgical robot system is to be driven in the position control mode or the speed control mode Can be specified. In addition, the operator can determine whether the object manipulated by the manipulation of the arm manipulation part 430 is a surgical tool, a laparoscope, or both a surgical tool and a laparoscope using a tool / vision selection (IVS) button.

In addition, as described above, the PVM button and the IVS button may be constituted by a plurality of buttons whose functions are individually designated, or may be a single button implemented so that a driving mode can be selected by various operating modes. Of course, the implementation of PVM and IVS buttons will be different.

The position control mode of the surgical robot system according to the present embodiment includes a position control mode for controlling the position of the surgical tool, a laparoscopic position control mode for controlling the position of the laparoscope (5), and an integrated position control of the surgical tool and the laparoscope An integrated position control mode may be included. Hereinafter, a method of determining the position control mode of the surgical robot system will be described with reference to FIG.

Referring to FIG. 7, in step 705, the arm control unit 430 is operated by the operator.

In step 710, the control unit 460 determines whether the position control / speed control mode (PVM) button is selected so that the surgical robot system is operated in the position control mode. The position control / speed control mode (PVM) button is used to designate whether the operation using the arm control unit 430 or the like of the master robot 1 functions in the position control mode or the speed control mode.

As a result of the determination in step 710, if the PVM button has been selected to operate in the speed control mode, the surgical robot system proceeds to step 810 of FIG.

However, if the PVM button has been selected as the position control mode for position control of at least one of the surgical tool and the laparoscope 5,

In step 720, the control unit 460 determines whether the tool / vision selection (IVS) button has been selected so that the surgical robot system is operated in the tool control mode. The tool / vision selection (IVS) button is used to allow the operator to operate one or more of the laparoscope 5 and the surgical tool in accordance with the operation result when performing an operation using the dark operation unit 430 or the like of the master robot 1 Button.

As a result of the determination in step 720, if the IVS button is selected to operate in the tool control mode, the operation signal generating unit 450 proceeds to step 730, where the operation signal generating unit 450 generates an operation command Generates an operation signal to be applied to the position control of the surgical tool, and transmits it to the slave robot (2). At this time, the generated operation signal can be generated by applying the fixed or adaptive moving scale factor as described above.

The surgical tool provided to the slave robot 2 is controlled to move in accordance with the operation of the arm control unit 430 by the operation signal transmitted to the slave robot 2 in step 730. [

However, if it is determined in step 720 that the IVS button is not selected to operate in the tool control mode, the process proceeds to step 740.

In step 740, the control unit 460 determines whether the tool / vision selection (IVS) button has been selected so that the surgical robot system is operated in the vision control mode.

As a result of the determination in step 740, if the IVS button is selected to operate in the vision control mode, the operation signal generator 450 proceeds to step 750. In step 750, the operation signal generator 450 generates an operation command Generates an operation signal to be applied to the position control of the laparoscope 5, and transmits it to the slave robot 2. At this time, the generated operation signal can be generated by applying the fixed or adaptive moving scale factor as described above.

The laparoscope 5 provided in the slave robot 2 will be controlled so as to correspond to the operation of the arm control unit 430 by the operation signal transmitted to the slave robot 2 by the step 750.

However, if it is determined in step 740 that the IVS button is not selected to operate in the vision control mode, the process proceeds to step 760.

In step 760, the control unit 460 determines whether the tool / vision selection (IVS) button has been selected so that the surgical robot system is operated in the integrated control mode.

As a result of the determination in step 760, if the IVS button is selected to operate in the integrated control mode, the operation signal generation unit 450 proceeds to step 770, where the operation tool 430 and the laparoscope 5 ) Is generated and transmitted to the slave robot.

Here, the operation signal is set so that the surgical tool and the laparoscope 5 move together in the direction in which the arm control part 430 is operated, and the aligned states (for example, mutual relative positions, etc.) . That is, even if the physical position of the imaging region is changed in accordance with the movement of the laparoscope 5, the relative position of the surgical instrument with respect to the center of the imaging region on the display screen confirmed by the surgeon through the screen display unit 420, Lt; / RTI > The operator can terminate the interlocking movement of the laparoscope 5 and the surgical tool by, for example, selecting a master repositioning button, switching the operation mode to another mode, and the like.

The operation state of the integrated position control mode will be briefly described as follows, for example.

When the operator selects the PVM button in the position control mode for the purpose of changing the shooting area 510 and the IVS button is selected in the integrated control mode, the operation signal generating unit 450 generates the operation signal in the direction in which the arm control unit 430 is operated And transmits an operation signal to the slave robot 2 to move the laparoscope 5 and the surgical tool together.

At this time, although the alignment state between the laparoscope 5 and the surgical tool is maintained and moved, a state in which any one of the surgical tools holds something (for example, 5b), in order to ensure the safety of the surgical patients, it is natural that the surgical instruments do not move interlockedly with the laparoscope (5) and can wait at the position.

When referring to the image information displayed through the screen display section 420, a surgical tool whose end is not displayed in the display screen (that is, a surgical tool whose end is located outside the imaging range photographed by the laparoscope 5) It is of course possible to wait at the corresponding position without moving interlockingly with the laparoscope 5.

Further, when the jaws of the surgical tool are closed, the surgical tool may be controlled so as not to be moved for the safety of the patient.

In addition, it is natural that the method for controlling the movement of any surgical instrument may be more various. For example, a surgical tool directly designated by the operator can be prevented from moving, and a specific button (for example, a master repositioning button may be located in each of the arm control portions 430) is provided in the left and right lock operation portions 430 So that the surgical tool to which the button is pushed can be prevented from moving when the corresponding button is pressed.

However, if it is determined in step 760 that the IVS button is not selected to operate in the integrated control mode, the operation signal generation unit 450 proceeds to step 780 to determine whether the operation tool and / or the laparoscope 5) is operated and transmitted to the slave robot. For example, in the hold mode or the master repositioning mode, an operation signal is generated and transmitted to the slave robot so that the surgical tool and / or the laparoscope 5 can maintain the last control position regardless of the operation of the master.

8 is a flowchart illustrating a method for determining a speed control mode of a surgical robot system according to another embodiment of the present invention.

As described above, the operator can designate the operation mode of the surgical robot system as the position control mode or the speed control mode by using the position control / speed control mode (PVM) button provided in the master robot 1. In addition, the operator can determine, using at least one of the tool / vision selection (IVS) buttons, one or more of the surgical tool and the laparoscope to be manipulated by manipulation of the arm manipulation part 430. [

The speed control mode of the surgical robot system according to the present embodiment includes a speed control mode for controlling the moving speed of the surgical tool, a laparoscopic speed control mode for controlling the moving speed of the laparoscope 5, An integrated speed control mode may be included. Hereinafter, a method for determining the speed control mode of the surgical robot system will be described with reference to FIG.

As described above with reference to FIG. 7, in step 710, the control unit 460 determines whether the position control / speed control mode (PVM) button is selected so that the surgical robot system is operated in the position control mode, If the surgical robot system is selected to operate in the speed control mode, the process proceeds to step 810 of FIG.

8, in step 810, the control unit 460 determines whether the tool / vision selection (IVS) button has been selected so that the surgical robot system is operated in the tool control mode.

As a result of the determination in step 810, if the IVS button is selected to operate in the tool control mode, the operation signal generator 450 proceeds to step 820, Generates an operation signal for applying an operation command according to the operation to the speed control of the surgical tool, and transmits the operation signal to the slave robot 2. The surgical tool provided to the slave robot 2 will be controlled so as to correspond to the operation of the arm control unit 430 by the operation signal transmitted to the slave robot 2 by the step 820. [

At this time, the generated operation signal can be generated by applying the fixed or adaptive moving scale factor as described above.

In addition, the operation signal may be generated such that the speed at which the surgical tool moves varies depending on how far the arm control unit 430 has moved from the starting point. That is, when the arm control part 430 moves a little in a specific direction (that is, when the operation state of the arm control part 430 is maintained after a slight movement), the surgical tool continuously and slowly moves in that direction, If the user moves a lot in the corresponding direction (that is, if the operation state of the arm operation unit 430 is maintained after the large movement), the surgical tool can be controlled to move at a high speed continuously. For example, when the arm control section 430 is formed as shown in FIG. 2, the arm control section 430 may control the moving speed of the arm control section 430 so that the arm control section 430 is moved more than the moving speed of the surgical tool when the arm control section 430 is operated by 5 degrees in an arbitrary direction So that the moving speed of the surgical tool can be increased.

The information on the moving speed of the surgical tool corresponding to the manipulated variable of the arm control unit 430 may be previously set in advance in the master interface 4, And may be stored in a storage unit.

Of course, when the arm control portion 430 returns to the starting point, the movement of the surgical tool described above will be stopped. In the speed control mode, when the arm control unit 430 returns to the start position, the speed is set to 0 (i.e., the motion is stopped). However, when the arm control unit 430 returns to the start position, do.

However, if it is determined in step 810 that the IVS button is not selected to operate in the tool control mode, the process proceeds to step 830.

In step 830, the control unit 460 determines whether the tool / vision selection (IVS) button has been selected so that the surgical robot system is operated in the vision control mode.

As a result of the determination in step 830, if the IVS button is selected to operate in the vision control mode, the operation signal generation unit 450 proceeds to step 840 to determine whether the operation instruction according to the operation of the arm operation unit 430 is a laparoscope 5 And transmits the generated operation signal to the slave robot 2. At this time, the generated operation signal can be generated by applying the fixed or adaptive moving scale factor as described above. The laparoscope 5 provided in the slave robot 2 will be controlled so as to correspond to the operation of the arm control unit 430 by the operation signal transmitted to the slave robot 2 in step 840. [

The moving speed control method of the laparoscope 5 is the same as the moving speed control method of the surgical tool described above, so that a description thereof will be omitted.

However, if it is determined in step 830 that the IVS button is not selected to operate in the vision control mode, the process proceeds to step 850.

In step 850, the control unit 460 determines whether the tool / vision selection (IVS) button has been selected to operate the surgical robot system in the integrated control mode.

As a result of the determination in step 850, if the IVS button has been selected to operate in the integrated control mode, the operation signal generator 450 proceeds to step 860, and the operation signal generator 450 controls the surgical tool and the laparoscope 5 ) Is generated and transmitted to the slave robot.

Here, the operation signal moves the surgical tool and the laparoscope 5 together in the direction in which the arm control unit 430 is operated, and the aligned states (for example, mutual relative positions, etc.) To be moved at a speed corresponding to the operation amount of the operation lever 430. That is, even if the physical position of the imaging region is changed in accordance with the movement of the laparoscope 5, the relative position of the surgical instrument with respect to the center of the imaging region on the display screen confirmed by the surgeon through the screen display unit 420, Lt; / RTI >

Also, even when the surgical tool is moved in the speed control mode, there may be a case where the movement of the surgical tool is limited as described above in the operation state of the integrated position control mode. For example, in a state in which one of the surgical tools holds something (for example, a state in which a certain portion of the surgical site is held, see FIGS. 5A and 5B), or the end of the surgical tool is not displayed in the display screen Or a jaw of the surgical tool is closed or a surgical tool designated to keep the operator in a fixed state or when the fixed state maintenance button provided on the arm control part 430 is depressed, In the case of such a surgical tool, the movement may be restricted.

However, if it is determined in step 850 that the IVS button has not been selected to operate in the integrated control mode, the operation signal generator 450 proceeds to step 870 to determine whether the operation tool and / or the laparoscope 5) is operated and transmitted to the slave robot. For example, in the hold mode or the master repositioning mode, an operation signal is generated and transmitted to the slave robot so that the surgical tool and / or the laparoscope 5 can maintain the last control position regardless of the operation of the master.

It is a matter of course that the adaptive adjustment method of the moving scale factor and the adaptive control method according to the cancer operation characteristic can be performed in an automated procedure according to a time series sequence by a software program or the like built in the digital processing device. The codes and code segments that make up the program can be easily deduced by a computer programmer in the field. In addition, the program is stored in a computer readable medium, readable and executed by a computer, thereby implementing the method. The information storage medium includes a magnetic recording medium, an optical recording medium, and a carrier wave medium.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. And changes may be made without departing from the spirit and scope of the invention.

1: Master robot 2: Slave robot
3: Robot arm 5: Laparoscope
410: image input unit 420:
430: Cancer control unit 440: Scale factor calculation unit
450: Operation signal generator 460:

Claims (33)

A master interface for controlling a position of the control object in a surgical robot system to which a control object, which is at least one of a surgical endoscope and a surgical tool,
An operation unit for receiving a user command for moving the control object;
A storage unit for storing a reference image which is at least one of image information provided from the surgical endoscope and a motion scale factor corresponding to the reference image;
A scaling factor for determining whether the current image information currently provided from the surgical endoscope is reduced or enlarged relative to the reference image and adjusting the moving scale factor so as to linearly or non- An operation unit; And
And a control signal generation unit for generating a control signal for the control object by converting a user command using the control unit to correspond to the reset moving scale factor, The master interface being adapted to be subordinate to the master interface.
The method according to claim 1,
The scale factor operation unit may detect the current image information based on a preset image analysis technique and determine a zone in which the current image information matches the reference image in order to determine whether the current image information is enlarged or reduced compared to the reference image Features a master interface.
3. The method of claim 2,
The scale factor operation unit may adjust the moving scale factor using a proportional relationship between conversion information that is at least one of a horizontal size, a vertical size, a diagonal line length, and an area of the set region and information corresponding to the conversion information in the reference image Features a master interface.
delete The method according to claim 1,
Wherein the surgical endoscope is at least one of a laparoscope, a thoracoscopic, an arthroscope, a non-diameter, a cystoscope, a rectum, a duodenal, a mediastinum, and a heart.
A master interface for controlling a position of the control object in a surgical robot system to which a control object, which is at least one of a surgical endoscope and a surgical tool,
An operation unit for receiving a user command for moving the control object;
A storage unit for storing one or more moving scale factors according to photographing conditions of a surgical endoscope;
A scale factor operation unit for receiving information on an imaging condition of the surgical endoscope and setting a corresponding mobile scale factor as a mobile scale factor to be applied to the storage unit; And
And an operation signal generation section for converting the user command using the operation section to correspond to the set mobile scale factor to generate an operation signal for the control object.
The method according to claim 6,
Wherein the photographing condition is at least one of a photographing zoom magnification of the surgical endoscope, a photographing position, a distance from the surface of the surgery part, and an actual size of a surgical part to be photographed.
8. The method of claim 7,
Wherein the moving scale factor when the imaging zoom magnification is high is relatively smaller than the moving scale factor when the imaging zoom factor is low.
8. The method of claim 7,
Wherein a moving scale factor when the surgical endoscope is relatively inserted into the human body is smaller than a moving scale factor when the surgical endoscope is inserted relatively less.
8. The method of claim 7,
Receiving a sensing value from a sensor provided to sense a distance to the surgical site surface,
Wherein the mobile scale factor is set to a relatively small value as the distance from the surface of the surgical site is closer.
8. The method of claim 7,
Wherein the moving scale factor is set to a relatively small value as the actual size of the surgical region to be photographed is reduced.
The method according to claim 6,
Wherein the moving distance of the control object according to the user command is adjusted depending on the moving scale factor.
8. The method of claim 7,
Wherein the moving scale factor is computed in real time according to the variation of the photographing condition and stored in the storage unit.
The method according to claim 6,
Wherein the scale factor calculator comprises:
Wherein the master interface sets a larger scale factor than when the operating section is operated with a relatively small manipulated variable when the operating section is operated with a relatively large manipulated variable for a moving operation of the surgical tool.
The method according to claim 6,
Wherein the information about the photographing condition is depth information calculated by analyzing image information photographed by the surgical endoscope when the surgical endoscope is a three-dimensional image acquiring device.
The method according to claim 6,
Wherein the surgical endoscope is at least one of a laparoscope, a thoracoscopic, an arthroscope, a non-diameter, a cystoscope, a rectum, a duodenal, a mediastinum, and a heart.
An adaptive control method of a control object, which is at least one of a surgical endoscope and a surgical tool performed in the master interface according to claim 1,
Storing, in the storage, a reference image that is at least one of image information provided from the surgical endoscope and a motion scale factor corresponding to the reference image;
Determining whether the current image information currently provided from the surgical endoscope is reduced or enlarged in comparison with the reference image, in the scale factor operation unit;
Adjusting and resetting the moving scale factor in a linear or nonlinearly proportional manner to the determined reduction or enlargement degree in the scale factor computing unit; And
And generating an operation signal for the control object by converting the user command input for the movement operation of the control object to correspond to the reset mobile scale factor in the operation signal generation unit, Control method.
18. The method of claim 17,
The determining may include detecting and matching a region of the current image information that matches the current image information according to a predetermined image analysis technique to determine whether the current image information is reduced or expanded relative to the reference image Wherein the control object is an adaptive control object.
19. The method of claim 18,
The adjusting and resetting may further include adjusting the moving scale factor using a proportional relationship between conversion information that is at least one of a horizontal size, a vertical size, a diagonal length, and an area of the set region and information corresponding to the conversion information in the reference image, Wherein the control object is a control object.
18. The method of claim 17,
Wherein the moving distance of the control object according to the user command is adjusted depending on the moving scale factor.
18. The method of claim 17,
Wherein the surgical endoscope is at least one of a laparoscope, a thoracoscopic, an arthroscope, a nystagmus, a bladder, a rectum, a duodenal, a mediastinum, and a heart.
An adaptive control method of a control object, which is at least one of a surgical endoscope and a surgical tool performed in the master interface according to claim 6,
Storing, in the storage, one or more moving scale factors according to an imaging condition of the surgical endoscope;
Receiving information on the imaging condition of the endoscope for operation in the scale factor operation unit and setting a corresponding mobile scale factor as a mobile scale factor to be applied to the storage unit; And
And generating an operation signal for the control object by converting the user command inputted for the movement operation of the control object to correspond to the set mobile scale factor in the operation signal generator, Way.
23. The method of claim 22,
Wherein the photographing condition is at least one of an imaging zoom magnification of the endoscope for surgery, a photographing position, a distance from a surface of a surgical site, and an actual size of a surgical site to be photographed.
24. The method of claim 23,
Wherein the moving scale factor when the image capturing zoom factor is high is relatively smaller than the moving scale factor when the image capturing zoom factor is low.
24. The method of claim 23,
Wherein the moving scale factor when the surgical endoscope is relatively inserted into the human body has a smaller value than the moving scale factor when the surgical endoscope is relatively less inserted. .
24. The method of claim 23,
And receiving distance information from the surface of the surgical site sensed by the sensor,
Wherein the moving scale factor is set to a relatively small value as the distance from the surface of the surgical site is closer.
24. The method of claim 23,
Wherein the moving scale factor is set to a relatively small value as the actual size of the surgical region to be imaged becomes smaller.
23. The method of claim 22,
Wherein the moving distance of the control object according to the user command is adjusted depending on the moving scale factor.
24. The method of claim 23,
Wherein the moving scale factor is computed in real time according to the variation of the photographing condition and stored in the storage unit.
23. The method of claim 22,
Wherein the setting step comprises:
Wherein when the user command is for a manipulation amount of a relatively large manipulation part for the manipulation of the control object, the manipulation part is set to a larger moving scale factor than when the manipulation part is manipulated with a relatively small manipulation amount Adaptive control method.
23. The method of claim 22,
Wherein the information about the photographing condition is depth information calculated through analysis of image information photographed by the surgical endoscope when the surgical endoscope is a three-dimensional image acquiring apparatus. Control method.
23. The method of claim 22,
Wherein the surgical endoscope is at least one of a laparoscope, a thoracoscopic, an arthroscope, a nystagmus, a bladder, a rectum, a duodenal, a mediastinum, and a heart.
delete

Images