WO2010021447A1

WO2010021447A1 - Three-dimensional display system for surgical robot and method for controlling same

Info

Publication number: WO2010021447A1
Application number: PCT/KR2009/001388
Authority: WO
Inventors: 원종석; 최승욱
Original assignee: (주)미래컴퍼니
Priority date: 2008-08-21
Filing date: 2009-03-18
Publication date: 2010-02-25
Also published as: CN102186434B; CN102186434A; KR100998182B1; KR20100023086A

Abstract

Disclosed are a three-dimensional display system for a surgical robot and a method for controlling same. The three-dimensional display system includes a 3D display for displaying 3-dimensional video information acquired from a surgery, a robotic arm coupled to the three-dimensional display, a surgical robot to which the robotic arm is connected, and a controller contained within the surgical robot. The controller drives the robot arm to move the three-dimensional display to a predetermined position. In the display system for a surgical robot of the present invention, a three-dimensional display is mounted on an articulated robotic arm, and the robotic arm is driven in a manual or automated system to enable the three-dimensional display to track and move along the position of the eye of an operator, to thereby eliminate the necessity of changing the posture of the operator even when the operator intends to check three-dimensional video information, and eliminate the necessity of putting a separate display on the body of the operator. Further, the present invention presets the position of the three-dimensional display for each operator, to thereby constantly provide accurate three-dimensional video information irrespective of the operator and the posture thereof.

Description

3D display system of surgical robot and its control method

The present invention relates to a 3D display system of a surgical robot and a control method thereof.

Medically, surgery refers to repairing a disease by cutting, slitting, or manipulating skin, mucous membranes, or other tissues with a medical device. In particular, open surgery, which incise the skin of the surgical site and open, treat, shape, or remove the organs inside of the surgical site, has recently been performed using robots due to problems such as bleeding, side effects, patient pain, and scars. This alternative is in the spotlight.

As described above, in the case of robotic surgery using a surgical robot, the doctor proceeds to surgery while looking at the patient's body as a 3D image through a 3D (3-dimension) display. As a method of implementing a 3D display in a surgical robot system, a first microscope display method may be used.

The microscope method is a method of performing robot surgery while looking at the surgical site in three dimensions through a microscope installed in the surgical robot, and there is a fear that the shoulder or neck of the doctor may not be able to concentrate on the operation or cause malfunction of the robot during a long operation. have.

Second, HMD (head mount display) is a method that uses a helmet-shaped display on the head and checks three-dimensional images through this operation, but HMD is generally heavy and inconvenient, and sees other things during surgery. In order to remove the helmet is a hassle.

Third, there is a polarizing display in the form of glasses, which realizes a three-dimensional image in a polarized manner, and thus the image is dark in general, and there is a limitation in that the polarized glasses must be removed to see another one during surgery.

The background art described above is technical information possessed by the inventors for the derivation of the present invention or acquired during the derivation process of the present invention, and is not necessarily known technology disclosed to the general public before the present application.

The present invention, by moving the 3D display according to the operator's position in the robot surgery process, even if the operator does not change the posture to see the image or wear the display device on the operator's body, robot surgery while simply checking the three-dimensional image as needed It is to provide a 3D display system and a control method of the surgical robot that can proceed.

Technical problems other than the present invention will be easily understood through the following description.

According to an aspect of the present invention, a 3D display for displaying three-dimensional image information obtained from the surgical site, a robot arm coupled to the 3D display, a surgical robot to which the robot arm is connected, a controller included in the surgical robot Including, but the controller is provided with a 3D display system of the surgical robot, characterized in that for driving the robot arm to move the 3D display to a predetermined position.

The image information may be image information taken by a vision system inserted into a surgical patient and processed to be displayed on a 3D display. The robotic arm may include an articulated link driven by a controller to move the 3D display to a predetermined position. One end of the robot arm is fixed to the structure of the operating room, the robot arm may be electrically connected to the surgical robot.

The surgical robot is provided with a console unit for user manipulation, and the controller can move the 3D display according to the position of the user's eyes. In this case, an operation handle is installed in the console, and the controller may move the 3D display according to a user's operation on the operation handle.

The controller is connected to a sensing unit for sensing the position of the user's eyes, and the controller may receive a signal from the sensing unit and move the 3D display according to the position of the user's eyes. The controller is connected to a storage unit for storing data regarding the position of the 3D display to be moved according to the eye position of each user, and the controller loads the data stored in the storage unit in accordance with the user who operates the console unit, and previews the 3D display in advance. You can move it to a stored location.

Meanwhile, according to another aspect of the present invention, there is provided a method of controlling a 3D display coupled to a robot arm electrically connected to a surgical robot, the method comprising: acquiring information about a user accessing the surgical robot; Accordingly, a method of controlling a 3D display system of a surgical robot including driving a robot arm to move a 3D display is provided.

The 3D display may display image information captured by the vision system inserted into the body of the surgical patient, and the information about the user may include information about the position of the user's eyes.

The acquiring step may include sensing a position of an eye of the user by a sensor, and receiving a signal from the sensor to derive information about the user. Meanwhile, the information about the user is stored in advance in the form of data regarding the position of the 3D display to be moved according to the position of the user's eyes, and in this case, the obtaining step may include loading the data.

Other aspects, features, and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to a preferred embodiment of the present invention, the 3D display is mounted on the articulated robot arm and the robot arm is manually or automatically driven so that the 3D display is moved by tracking the position of the operator's eyes, so as to confirm the 3D image. There is no need for the operator to change posture, no need for a separate display device to be worn on the operator's body, and the 3D display is preset for each operator, so that the operator and the operator's posture are always correct. Dimensional image information can be provided.

1 is a perspective view showing a 3D display system of a surgical robot according to an embodiment of the present invention.

Figure 2 is a flow chart showing a 3D display system control method of a surgical robot according to an embodiment of the present invention.

1: Surgical Robot 3: Console

5: operation handle 10: 3D display

12: robot arm 20: controller

22: sensing unit 24: storage unit

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. As used herein, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

In addition, in the description with reference to the accompanying drawings, the same components regardless of reference numerals will be given the same reference numerals and duplicate description thereof will be omitted. In the following description of the present invention, if it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

1 is a perspective view showing a 3D display system of a surgical robot according to an embodiment of the present invention. Referring to FIG. 1, a surgical robot 1, a console unit 3, an operation handle 5, a 3D display 10, a robot arm 12, a controller 20, a sensing unit 22, and a storage unit 24 is shown.

The present embodiment is a so-called 'articulated 3D display' method, and does not fix the part where the 3D image is displayed on the console or wear it on the user's head as in the microscope method or the HMD method. It is characterized in that the 3D display 10 is free to move by connecting the robot arm 12 to 10).

3D display is a system that reproduces 3D images, and includes a software technology that can be viewed in 3D and hardware that implements the contents created by the software in 3D. Since the human eye is about 65 mm apart, when images with a slight parallax (binocular disparity) of left and right information enter both eyes, it is transmitted through the optic nerve to the brain and the two images merge in the brain. You will feel a three-dimensional sense of space.

As such, the 3D display is a system that allows a user to feel a virtual three-dimensional feeling in a flat display hardware. A two-dimensional display device simultaneously displays two left and right images and sends them to each eye to create a virtual three-dimensional effect.

In such a 3D display, a polarization method that separates images of left and right eyes using a light shielding effect by a combination of orthogonal polarizing elements, and a screen in which images of left and right channels are alternately rapidly displayed on one screen through shutter glasses or the like. Time-division method that allows fast blocking and opening / closing, allowing only one direction image to enter one eye.HMD that displays the screen in front of the eye with a helmet-type display device and focuses clearly using the lens. There is a color difference method for realizing a stereoscopic by separating the images of both channels from the color dimension with glasses of red color on one side and blue color on the other side.

In this embodiment, a robot arm having a 3D display 10 which enables the display of each of the left and right in various ways as described above is compact and lightweight as shown in FIG. 1, and has a predetermined degree of freedom. Combination of the 12 allows the user to move the 3D display 10 as desired.

For example, the doctor performing robotic surgery may move the 3D display 10 to a position desired by the doctor by tracking the posture of the doctor, as well as when the surgeon is sitting or standing. . This movement of the 3D display 10 can be manually or automatically driven by the robot arm 12 to be the most comfortable position for the user to check the 3D image.

To this end, the 3D display system of the surgical robot according to the present embodiment is coupled to the robot arm 12 to the 3D display 10, the robot arm 12 is connected to the surgical robot 1, the surgical robot ( The basic structure is a configuration for controlling the movement of the 3D display 10 by the controller 20 included in 1).

The 3D display 10 is a device for displaying three-dimensional image information taken from the operating field, such as the operating room, the detailed configuration may be implemented by applying a variety of methods as described above, such as polarization method, time division method, color difference method, Here, a detailed description of the configuration of the 3D display 10 itself is omitted.

On the other hand, the vision system (vision system) serves to provide the image information photographed in the surgical process, it is a system including a device such as a laparoscope, an endoscope, a microscope, a magnifying glass, a reflector. Hereinafter, a case where a laparoscope is used as an example of a vision system will be described.

The image displayed through the 3D display 10 may be an image taken from the laparoscope inserted into the body of the surgical patient, and if necessary, the image information taken from the laparoscope is displayed as a 3D image on the 3D display 10. The image processed data can be used.

As described above, the robot arm 12 is coupled to the 3D display 10 according to the present exemplary embodiment so that the user can manually or automatically move the 3D display 10. When the robot arm 12 is driven by receiving an electrical signal by the controller 20 as described below, the robot arm 12 according to the present embodiment may be formed of a multi-articulated link. For example, when the 3D display 10 is moved to a specific point in space using the robot arm 12 having three degrees of freedom, the robot arm 12 may be configured in the form of a link connected by three joints. have.

However, the robot arm 12 according to the present embodiment is not necessarily configured in the form of a multi-joint link, and the robot is applied by applying various mechanisms capable of freely moving the 3D display 10 by receiving a signal from the controller 20. Of course, the arm 12 may be configured.

The robot arm 12 according to this embodiment is driven by the controller 20. The controller 20 may be implemented in various forms such as a microprocessor embedded in the surgical robot 1, a control box connected to the surgical robot 1, or a personal computer connected to the surgical robot 1 through wired / wireless communication. Can be.

The controller 20 receives a user's input in the case of the manual method, a pre-stored data or a signal sensed from the sensor in the case of the automatic method, and determines the final position to which the 3D display 10 moves, and the 3D display 10. Calculates the degree to which the robot arm 12 should be driven to move from the current position to the final position, and then generates and transmits a signal such that the robot arm 12 is driven by the calculated degree.

As described above, when the robot arm 12 is configured in the form of articulated links, the controller 20 calculates the degree of rotation of each joint, and then transmits a driving signal to each joint so that each joint rotates by the calculated degree. .

The robot arm 12 according to this embodiment is connected to the surgical robot 1. That is, as shown in FIG. 1, the other end of the robot arm 12 having the 3D display 10 coupled to one end thereof may be coupled to the surgical robot 1. The robot arm 12 coupled to the surgical robot 1 may also be electrically connected to the surgical robot 1 to receive a driving signal generated from the controller 20.

The surgical robot 1 is a master / slave system, that is, a master robot that generates and transmits a signal required by a user's operation, and a slave that receives a signal from a master robot and directly applies a manipulation necessary for surgery to a patient. ) The instruments mounted on the surgical robot (1) by the user's operation of the robot is formed separately separately, or the master-slave integrated type, that is, integrated surgical robot (1) to perform the operation required for surgery It can be made in the form of adding.

However, the robot arm 12 according to the present embodiment is not necessarily coupled to the surgical robot 1. For example, the robot arm 12 is fixed to the ceiling of an operating room and the like, and is electrically connected to the surgical robot 1 by cable or wired / wireless communication, whereby the robot arm 12 is connected by the controller 20. It can also be configured to be driven.

As such, by coupling the 3D display 10 to the robotic arm 12 and electrically connecting the robotic arm 12 to the surgical robot 1, the 3D display 10 may be connected to the surgeon no matter what position the doctor takes. The 3D display 10 may be freely positioned so that a doctor or a nurse or an assistant may view a 3D image if necessary.

When the console part 3 for the user's operation is installed in the surgical robot 1, the 3D display 10 may be moved by manual operation by a user or automatic operation by preset data or sensors. In this case, the controller 20 drives the robot arm 12 so that the 3D display 10 is located in front of the user's eyes.

That is, the controller 20 according to the present exemplary embodiment transmits a driving signal to the robot arm 12 so that the 3D display 10 moves to a point where the user's eyes are located, where the user's eyes are located. It can grasp | ascertain by operation by, preset data, the value detected by the sensor, etc.

When the 3D display 10 is moved by a manual operation by a user, an operation handle 5 is installed in the console unit 3, and the user operates the operation handle 5 so that the 3D display 10 is located in the user's eyes. ), The controller 20 drives the robot arm 12 according to the value input from the manipulation handle 5 to position the 3D display 10 in front of the user's eyes.

The manipulation handle 5 according to the present embodiment may be a dedicated handle for the movement of the 3D display 10, and the main handle for the manipulation of the surgical instrument may move the 3D display 10. It can also be used as an operation handle (5) for. In the latter case, a selection button or the like is additionally installed on the main handle, and the main handle is normally used for the operation of the instrument, and when the 3D display 10 is moved, the main handle is operated by pressing the selection button after pressing the selection button. In this embodiment, it can also be used as another operation handle (5).

In addition, by installing the operation handle 5 according to the present embodiment on the 3D display 10, the user may grasp the operation handle 5 and move the 3D display 10 directly.

Meanwhile, the 3D display 10 may automatically move in front of the user by sensing a point where the user's eyes are located. In this case, the sensing unit 22 which detects the user's eyes is connected to the controller 20, and the controller 20 receives the signal transmitted by the sensing unit 22 sensing the position of the user's eyes and the robot arm. (12) is driven.

The sensing unit 22 may include a sensor for detecting the eye of the user, a cable for signal transmission, and in some cases a processor for processing a signal. For example, when using the image sensor to determine the position of the user's eyes, the processor analyzes the image obtained from the image sensor to calculate the coordinates of the location where the user's eyes are located, the calculated coordinate value is a controller ( 20, the controller 20 may cause the 3D display 10 to move to the position of the user's eyes.

However, the sensing unit 22 according to the present embodiment does not necessarily sense the user's eyes, and when the user's face or other part of the body or the user wears a specific device, the corresponding device is detected and therefrom. The position of the eye of the user may be determined and a signal corresponding thereto may be transmitted to the controller 20.

For example, when the user wears a dedicated mark that can be detected by the sensing unit 22 according to the present embodiment at a specific position of the body and performs robot surgery, the sensing unit 22 detects the corresponding mark. The position of the eye of the user may be calculated from the position of the mark, and the sensing signal may be generated and transmitted accordingly.

As a result, the 3D display 10 system according to the present exemplary embodiment may detect the position or movement of the user, and allow the 3D display 10 to automatically track and move the position of the user's eyes within a predetermined range.

On the other hand, for each user who operates the surgical robot 1 according to the present embodiment is set in advance to the point where the 3D display 10 should be located, so that when the user operates the surgical robot (1) The 3D display 10 may be configured to move.

In this case, the storage unit 24, which stores data about the position of the 3D display 10, for example, the eye position, for each user in advance, is connected to the controller 20, and the user is a surgical robot. When accessing (1), the controller 20 loads data about the user from the storage unit 24 to drive the robot arm 12 so that the 3D display 10 is located in front of the user's eyes.

To this end, the surgical robot 1 according to the present exemplary embodiment may further include an ID identification unit for determining a user, so as to identify a user accessing the surgical robot 1. The ID identification unit may be configured to manually recognize an ID card possessed by the user, or automatically recognize an ID chip worn on the user's body, or install a fingerprint reader to recognize the user. An identification device can be used.

Accordingly, when it is determined that a specific user has accessed the surgical robot 1, the controller 20 retrieves data about the user from the storage unit 24 and positions the 3D display 10 to fit the user. Let's go.

On the other hand, in the case of a user whose data is not stored in the storage unit 24, the 3D display 10 is moved by the above-described manual operation or sensing method, and the data used at this time is stored as data about the user. It can be stored in addition to section 24.

In addition, even if the user has already stored data in the storage unit 24, the data related to the position of the 3D display 10 can be updated and stored as needed. Data can be stored. In this case, after identifying not only the user's ID but also information on the posture or motion, the controller 20 loads the pre-stored data about the posture of the user and moves the 3D display 10 to the desired position. You can.

As a result, the 3D display system according to the present embodiment individually stores the positions where the 3D display 10 is to be positioned for each user having different physical conditions, and the like, so that the 3D display is optimized for each user who uses the surgical robot 1. The drive of (10) can be managed.

On the other hand, since the 3D display system according to the present embodiment can be used regardless of the presence or absence of a slave robot for the operation of the surgical instrument, as described above, the instrument of the master-slave integrated surgical robot 1, that is, 'intelligent' It can also be used for 'intelligent instruments'.

2 is a flowchart illustrating a method of controlling a 3D display system of a surgical robot according to an exemplary embodiment of the present invention. Hereinafter, a method of controlling the above-described 3D display system will be described.

The 3D display 10 according to the present embodiment is electrically connected to the surgical robot 1, and the surgical robot 1 is equipped with a controller 20 so that the 3D display 10 connected to the surgical robot 1 ( 10) to control the movement.

This embodiment is characterized in that the control method for moving the 3D display 10 to fit the user using the surgical robot 1 to position the 3D display 10 most suitable for each user and the posture of each user. To this end, first, information about a user accessing the surgical robot 1 is obtained (S10).

The information about the user corresponds to input information for moving the 3D display 10. For example, when the 3D display 10 is positioned in front of the user's eyes, the information about the user is related to the position of the user's eyes. Can be coordinates.

The image displayed through the 3D display 10 may be an image taken from the abdominal cavity inserted into the body of the surgical patient, and if necessary, display the image information taken from the laparoscope, etc., as a 3D image on the 3D display 10. As described above, the image processed data may be used.

As a method of acquiring information about a user, a method of sensing a position of a user's eyes by a sensor (S12), a method of loading data about a position of an eye of a user stored in advance (S16), and the like can be given. .

Specific means for sensing the position of the user's eyes have been described above, so a detailed description thereof will be omitted. When the position of the user's eyes is sensed by the sensor, the sensor generates and transmits a predetermined signal according to the sensing, and the controller 20 according to the present embodiment receives the information about the user, that is, the eye of the user. Position coordinates and the like are derived (S14).

When information about a user is stored in advance in the form of data and loaded as necessary, the data is a position to which the 3D display 10 is to be moved according to the position of the user's eyes for each user and for each user's posture as described above. It may be made in the form of a coordinate with respect to. Since the type of data to be stored for each user and the posture of each user and the specific means for storing and loading the data have been described above, a detailed description thereof will be omitted.

As described above, after sensing the position of the eye of the user (S12) or loading the position at which the 3D display 10 is to be moved according to previously stored data (S16), the robot arm 12 according to the obtained information. The 3D display 10 is moved to drive (S20).

Thus, when a specific user accesses the surgical robot 1 according to the present embodiment, the 3D display 10 is automatically moved to the optimal position to fit the user.

As described above, the 3D display system and a control method of the surgical robot according to an embodiment of the present invention has been described in accordance with an embodiment, but is not necessarily limited thereto, the 3D display 10 and the surgical robot ( Even if the detailed configuration of 1), the connection method between the robot arm 12 and the surgical robot 1, the specific operating mode of the controller 20, etc. are different, the overall operation and effects are not different. It can be included in the scope of the right, and those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention described in the claims below. You will understand.

Claims

3D display for displaying three-dimensional image information obtained from the surgical site;

A robot arm coupled to the 3D display;

A surgical robot to which the robot arm is connected;

Including a controller included in the surgical robot,

And the controller drives the robot arm to move the 3D display to a predetermined position.
The method of claim 1,

And the image information is image information processed by a vision system inserted into a surgical patient and processed to be displayed on the 3D display.
The method of claim 1,

The robotic arm includes a articulated link driven by the controller to move the 3D display to a predetermined position.
The method of claim 1,

One end of the robot arm is fixed to the structure of the operating room, the robot arm is 3D display system of the surgical robot, characterized in that electrically connected with the surgical robot.
The method of claim 1,

The surgical robot is provided with a console unit for the user's operation, the controller 3D display system of the surgical robot, characterized in that for moving the 3D display corresponding to the position of the user's eyes.
The method of claim 5,

The control unit is provided with an operation handle, the controller 3D display system of the surgical robot, characterized in that for moving the 3D display in accordance with a user operation on the operation handle.
The method of claim 5,

The controller is connected to a sensing unit for detecting the position of the user's eyes, the controller receives a signal from the sensing unit, and moves the 3D display according to the position of the user's eyes 3D of the surgical robot Display system.
The method of claim 5,

The controller is connected to a storage unit for storing data relating to a position to which the 3D display is to be moved according to the eye position of each user, and the controller loads data stored in the storage unit corresponding to a user who operates the console unit. 3D display system of the surgical robot, characterized in that for moving the 3D display to a pre-stored position.
A method of controlling a 3D display coupled to a robotic arm electrically connected to a surgical robot,

Acquiring information about a user accessing the surgical robot; And

And driving the robot arm in accordance with the obtained information to move the 3D display.
The method of claim 9,

The 3D display is a control method for a 3D display system of a surgical robot, characterized in that for displaying the image information taken by the vision system inserted into the body of the surgical patient.
The method of claim 9,

The information about the user 3D display system control method of the surgical robot, characterized in that it includes information about the position of the user's eyes
The method of claim 9,

The obtaining step,

Sensing a position of an eye of a user by a sensor; And

Receiving a signal from the sensor to derive information about the user 3D display system control method for a surgical robot comprising the.
The method of claim 9,

The information about the user is previously stored in the form of data regarding the position to which the 3D display is to be moved according to the position of the user's eyes,

The acquiring step includes the step of loading the data.