KR20100123160A - Method of detection for surgical instruments, recording media of the same, surgical system of using the same and surgical subsystem of using the same - Google Patents

Abstract

PURPOSE: A method of detection for surgical instruments, recording media of the same, a surgical system of using the same and the surgical subsystem of using the same are provided to maintain a surgical instrument optimum by minimize an effect due to a foreign material. CONSTITUTION: An image is generated by using a surgical image(S310). A surgical instrument is distinguished from a surgical image by using the generated image(S320). The surgical instrument is selected from the surgical image(S321). A background image is removed from the surgical image(S322). The surgical instrument is detected by using the generated image(S330).

Description

Surgical system for detecting surgical instruments, recording media recording surgical instrument detection methods, surgical systems for performing surgical instrument detection methods, and surgical auxiliary systems for performing surgical instrument detection methods of using the same and Surgical Subsystem of using the same}

The present invention relates to a surgical tool detection method, a recording medium recording a surgical tool detection method, a surgical system for performing a surgical tool detection method and a surgical auxiliary system for performing a surgical tool detection method, in more detail using a surgical image The present invention relates to a surgical instrument for performing a surgical instrument detection method, a recording medium recording the surgical instrument detection method, a surgical system for performing the surgical instrument detection method, and a surgical auxiliary system for performing the surgical instrument detection method.

Surgical techniques that minimize invasion have advantages such as minimal scarring, reduced patient pain, shorter recovery time, shorter hospital stay, and rapid return to normal life compared to traditional open surgery techniques.

Laparoscopic surgery is a surgical technique that minimizes invasion and uses a tool called a laparoscope to show the anatomy and pelvis in the abdomen. The abdominal cavity is expanded by the injection of an absorbable gas, such as carbon dioxide, to secure the view of the laparoscope. In addition, with minimal incisions into the skin, probes or other surgical instruments can be introduced into the surgical site. Therefore, the operator can see the abdominal cavity through the laparoscopic image and can perform surgery such as incision using the surgical tool.

Conventionally, in the case of laparoscopic surgery, the surgical assistant played a role of controlling the laparoscopic image, and accordingly, the operator had to give the surgical assistant a control command of which direction to operate the laparoscopic image. In other words, the operator could not concentrate on the operation only, and had to distract his attention to the manipulation of the laparoscopic image. In addition, the surgical assistant had a difficulty in maintaining laparoscopic images without shaking.

In order to solve this problem, a surgical robot and a surgical assistant robot have appeared, and since the laparoscope image can be fixed and controlled by such a robot, a more stable laparoscope image can be provided. However, there was a problem that a manual command of an operator is required even for the control of laparoscopic images by a robot. That is, the operator still needs additional operations such as delivering a voice command to the robot, using a pedal to transmit a command, or transmitting a command through vision.

1 is a conceptual diagram of a surgical system using a laparoscope incorporating a conventional surgical robot.

Referring to FIG. 1, in place of a surgical assistant, a surgical robot may fix a laparoscopic image and control a position of the laparoscopic image according to an operator's command. The image of the abdominal cavity is transmitted as an image signal by the photographing unit 120, and the transmitted image signal is output from the display unit 130 so that the operator may visually confirm the laparoscope image through the display unit 130. In addition, the operator can operate the surgical tool by using the surgical device 110, thereby performing the surgery. Here, the surgical tool refers to a tool such as a surgical scalpel, a surgical hook, a surgical pincette, a surgical scissor, and the like. In addition, there is a photographing position adjusting unit 150 to manipulate the photographing unit 120, the operation command to the photographing position adjusting unit 150 to adjust the position of the laparoscopic image confirmed by the operator from the display unit 130 The command input unit 160 is configured to be delivered. The command input unit 160 may exist in various forms. That is, the voice recognition device may be in a form of recognizing the operator's voice command, it may be a pedal that receives the command using the operator's foot. In addition, it may be in the form of receiving the movement command of the eye by recognizing the pupil of the operator. However, there is still a problem that it is difficult for the operator to concentrate on the surgery due to the necessity of manipulation of the command input unit 160 as described above.

2 is a conceptual diagram of a surgical assistance system using a laparoscope incorporating a conventional surgical assistant robot.

Referring to Figure 2, in place of the surgical assistant, the surgical assistant robot to fix the laparoscopic image and to control the position of the laparoscopic image according to the operator's command and that the operator directly controls the surgical tool 210 to perform the operation Except for the role of other components may be the same as described above in FIG. The surgical tool may be a tool such as a laparoscopic mess, a laparoscopic hook, a laparoscopic pincette, a laparoscopic scissor, and the like. However, even in the case of the surgical assistance system, there is still a problem that it is difficult for the operator to concentrate solely on the surgery due to the necessity of manipulation of the command input unit 260 as described above.

An object of the present invention is to provide a surgical tool detection method using a surgical image.

Another object of the present invention is to provide a recording medium recording a method for detecting a surgical tool using a surgical image.

Another object of the present invention relates to a surgical system for performing a surgical tool detection method using a surgical image.

Another object of the present invention relates to a surgical assistance system for performing a surgical tool detection method using a surgical image.

Surgical tool detection method according to an embodiment of the present invention for achieving the above object of the present invention in the method for detecting a surgical tool in the surgical image, using the surgical image pixel information of the surgical image Generating image information including at least one of an RGB value and an HSI value representing an HSI value; Classifying the surgical tool in the surgical image by using the generated image information; And inspecting the surgical tool in an image in which the surgical tool is divided using the generated image information.

Here, the surgical image may be an image representing the surgical tool and the surgical site.

Here, the surgical image may be any one of a laparoscope image and an endoscope image.

The generating of the image information may further include removing noise with respect to the surgical image.

Here, the step of distinguishing the surgical tool from the surgical image by using the generated image information is at least one of selecting a surgical tool from the surgical image and removing the background area from the surgical image It may be configured to include.

값 및

값 중의 어느 하나에 대하여 이진(binary)화를 수행하는 방법, 상기 RGB 값의 R, G 및 B 간의 차이값을 이용하여 이진(binary)화를 수행하는 방법 및 상기 RGB 값을 이용한 RGB 표준편차값(standard deviation)을 이용하여 이진(binary)화를 수행하는 방법 중 적어도 하나를 적용하여 상기 수술도구를 선택하는 것일 수 있다.Here, the step of selecting a surgical tool in the surgical image using the HSI value

Value and

A method of performing binarization on any one of the values, a method of performing binarization using difference values between R, G, and B of the RGB value, and an RGB standard deviation value using the RGB value It may be to select the surgical tool by applying at least one method of performing binarization using a standard deviation.

The removing of the background area from the surgical image may include skin color detection method using the RGB value, red color detection method using the RGB value, R of the RGB value, The background area may be removed by applying at least one of a min max RGB comparison method of comparing the minimum value and the maximum value in G and B. FIG.

Here, the step of inspecting the surgical tool in the image divided by the surgical tool by using the generated image information, the separation (erosion), opening (opening) and It may be to use at least one of a method of applying at least one image processing of the closing (closing) and a method of applying a condition that the surgical tool passes the boundary of the surgical image.

In the recording medium recording the surgical tool detection method according to an embodiment of the present invention for achieving another object of the present invention described above in the recording medium recording the method for detecting the surgical tool in the surgical image, Generating image information including at least one of an RGB value and an HSI value representing pixel information of the surgical image using the image; Classifying the surgical tool in the surgical image by using the generated image information; And inspecting the surgical tool in an image of the surgical tool by using the generated image information.

Surgical system for performing a surgical tool detection method according to an embodiment of the present invention for achieving another object of the present invention described above in the surgical system using a surgical image, for controlling the surgical tool and the surgical tool A surgical device unit including a surgical tool controller; A photographing unit for generating a surgical image for the surgical tool and the surgical site; A display unit for receiving and outputting a surgical image from the photographing unit; A detection unit for receiving a surgical image from the photographing unit and detecting a surgical tool in the surgical image; And a photographing position controller for controlling a photographing position of the photographing unit by using the detection result of the surgical tool of the detecting unit.

The apparatus may further include a command input unit for inputting a control command for controlling the photographing position of the photographing unit to the photographing position controller.

In this case, the photographing position controller may preferentially process a control command input by the command input unit.

Here, the command input unit may be a pedal.

Here, the surgical image taken by the imaging unit may be any one of a two-dimensional image and a three-dimensional image.

The detecting unit may generate image information including at least one of an RGB value and an HSI value representing pixel information of the surgical image using the surgical image, and using the generated image information. Distinguishing the surgical tool from a surgical image and inspecting the surgical tool on the image of the surgical tool by using the generated image information. .

Here, the step of separating the surgical tool from the surgical image by using the generated image information of the detection unit selecting a surgical tool from the surgical image and removing the background area from the surgical image It may be configured to include at least one of.

값 및

값 중의 어느 하나에 대하여 이진(binary)화를 수행하는 방법, 상기 RGB 값의 R, G 및 B 간의 차이값을 이용하여 이진(binary)화를 수행하는 방법 및 상기 RGB 값을 이용한 RGB 표준편차값(standard deviation)을 이용하여 이진(binary)화를 수행하는 방법 중 적어도 하나를 적용하여 상기 수술도구를 선택하는 것일 수 있다.Here, the step of selecting a surgical tool in the surgical image using the HSI value

Value and

A method of performing binarization on any one of the values, a method of performing binarization using difference values between R, G, and B of the RGB value, and an RGB standard deviation value using the RGB value It may be to select the surgical tool by applying at least one method of performing binarization using a standard deviation.

The removing of the background area from the surgical image may include skin color detection method using the RGB value, redish color detection method using the RGB value, R of the RGB value, The background area may be removed by applying at least one of a min max RGB comparison method of comparing the minimum value and the maximum value in G and B. FIG.

Here, the step of inspecting the surgical tool in the image divided by the surgical tool by using the generated image information, the separation (erosion), opening (opening) and It may be to use at least one of a method of applying at least one image processing of the closing (closing) and a method of applying a condition that the surgical tool passes the boundary of the surgical image.

Here, the photographing position controller may control the photographing position of the photographing unit so that the surgical tool is located inside the surgical image.

Surgical assistant system for performing a surgical tool detection method according to an embodiment of the present invention for achieving another object of the present invention described above, Surgical assistant system using a surgical image, a surgical tool for performing surgery; A photographing unit for generating a surgical image for the surgical tool and the surgical site; A display unit for receiving and outputting a surgical image from the photographing unit; A detection unit for receiving a surgical image from the photographing unit and detecting a surgical tool in the surgical image; And a photographing position controller for controlling a photographing position of the photographing unit by using the detection result of the surgical tool of the detecting unit.

The apparatus may further include a command input unit for inputting a control command for controlling the photographing position of the photographing unit to the photographing position controller.

According to the surgical tool detection method according to the invention, the recording medium recording the surgical tool detection method, the surgical system for performing the surgical tool detection method and the surgical auxiliary system for performing the surgical tool detection method, so that the surgical tool is located in the surgical image It can be controlled automatically and continuously tracked so that surgical instruments can be placed within the surgical image. As a result, the operator can perform surgery while minimizing further manipulation of the surgical image. In addition, since the surgical tool does not use additional markers such as to place the surgical tool within the surgical image, the convenience of disinfection for the surgical tool is increased and the influence of foreign substances on the surgical tool can be minimized. Can be maintained in an optimal state.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description.

However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. In the following description of the present invention, the same reference numerals are used for the same elements in the drawings and redundant descriptions of the same elements will be omitted.

3 is a flowchart illustrating a surgical tool detection method according to an embodiment of the present invention.

Referring to FIG. 3, in the method for detecting a surgical tool according to an embodiment of the present invention, in the method for detecting a surgical tool in a surgical image, using the surgical image, RGB representing pixel information of the surgical image is used. Generating image information comprising at least one of a value and an HSI value; Classifying the surgical tool in the surgical image by using the generated image information; And inspecting the surgical tool in an image of the surgical tool by using the generated image information.

First, the surgical image may be an image representing the surgical tool and the surgical site. In other words, the surgical image is an image representing the surgical tool and the surgical site, regardless of what kind of image is taken by the imaging device. In particular, the surgical image may be any one of a laparoscope image and an endoscope image, but the present invention is not limited thereto.

를 이용하여 계산될 수 있으며, RGB 값과 마찬가지로 수치 범위는 자유롭게 정할 수 있으나, 일반적으로 H는 0^o 내지 360^o의 각도(degree) 값으로 표현될 수 있으며, S는 하얀색이 100%인 0에서 하얀색이 전혀 섞이지 않은 1 사이의 값으로 표현될 수 있으며, I는 검은색인 0 내지 하얀색인 1의 값을 가지는 것으로 표현될 수 있다. 즉, H(색상)는 관측자에 의하여 인지되는 지배적인 색이고, S(채도)는 색의 순수도 또는 순수한 색이 흰 빛에 의해 희석된 정도이고, I(명도)는 색의 밝기의 척도에 해당한다. 더불어 상기 S에 대신하여 S를 수정한

을 사용할 수 있다.

은

으로 계산될 수 있을 것이다.Next, the generating of the image information including at least one of the RGB value and the HSI value representing the pixel information of the surgical image by using the surgical image (S310) using the provided surgical image RGB value Generating image information including the image information represented by or generating image information including the image information represented by the HSI value using the provided surgical image, or image information including both the RGB value and the HSI value Can be generated. Here, R is red, G is green, and B is blue, and each component is expressed numerically, and the numerical range can be freely defined. In general, it is normalized to 0 The numerical range may be set to have a value between 1 and 1, or may be set to have a value between 0 and 255. In addition, H represents Hue, S represents Saturation, and I represents Intensity. When the HSI value has an RGB value between 0 and 1,

It can be calculated using, and the numerical range can be set freely like the RGB value, but in general, H can be expressed as a degree value of 0 ^o to 360 ^o , and S is at 0 with 100% white color. White may be expressed as a value between 1 that is not mixed at all, and I may be expressed as having a value of 0, which is black, to 1, which is white. That is, H (color) is the dominant color perceived by the observer, S (saturation) is the purity of the color or the degree to which the pure color is diluted by white light, and I (brightness) is a measure of the brightness of the color. Corresponding. In addition to modifying S instead of S

Can be used.

silver

It can be calculated as

The generating of the image information (S310) may further include removing noise with respect to the surgical image. The noise may be removed by using blurring or by using a Gaussian filter or an averaging filter instead of blurring. In addition, it will be apparent to those skilled in the art that all image processing showing similar effects can be used.

Figure 4 is an exemplary view for explaining the RGB value and HSI value in the surgical tool detection method according to an embodiment of the present invention.

Referring to FIGS. 4 and 3 in parallel, according to the method for detecting a surgical tool according to an embodiment of the present invention, an image represented by an RGB value or an HSI value using a provided surgical image (FIG. 4A). Information can be generated. That is, using all of the RGB value or the HSI value can be represented on the same screen as the provided surgical image (FIG. 4 (a)), and if the surgical image received using only the H value among the HSI values is represented, Fig. 4 It may be represented by (b), and if the laparoscopic screen is expressed using only the S value, it may be represented by FIG. 4 (c).

Next, the step of classifying the surgical tool in the surgical image by using the generated image information (S320) selecting a surgical tool in the surgical image (S321) and the background area in the surgical image It may be configured to include at least one of the step (S322). That is, the step (S320) of distinguishing the surgical tool from the surgical image by using the generated image information may be configured to include only the step (S321) of selecting a surgical tool from the surgical image, the surgery It may be configured to include only the step (S322) to remove the background area from the image, and in addition to selecting a surgical tool in the surgical image (S321) and removing the background area from the surgical image (S322) It can be configured to include all.

값 및

값 중의 어느 하나에 대하여 이진(binary)화를 수행하는 방법을 적용하여 상기 수술도구를 선택하는 것일 수 있다.Here, the step (S321) of selecting a surgical tool in the surgical image using the HSI value

Value and

The surgical tool may be selected by applying a method of performing binarization on any one of the values.

값 및

값에 대하여 이진화를 수행하는 방법을 설명하기 위한 예시도이다.5 is a surgical tool detection method according to an embodiment of the present invention

Value and

An example diagram for explaining a method of performing binarization on a value.

값으로 표현되는 영상(도 5(a))을 고려해보면 수술도구를 제외한 배경구역은 흰색 구역으로 표현되는 것을 알 수 있고,

값으로 표현되는 영상(도 5(b))을 고려해보면 수술도구를 제외한 배경구역은 검은색 구역으로 표현되는 것을 알 수 있다. 이와 같은 특징을 고려하여 각 픽셀의

값을 그래프로 표현하면 도 5(c)와 같은 그래프를 구성할 수 있다. 이 경우에 수술도구는 0 내지 0.4 사이의

값을 가지며, 수술도구를 제외한 배경구역은 0.9 내지 1 사이의

값을 가짐을 알 수 있으므로, 상기 그래프(도 5(c))에서 수술도구를 제외한 구역은 높은

값을 갖는 특징을 이용하여, 두 개의 그룹으로 분리하기 위한 임계치를 찾아내는 이진화를 수행하는 방법을 적용할 수 있을 것이다. 이진화 방법은 다양하게 존재할 수 있음은 당업자에게 있어서 자명할 것이다. 여기에서는 이진화 방법 중의 하나인 오추 알고리즘(Otsu Algorithm)을 적용하여 도 5(c)에서 화살표로 표시되는 임계치(Threshold)를 얻을 수 있다. 상기 얻어진 임계치를 이용하여 수술도구를 선택할 수 있을 것이다. 도 5(d)는

영상에 대하여 이진화를 수행하는 방법을 적용한 것으로서,

영상이 임계치보다 큰 배경구역을 초록색으로 표현한 것이다.Referring to FIG. 5 and FIG. 3 in parallel, when the provided surgical image is expressed as an HSI value

Considering the image represented by the value (FIG. 5 (a)), it can be seen that the background area except the surgical tool is represented by the white area.

Considering the image represented by the value (FIG. 5 (b)), it can be seen that the background area except the surgical tool is represented by the black area. Considering these features, each pixel

If the value is expressed as a graph, a graph as shown in FIG. 5 (c) can be configured. In this case the surgical tool is between 0 and 0.4

Value and the background area excluding surgical instruments is between 0.9 and 1.

Since it can be seen that the value, the area except the surgical instruments in the graph (Fig. 5 (c)) is high

Using a feature with a value, one could apply a method of performing binarization to find a threshold for splitting into two groups. It will be apparent to those skilled in the art that there may be a variety of binarization methods. In this case, a threshold indicated by an arrow in FIG. 5C may be obtained by applying an Otsu algorithm, which is one of binarization methods. The obtained threshold may be used to select a surgical instrument. 5 (d) is

As a method of performing binarization on an image,

The background area where the image is larger than the threshold is expressed in green.

Here, in the selecting of the surgical tool in the surgical image (S321), the surgical tool is selected by applying a method of performing binarization using a difference value between R, G, and B of the RGB value. It may be.

6 is an exemplary view for explaining a method of performing binarization using difference values between R, G, and B in a surgical tool detection method according to an embodiment of the present invention.

값에 대하여 이진화를 수행한 결과를 이용하여 상기 수술용 영상에서 각 픽셀의 R, G 및 B 간의 차이값의 한 형태인 차이평균(

) 값을 색 분포를 이용하여 표현한 것이다. 그러나

값에 대하여 이진화를 수행한 결과를 이용하지 아니하고, 상기 이미지 정보를 생성하는 단계(S310)에서 생성된 이미지 정보를 이용하는 것도 가능함은 당업자에게 있어서 자명할 것이다. 붉은 색으로 표현된 영역은 RGB 값의 차이평균(

) 값이 큰 것을 나타내며, 파란 색으로 표현된 영역은 RGB 값의 차이평균(

) 값이 작은 것을 나타낸다. 즉, 수술도구 영역은 파란 색으로 표현되고 있는 것을 알 수 있다. RGB 값의 차이평균(

) 값을 계산하기 위하여,

와 같은 계산식을 적용하였다. 그러나 상기 RGB 값의 R, G 및 B 간의 차이값을 이용하는 방법에는 다양한 형태가 존재할 수 있으며, 상기 RGB 값의 R, G 및 B 간의 차이값을 이용하는 개념이 동일하다면, 당업자에게 있어서 R, G 및 B 간의 차이값을 이용하는 다양한 형태 는 모두 동일한 목적과 효과를 나타냄을 알 수 있을 것이다. 도 6(b)는 RGB 값의 차이평균(

) 값에 대하여 이진(binary)화를 수행하는 방법 중의 하나인 오추 알고리즘(Otsu Algorithm)을 적용한 그래프이다. 상술한 바와 같이 이진화를 수행하는 방법 중의 하나인 오추 알고리즘에 의하여 수술도구 영역을 구분하기 위해 최적의 임계치(Threshold)를 찾고 이를 경계로 하여 RGB 값의 차이평균(

) 값이 작은 영역인 파란 색으로 표현된 영역을 검출할 수 있을 것이다. 여기에서 이진화를 수행하는 방법은 오추 알고리즘이 아닌 다양한 방법도 가능할 것이다.Referring to Figure 6 and 3 in parallel, to use the general features of the surgical tool to detect the surgical tool, the end of the surgical tool is generally composed of a metal component, the bag of the surgical tool is composed of black Therefore, each of the R, G, and B values may use features having the same or similar values. 6 (a) is

Using the result of performing binarization on the value, the difference average, which is a form of difference value between R, G, and B of each pixel in the surgical image (

) Values are expressed using color distribution. But

It will be apparent to those skilled in the art that it is possible to use the image information generated in the step S310 of generating the image information without using the result of performing the binarization on the value. The red area shows the difference average of RGB values (

) Indicates a large value, and the area represented in blue is the difference average of the RGB values (

) Indicates a small value. In other words, it can be seen that the surgical tool region is represented in blue. Mean difference of RGB values (

) To calculate the value,

The following formula was applied. However, various methods may exist in a method of using the difference between R, G, and B of the RGB values, and if the concept of using the difference between R, G, and B of the RGB values is the same, those skilled in the art will appreciate R, G, and It will be appreciated that the various forms using the difference between B all serve the same purpose and effect. 6 (b) shows the difference average of RGB values (

This graph is applied to Otsu Algorithm, which is one of the methods of performing binarization on. As described above, the optimal threshold is found to distinguish the surgical instrument region by the Ochu algorithm, which is one of the methods for performing binarization, and the difference average of RGB values is defined as the boundary.

The area represented by the blue color, which is a small value of), may be detected. Here, the binarization may be performed in various ways other than the Otsu algorithm.

Here, in the selecting of the surgical tool in the surgical image (S321), the surgical tool is applied by applying a method of performing binarization using an RGB standard deviation value using the RGB value. It may be to choose.

FIG. 7 is an exemplary diagram for describing a method of performing binarization using a standard deviation of RGB in a surgical tool detection method according to an exemplary embodiment of the present invention.

)과 최대 RGB 표준편차 값(

) 간의 비율값(

)을 색 분포를 이용하여 표현한 것이다. 붉은 색으로 표현된 영역은 비율값(

)이 큰 것을 나타내며, 파란 색으로 표현된 영역은 비율값(

)이 작은 것을 나타낸다. 즉, 수술도구 영역은 파란 색으로 표현되고 있는 것을 알 수 있다. RGB 표준편차값(

)은

로 계산되며, 최대 RGB 표준편차값(

)은 상기 수술용 영상 내에서 가장 큰 RGB 표준편차값(

)을 말하는 것이다.Referring to FIG. 7 and FIG. 3 in parallel, in order to detect a surgical tool as described above, each of the R, G and B values of the surgical tool uses a feature having the same or similar values, as shown in FIG. ) Is an RGB standard deviation value of each pixel in the surgical image using the generated image information.

) And the maximum RGB standard deviation value (

Ratio between

) Is expressed using color distribution. The area represented in red is the ratio value (

) Is large, and the area shown in blue is the ratio value (

) Is small. In other words, it can be seen that the surgical tool region is represented in blue. RGB standard deviation value (

)silver

Calculated using the maximum RGB standard deviation (

) Is the largest RGB standard deviation value () in the surgical image.

).

)에 대하여 이진(binary)화를 수행하는 방법 을 적용한 것으로서, 비율값(

)이 임계치보다 큰 배경영역을 초록색으로 표현한 것이다. 도 7(c)는 이진화를 수행한 후의 결과를 RGB 표준편차값(standard deviation)과 최대 RGB 표준편차값 간의 비율값(

)으로 다시 표현한 것으로 임계치를 0.5로 설정하여 이 진화를 수행한 후의 결과이다. 즉, RGB 표준편차값(standard deviation)과 최대 RGB 표준편차값 간의 비율값(

)에 대하여 이진(binary)화를 수행하면 수술용 영상에서 수술도구를 효율적으로 선택할 수 있을 것이다. 여기에서, 상기 임계치로 0.5가 아닌 다른 값을 적용할 수 있음은 자명할 것이다.7 (b) shows the ratio value between the RGB standard deviation value and the maximum RGB standard deviation value (

The method of performing binarization on) is applied.

) Represents the background area larger than the threshold in green. 7 (c) shows the result after performing binarization and shows the ratio value between the RGB standard deviation value and the maximum RGB standard deviation value (

This is the result after this evolution with the threshold set to 0.5. That is, the ratio between the RGB standard deviation and the maximum RGB standard deviation value (

Performing binarization on) will allow efficient selection of surgical instruments in surgical images. Here, it will be apparent that a value other than 0.5 may be applied as the threshold.

값을 이용할 수도 있을 것이다.

값은

으로 계산될 수 있을 것이다. 상술한 RGB 표준편차값을 이용하여 이진화하는 방법 외에도 다양하게 변형된 RGB 표준편차값을 이용하는 이진화 방법이 있음은 당업자에게 있어서 자명할 것이다.In addition, a method of performing binarization using an RGB standard deviation may be performed by adding a ratio value between an RGB standard deviation value and a maximum RGB standard deviation value and an S value in the HSI value. It may be to perform binarization (binary) for. This is to perform the binarization by additionally adding the S value in the above-described method, so that the difference between the surgical tool zone and the background zone can be made larger, so that the surgical tool zone and the background zone can be more clearly distinguished through the binarization. Instead of using the S value here,

You can also use the value.

The value is

It can be calculated as It will be apparent to those skilled in the art that there are binarization methods using various modified RGB standard deviation values in addition to the binarization method using the above-described RGB standard deviation values.

Here, the step of removing the background area from the surgical image (S322) may be to remove the background area by applying a skin color detection method using the RGB value.

8 is an exemplary view for explaining a skin color detection method in the surgical tool detection method according to an embodiment of the present invention.

의 조건을 만족하는 픽셀을 분홍색 영역으로 표현한 것이다.("A Survey on Pixel-Based Skin Color Detection Techniques", by Vladimir Vezhnevets, Vassili Sazonov, Alla Andreeva) 즉, 수술도구를 제외한 배경구역은 피부색 검출 방법을 이용하여 제거할 수 있을 것이다. 더불어, 상기 수술용 영상에서 수술도구를 선택하는 단계(S321)를 적용하는 것과 상기 RGB 값을 이용하는 피부색 검출(Skin Color Detection) 방법을 적용하는 것을 동시에 적용하면 보다 향상된 수술도구를 제외한 배경구역 제거의 효과를 거둘 수 있을 것이다.Referring to FIG. 8 and FIG. 3 in parallel, the provided surgical image may be divided into a surgical tool zone and a background zone except for the surgical tool, and in this case, the background zone except for the surgical tool may be represented by a skin color. Able to know. Therefore, it can be seen that applying the skin color detection method to the provided surgical image can remove the background area except for the surgical tool. 8 (a) represents an image to which the step (S321) of selecting a surgical tool in the surgical image is applied. However, it may be possible to use an image that does not apply the step (S321) of selecting the surgical tool from the surgical image first. FIG. 8 (b) shows a skin color detection method in which pink is expressed as a skin color using a method of detecting skin color by setting a range of RGB values representing skin color in general with respect to RGB values.

Pixels that satisfy the conditions of the image are expressed as pink areas ("A Survey on Pixel-Based Skin Color Detection Techniques", by Vladimir Vezhnevets, Vassili Sazonov, Alla Andreeva). Can be removed. In addition, applying the step of selecting a surgical tool in the surgical image (S321) and applying the skin color detection method using the RGB value at the same time to remove the background area except for improved surgical tools It will work.

Here, the step of removing the background area from the surgical image (S322) may be to remove the background area by applying a red color detection method using the RGB value.

9 is an exemplary view for explaining a red color detection method in the surgical tool detection method according to an embodiment of the present invention.

9 and 3, the red color detection method has a background theory similar to the skin color detection method. In the surgical images, most of the background areas except for the surgical instruments consist of organs (pink), blood (red), and fat (yellow) on the surface of organs. These three colors indicate that R values are much higher than G and B values. There is a characteristic.

의 조건을 만족하는 픽셀을 분홍 색 영역으로 표현한 것이다. 즉, 상기와 같은 붉은색 검출 방법을 만족할 경우도 수술도구 구역이 아닌 배경구역이라고 분리할 수 있을 것이다. 상기 수식 조건에서 2배의 경우를 이용하였으나, 2배의 경우에 한정되는 것은 아니므로 다양한 값이 이용될 수 있을 것이다.9 (a) represents an image to which the step (S321) of selecting a surgical tool in the surgical image is applied. However, it may be possible to use an image that does not apply the step (S321) of selecting the surgical tool from the surgical image first. FIG. 9 (b) shows a red color detection method in which a red color detection area is expressed in pink by setting a range of RGB values representing fats of organs, blood, and organ surface with respect to RGB values.

The pixel that satisfies the condition is expressed in pink area. That is, even if the red detection method as described above is satisfied, it may be separated into a background region rather than a surgical instrument region. Although the case of 2 times is used in the above formula condition, various values may be used since the case is not limited to 2 times.

Here, the step of removing the background area from the surgical image (S322) is a background area by applying a min max RGB Comparison method that compares the minimum value and the maximum value in the R, G and B of the RGB value It may be to remove the.

10 is an exemplary view for explaining a minimum maximum comparison method for comparing the minimum value and the maximum value in the R, G and B values in the surgical tool detection method according to an embodiment of the present invention.

값과

값은 서로 비슷한 값을 가진다는 특징을 이용하는 것이다. 도 10(a)는 상기 수술용 영상에서 수술도구를 선택하는 단계(S321)를 적용한 영상을 표현하고 있다. 그러나 1차적으로 상기 수술용 영상에서 수술도구를 선택하는 단계(S321)를 적용하지 아니한 영상을 이용할 수도 있을 것이다. 도 10(b)은 상기 RGB 값의 R, G 및 B에서 최소값과 최대값을 비교하는 최소 최대 비교(min max RGB Comparison) 방법을 적용한 것으로서

인 경우에는 분홍색으로 표현하고 있다. 결국, 분홍색으로 표현된 영역은 배경구역으로 판단할 수 있는 것이다. 또한, 여기에서,

값이

값의 2배 이상인 영역을 배경구역으로 판단하였으나, 배율은 상황에 따라 다르게 적용할 수 있을 것이다.Referring to FIG. 10 and FIG. 3 in parallel, in order to detect a surgical tool as described above, each of the R, G and B values of the surgical tool uses features having the same or similar values to each other. Of belonging pixels

Value and

Values use the feature that they have similar values. 10 (a) represents an image to which the step (S321) of selecting a surgical tool in the surgical image is applied. However, it may be possible to use an image that does not apply the step (S321) of selecting the surgical tool from the surgical image first. FIG. 10 (b) illustrates a method of applying a min max RGB comparison method for comparing the minimum and maximum values of R, G, and B of the RGB values.

If is expressed in pink. After all, the area represented by pink can be judged as the background area. Also, here,

Value is

The area that is more than twice the value is determined as the background area, but the magnification may be applied differently depending on the situation.

Next, the step (S330) of inspecting the surgical tool in the image divided by the surgical tool by using the generated image information (dilation), erosion, opening ( At least one of the method of applying the image processing of at least one of the opening (closing) and the closing (closing) and the method of applying a condition that crosses the boundary of the surgical image may be used.

Here, the method of applying at least one image processing of dilation, erosion, opening and closing to the image divided by the surgical instruments using the generated image information After applying the step S320 of distinguishing the surgical tool from the surgical image may be to apply an image processing to compensate for errors that may exist in the surgical tool zone.

11 is an exemplary view for explaining image processing such as expansion, erosion, opening and closing in the surgical tool detection method according to an embodiment of the present invention.

Referring to FIG. 11, FIG. 11 (a) represents a surgical tool region in white as a binarized image by the step (S320) of distinguishing the surgical tool from the surgical image by using the generated image information. This is a black image of the background area except for surgical instruments. It can be seen from the image that an error due to some noise or the like has not yet been removed. By using at least one image processing among dilation, erosion, opening and closing in an image as shown in FIG. 11 (a), an image as shown in FIG. 11 (b) may be obtained. . In addition, the image processing method of filling a hole (hole) generated in the region of the surgical tool may be used for the image processing. It will be appreciated that surgical tools can be detected without error by applying such additional image processing.

Here, the method for applying the condition that the surgical tool crosses the boundary of the surgical image is a surgical tool zone after applying the step (S320) to distinguish the surgical tool from the surgical image using the generated image information It may be to apply a process to compensate for errors that may exist in. In other words, the surgical tool is inevitably used to detect the surgical tool without error by using a feature existing to the outside of the surgical image.

In addition, the surgical tool was detected in the step S330 of inspecting the surgical tool from the image of the surgical tool using the generated image information, but not significantly smaller than the size of the general surgical tool. Another method may be applied.

12 is an exemplary view for explaining a result of performing a surgical tool detection method according to an embodiment of the present invention.

Referring to FIG. 12, a result of performing a surgical tool detection method according to an exemplary embodiment of the present invention with respect to the surgical image provided in FIG. 12 (a) is represented in FIG. 12 (b). In Figure 12 (b) it can be seen that the boundary of the surgical tool is represented by a red line, which represents the position of the surgical tool detection according to the result of performing the surgical tool detection method. That is, the results of performing the surgical tool detection method on the provided surgical image of Figure 12 (a) is shown in Figure 12 (b) at the same time. After all, according to the surgical tool detection method according to the present invention is to inform that the surgical tool can be detected accurately.

13 is a block diagram illustrating a surgical system according to an embodiment of the present invention.

Referring to Figure 13, the surgical system according to an embodiment of the present invention is a surgical system using a surgical image, a surgical device unit including a surgical tool and a surgical tool controller for controlling the surgical tool; A photographing unit for generating a surgical image for the surgical tool and the surgical site; A display unit for receiving and outputting a surgical image from the photographing unit; A detection unit for receiving a surgical image from the photographing unit and detecting a surgical tool in the surgical image; And a photographing position controller for controlling a photographing position of the photographing unit by using the detection result of the surgical tool of the detecting unit.

First, a surgical device unit 1310 including a surgical tool and a surgical tool controller for controlling the surgical tool receives a command for controlling the surgical tool by the operator, and transfers the command to the surgical tool to the operator's intention. It may be a device that can perform the operation according to. It will be apparent to those skilled in the art that the surgical tool and the surgical tool control unit for controlling the surgical tool are generally configured and additional components may be included.

Next, the photographing unit 1320 for generating a surgical image for the surgical tool and the surgical site may be configured to include a lens and a light that can emit light to photograph the surgical tool and the surgical site. That is, when the light shines at the desired position through the illumination, it will be possible to obtain a surgical image of the surgical tool and the surgical site by using the light reflected from the desired position. In addition, the surgical image photographed by the photographing unit 1320 may be any one of a two-dimensional image and a three-dimensional image.

Next, the display unit 1330 for receiving and outputting a surgical image from the photographing unit 1320 may provide a device such as a monitor so that the operator may recognize the surgical image received through the lens of the photographing unit. It may be a device for outputting through. In addition to the monitor may be a display device consisting of a screen.

Next, a detection unit 1340 for receiving a surgical image from the photographing unit 1320 and detecting a surgical tool in the surgical image is an RGB for representing pixel information of the surgical image by using the surgical image. Generating image information including at least one of a value and an HSI value, separating the surgical tool from the surgical image using the generated image information, and generating the surgical tool using the generated image information. It may be to perform a surgical tool detection method comprising the step of examining the surgical tool in the divided image.

Generating image information including at least one of an RGB value and an HSI value representing pixel information of the surgical image by using the surgical image of the detection unit may be represented by an RGB value using the provided surgical image. Generate image information including image information, generate image information including image information expressed as HSI values using the provided surgical image, or generate image information including both the RGB value and the HSI value. have. The generating of the image information may further include removing noise with respect to the surgical image. The noise may be removed by using blurring or by using a Gaussian filter or an averaging filter instead of blurring. In addition, it will be apparent to those skilled in the art that all image processing showing similar effects can be used.

값 및

값 중의 어느 하나에 대하여 이진(binary)화를 수행하는 방법, 상기 RGB 값의 R, G 및 B 간의 차이값을 이용하여 이진(binary)화를 수행하는 방법 및 상기 RGB 값을 이용한 RGB 표준편차값(standard deviation)을 이용하여 이진(binary)화를 수행하는 방법 중 적어도 하나를 적용하 여 상기 수술도구를 선택하는 것일 수 있다. 다음으로, 상기 수술용 영상에서 배경구역을 제거하는 단계는 상기 RGB 값을 이용하는 피부색 검출(Skin Color Detection) 방법, 상기 RGB 값을 이용하는 붉은색 검출(Reddish Color Detection) 방법 및 상기 RGB 값의 R, G 및 B에서 최소값과 최대값을 비교하는 최소 최대 비교(min max RGB Comparison) 방법 중 적어도 하나를 적용하여 배경구역을 제거하는 것일 수 있다.The step of distinguishing the surgical tool from the surgical image by using the generated image information of the detection unit is at least one of selecting a surgical tool from the surgical image and removing the background area from the surgical image It may be configured to include. First, selecting a surgical tool in the surgical image using the HSI value

Value and

A method of performing binarization on any one of the values, a method of performing binarization using difference values between R, G, and B of the RGB value, and an RGB standard deviation value using the RGB value It may be to select the surgical tool by applying at least one method of performing binarization using a standard deviation. Next, the step of removing the background area from the surgical image may include a skin color detection method using the RGB value, a red color detection method using the RGB value, R of the RGB value, The background area may be removed by applying at least one of a min max RGB comparison method of comparing the minimum value and the maximum value in G and B. FIG.

Examining the surgical tool in the image divided by the surgical tool using the generated image information, the separation (erosion), opening (opening) and closing (closing) And at least one of a method of applying at least one image processing and a method of applying a condition that the surgical tool crosses a boundary of the surgical image.

Next, the photographing position controller 1350 for controlling the photographing position of the photographing unit 1320 using the surgical tool detection result of the detector 1340 is such that the surgical tool is located inside the surgical image. It may be to control the photographing position of the photographing unit. In general, the surgical tool will be located in the center of the surgical image, if there are two or more surgical tools will be able to control the average value of the surgical tool position to be located in the center of the surgical image, the surgical tool is a surgical image You might want to control it so that it's in a specific place instead of the center of the.

Referring back to FIG. 13, the surgical system according to an exemplary embodiment may further include a command input unit 1360 for inputting a control command for controlling the photographing position of the photographing unit to the photographing position controller. .

The command input unit 1360 for inputting a control command for controlling the photographing position of the photographing unit 1320 to the photographing position controller 1350 may operate in the surgical image detected according to the surgical tool detecting method described above. In automatically adjusting the position of the tool, it may be a command input device that can be manually adjusted when the operator wants to change it or make additional fine adjustments. The command input unit 1360 may be a pedal which can be adjusted by the operator's foot, may be a device that receives a command by recognizing the operator's voice, or may be a device that receives a command by recognizing the operator's time. In addition, the photographing position controller 1350 may preferentially process a control command input by the command input unit 1360. This is to give priority to the operator's manual control because the operator wants to perform additional control despite the automatic control.

14 is a block diagram illustrating a surgical assistance system according to an embodiment of the present invention.

Referring to FIG. 14, a surgical assistant system according to an embodiment of the present invention includes a surgical tool for performing a surgery in a surgical assistant system using a surgical image; A photographing unit for generating a surgical image for the surgical tool and the surgical site; A display unit for receiving and outputting a surgical image from the photographing unit; A detection unit for receiving a surgical image from the photographing unit and detecting a surgical tool in the surgical image; And a photographing position control unit for controlling a photographing position of the photographing unit by using the detection result of the surgical tool of the detecting unit.

The apparatus may further include a command input unit for inputting a control command for controlling the photographing position of the photographing unit to the photographing position controller.

Here, a photographing unit 1420 for generating a surgical image for the surgical tool and the surgical site, a display unit 1430 for receiving and outputting a surgical image from the photographing unit, the surgical image from the photographing unit A detection unit 1440 for detecting a surgical tool in the surgical image, a photographing position controller 1450 for controlling a photographing position of the photographing unit using the detection result of the surgical tool of the detecting unit, and photographing of the photographing unit The command input unit 1460 for inputting a control command for controlling the position to the photographing position controller is as described above.

The surgical assistant system 1400 is a surgical assistant robot in place of the surgical assistant to fix the surgical image and control the position of the surgical image according to the operator's command, compared to the surgical system, the operator directly operating tools (1410) The difference is that surgery is performed by controlling). The surgical tool 1410 for performing the surgery may be a tool such as a laparoscopy mes., A laparoscopy hook, a laparoscopic pincette, a laparoscopy scissor, or the like.

Although described with reference to the above embodiments, those skilled in the art can understand that the present invention can be variously modified and changed without departing from the spirit and scope of the invention described in the claims below. There will be.

1 is a conceptual diagram of a surgical system using a laparoscope incorporating a conventional surgical robot.

2 is a conceptual diagram of a surgical assistance system using a laparoscope incorporating a conventional surgical assistant robot.

3 is a flowchart illustrating a surgical tool detection method according to an embodiment of the present invention.

Figure 4 is an exemplary view for explaining the RGB value and HSI value in the surgical tool detection method according to an embodiment of the present invention.

값 및

값에 대하여 이진화를 수행하는 방법을 설명하기 위한 예시도이다.5 is a surgical tool detection method according to an embodiment of the present invention

Value and

An example diagram for explaining a method of performing binarization on a value.

6 is an exemplary view for explaining a method of performing binarization using a difference value between an R value, a G value, and a B value in a surgical tool detection method according to an exemplary embodiment of the present invention.

7 is an exemplary view for explaining a method for performing binarization using an RGB standard deviation value in a surgical tool detection method according to an embodiment of the present invention.

8 is an exemplary view for explaining a skin color detection method in the surgical tool detection method according to an embodiment of the present invention.

9 is an exemplary view for explaining a red color detection method in the surgical tool detection method according to an embodiment of the present invention.

10 is an exemplary view for explaining a minimum maximum comparison method for comparing the minimum value and the maximum value in the R, G and B values in the surgical tool detection method according to an embodiment of the present invention.

11 is an exemplary view for explaining image processing such as expansion, erosion, opening and closing in the surgical tool detection method according to an embodiment of the present invention.

12 is an exemplary view for explaining a result of performing a surgical tool detection method according to an embodiment of the present invention.

13 is a block diagram illustrating a surgical system according to an embodiment of the present invention.

14 is a block diagram illustrating a surgical assistance system according to an embodiment of the present invention.

Description of the Related Art

1310: surgical device 1410: surgical instruments

1320 and 1420: photographing unit 1330 and 1430: display unit

1340 and 1440: detection unit 1350 and 1450: shooting position control unit

1360, 1460: command input

Claims (22)

In the method for detecting a surgical tool in the surgical image, Generating image information including at least one of an RGB value and an HSI value representing pixel information of the surgical image using the surgical image; Classifying the surgical tool in the surgical image by using the generated image information; And And inspecting the surgical tool in an image of the surgical tool by using the generated image information. The method of claim 1, The surgical image is a surgical tool detection method, characterized in that the image representing the surgical tool and the surgical site. The method of claim 2, The surgical image is a surgical tool detection method, characterized in that any one of a laparoscope image and an endoscope image. The method of claim 1, The generating of the image information may further include removing noise from the surgical image. The method of claim 1, The step of classifying the surgical tool in the surgical image by using the generated image information includes selecting at least one surgical tool from the surgical image and removing the background area from the surgical image; Surgical tool detection method, characterized in that configured. The method of claim 5, Selecting a surgical tool in the surgical image using the HSI value

Value and

A method of performing binarization on any one of the values, a method of performing binarization using difference values between R, G, and B of the RGB value, and an RGB standard deviation value using the RGB value A surgical tool detection method comprising selecting the surgical tool by applying at least one method of performing binarization using a standard deviation. The method of claim 5, The removing of the background area from the surgical image may include a skin color detection method using the RGB value, a red color detection method using the RGB value, and R, G, and the RGB value. The method of detecting a surgical tool, characterized in that to remove the background area by applying at least one of the min max RGB Comparison method comparing the minimum value and the maximum value in B. The method of claim 1, Examining the surgical tool in the image divided by the surgical tool using the generated image information, the separation (erosion), opening (opening) and closing (closing) And at least one of a method of applying at least one image processing and a method of applying a condition that the surgical tool crosses a boundary of the surgical image. In a recording medium recording a method for detecting a surgical tool in the surgical image, Generating image information including at least one of an RGB value and an HSI value representing pixel information of the surgical image using the surgical image; Classifying the surgical tool in the surgical image by using the generated image information; And And inspecting the surgical tool in the image of the surgical tool by using the generated image information. In a surgical system using a surgical image, A surgical device unit including a surgical tool and a surgical tool controller for controlling the surgical tool; A photographing unit for generating a surgical image for the surgical tool and the surgical site; A display unit for receiving and outputting a surgical image from the photographing unit; A detection unit for receiving a surgical image from the photographing unit and detecting a surgical tool in the surgical image; And And a photographing position control unit for controlling a photographing position of the photographing unit by using the detection result of the surgical tool of the detecting unit. The method of claim 10, And a command input unit for inputting a control command for controlling a photographing position of the photographing unit to the photographing position controller. The method of claim 11, And the photographing position controller preferentially processes a control command input by the command input unit. The method of claim 11, And the command input unit is a pedal. The method of claim 10, Surgical image taken by the imaging unit is a surgical system, characterized in that any one of a two-dimensional image and a three-dimensional image. The method of claim 10, The detection unit generating image information including at least one of an RGB value and an HSI value representing pixel information of the surgical image by using the surgical image, and using the generated image information The surgical system characterized in that for performing a surgical tool detection method comprising the step of distinguishing the surgical tool and the surgical tool in the image divided by the surgical tool using the generated image information . The method of claim 15, The step of distinguishing the surgical tool from the surgical image by using the generated image information of the detection unit is at least one of selecting a surgical tool from the surgical image and removing the background area from the surgical image Surgical system, characterized in that comprises a. The method of claim 16, Selecting a surgical tool in the surgical image using the HSI value

Value and

A method of performing binarization on any one of the values, a method of performing binarization using difference values between R, G, and B of the RGB value, and an RGB standard deviation value using the RGB value A surgical system, characterized in that for selecting the surgical instrument by applying at least one method of performing a binarization (binary) using a (standard deviation). The method of claim 16, The step of removing the background area from the surgical image includes a skin color detection method using the RGB value, a red color detection method using the RGB value, and R, G, and B of the RGB value. A surgical system comprising removing a background area by applying at least one of a min max RGB Comparison method that compares a minimum value with a maximum value. The method of claim 15, Examining the surgical tool in the image divided by the surgical tool using the generated image information, the separation (erosion), opening (opening) and closing (closing) And at least one of a method of applying at least one image processing and a method of applying a condition that the surgical tool crosses a boundary of the surgical image. The method of claim 10, And the photographing position controller controls the photographing position of the photographing unit so that the surgical tool is positioned inside the surgical image. In the surgical assistance system using the surgical image, Surgical instruments for performing surgery; A photographing unit for generating a surgical image for the surgical tool and the surgical site; A display unit for receiving and outputting a surgical image from the photographing unit; A detection unit for receiving a surgical image from the photographing unit and detecting a surgical tool in the surgical image; And And a photographing position controller for controlling a photographing position of the photographing unit by using the detection result of the surgical tool of the detecting unit. 21. The method of claim 20, And a command input unit for inputting a control command for controlling a photographing position of the photographing unit to the photographing position controller.

Images