KR20200016694A - Image quality assessment and automatic extraction system of endoscope in minimally invasive surgery, and Image quality assessment and automatic extraction method of endoscope using the same - Google Patents

Abstract

The present invention relates to: an endoscopic image quality evaluation and automatic extraction system in minimally invasive surgery; and an endoscopic image quality evaluation and automatic extraction method using the same. Specifically, the present invention relates to: an endoscopic image quality evaluation and automatic extraction system in minimally invasive surgery which evaluates the quality of the endoscope image during robotic surgery to automatically extract the endoscope outside the surgical space without collision with biological major elements after undergoing confirmation of a surgeon for efficiency and safety of operation when it is determined that an endoscopic lens needs to be cleaned; and an endoscopic image quality evaluation and automatic extraction method using the same.

Description

Image quality assessment and automatic extraction system of endoscope in minimally invasive surgery, and image quality assessment and automatic extraction method of endoscope using the same}

The present invention relates to an endoscopic image quality evaluation and automatic extraction system in minimally invasive surgery, and to an endoscopic image quality evaluation and automatic extraction method using the same. Endoscopic image quality evaluation and automatic extraction system in minimally invasive surgery that automatically extracts the endoscope to the outside of the surgical space without colliding with major living body elements after confirming the chief physician for the efficiency and safety of the surgery if deemed necessary. The present invention relates to an endoscope image quality judgment and automatic extraction method using the same.

Laparoscopy refers to an endoscope for examining the gallbladder, peritoneum, liver, and the outside of the digestive tract by inserting a small hole in the abdominal wall and inserting it. It is a surgery that is performed through one or several holes rather than surgery.

In laparoscopic surgery, two or more small holes with a diameter of 5 mm to 1 cm are inserted into the abdomen, and the necessary end-effector is inserted through the holes, and the shape of the body taken by the laparoscope is recorded. This operation is performed while looking at a screen through a monitor.

In addition, robot surgery refers to a surgery in which a doctor controls a robot that can move a surgical tool. Laparoscopic robot surgery is a hole that is drilled after drilling a plurality of holes in the abdomen and a surgical tool for robot surgery along with a laparoscope. It means to operate by inserting robot arm of surgical robot with -effector attached.

In the case of laparoscopic robotic surgery, there is less incision area than conventional open surgery, so the patient's recovery time is faster, pain and scars are reduced, as well as more precise surgery is possible because the surgery is performed using the robot arm. The surgery is performed using three surgical instruments and one endoscope.

At present, a minimally invasive surgical technique, such as robotic surgery, is expanded by injecting carbon dioxide gas into a surgical site (such as the abdominal cavity) for the use of laparoscopic endoscopes and surgical instruments to secure a surgical space for laparoscopic surgery.

However, due to the narrow operating space, the focus and quality of the endoscope image are often degraded to the extent that the endoscopic image is not suitable for the surgical procedure due to water vapor and foreign substances generated during the surgical procedure. And the nurse manually removes the endoscope to the outside there is a problem that you have to re-enter after cleaning the endoscope.

In particular, the determination of the quality of the surgical image is subjective, and because of the hassle of washing, the operation is sometimes performed using an uncomfortable image. Also, the skill of the nurse is important because the endoscope is manually extracted and washed. During the detachment process, there is a concern that the endoscope may collide with vital organs such as organs, blood vessels including arteries and veins, and nerves existing in the surgical space.

The present invention is to solve the above problems, when evaluating the quality of the endoscope image during robotic surgery, if it is determined that the cleaning of the endoscope lens is necessary, the endoscope can be automatically extracted to the outside of the surgical space after confirmation of the collection It is an object of the present invention to provide an endoscopy image quality evaluation and automatic extraction system in invasive surgery and an endoscopy image quality evaluation and automatic extraction method using the same.

In addition, the present invention is an endoscopy image quality evaluation and automatic extraction system in the minimally invasive surgery that can prevent collision with the main body elements when extracting the endoscope outside the surgical space, and the endoscopy image quality evaluation and automatic extraction method using the same The purpose is to provide.

Endoscopic image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention, the endoscope is inserted into the inflated operating space before performing the robot surgery endoscope for real-time imaging of the surgical site during the robot surgery; An endoscope management image acquisition unit for acquiring an image of a surgical site photographed by the endoscope; An endoscope management quality evaluation unit configured to evaluate a quality of an image acquired by the endoscope management award acquirer; A signal generator for generating a predetermined signal when the quality of the acquired image is lowered to a predetermined level or less as a result of image quality evaluation by the endoscopy management quality evaluation unit; Acquisition consent unit for acquiring the consent of the concentrator for washing the endoscope when a signal is generated in the signal generator; And an endoscope automatic extraction unit for automatically extracting the endoscope to the outside of the surgical space when acquiring consent of the endograph for washing the endoscope.

In addition, the endoscopic image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention, the endoscopy quality evaluation unit quantifies the quality of the acquired image, the signal generation unit of the digitized image If the quality value is less than or equal to a predetermined value, a signal may be generated.

In addition, in the endoscopic image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention, the signal generator may generate different signals according to the range of the quality value of the digitized image.

In addition, the endoscope image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention, the endoscope management acquisition unit obtains the surgical site image taken by the endoscope at predetermined intervals, the endoscopy management quality The evaluation unit may quantify the quality of the obtained image by comparing the image with the image obtained immediately before the acquired image.

In addition, the endoscope image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention, the endoscope management acquisition unit obtains the surgical site image taken by the endoscope at predetermined intervals, the endoscopy management quality The evaluation unit digitizes the image using the quality of the acquired image, and the digitization may be performed by matching image elements including whether foreign matter is attached, fog, brightness, and clarity.

In addition, the endoscope image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention, the endoscope automatic extraction unit, the endoscope using the 3D image of the surgical space in which the main body of the living body is displayed An endoscope extraction path acquisition unit for obtaining an endoscope extraction movement path for extracting to the outside of the surgical space without collision with the main body of the living body, and an endoscope extraction unit for extracting the endoscope to the outside of the surgical space along the obtained endoscope extraction movement path It may include.

In addition, the endoscope image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention, the endoscope automatic extraction unit collision of the endoscope extracted from the main body and the outside of the surgical space displayed on the 3D image Further comprising: a collision prevention unit for preventing the collision, the collision prevention unit, the endoscope position extraction unit for extracting the real-time position of the endoscope; The endoscope is extracted to the outside of the main body and the surgical space by using the position information of the main body of the main body in the 3D image and the real-time position information of the endoscope in the 3D image extracted by the endoscope position extraction unit A collision risk determination unit for determining a collision risk; A risk signal generator for generating a danger signal when it is determined that the collision risk determining unit determines that the main body and the endoscope extracted outside the surgical space have a collision risk; And when the collision risk determination unit determines that there is a collision risk between the main body of the body and the endoscope extracted outside the surgery space, the collision controlling the endoscope extraction unit to prevent the collision between the main body of the body and the endoscope is prevented. Prevention control unit; may include.

In addition, the endoscope image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention, the collision risk determination unit is the distance from the real-time position of the main body and the endoscope extracted to the outside of the surgical space Calculate, determine whether the calculated distance is within a preset range, and if the calculated distance is within a preset range, the dangerous signal generator generates a dangerous signal, and the collision avoidance controller is the main body of the living body. The endoscope extraction unit may be controlled to prevent collision between the endoscope and the endoscope.

In addition, the endoscopic image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention, the endoscope image quality evaluation and automatic extraction system, the 3D image of the surgical space in which the main elements of the living body is displayed A 3D image processing unit may further include a 3D image processing unit. The 3D image processing unit may include: an image acquisition unit configured to acquire a plurality of 2D or 3D images photographed with respect to the inflated surgical space before performing robot surgery; A 3D image registration unit for matching a plurality of 2D or 3D images acquired by the image acquisition unit into one 3D image representing the inflated surgical space; A biological main element extracting unit extracting a biological main element included in the matched 3D image; And a 3D image generating unit generating a 3D image of the inflated surgical space in which the biometric main elements are displayed by displaying the biometric main elements extracted by the biometric main element extractor on the matched 3D image.

On the other hand, endoscopic image quality evaluation and automatic extraction method in the minimally invasive surgery according to an embodiment of the present invention, the endoscopic management photographing step of inserting into the inflated surgery space before performing the robot surgery to take a real-time imaging of the surgical site during the robot surgery; An endoscope management acquisition step of acquiring an image of a surgical site photographed in the endoscopy management step; An endoscope management quality determining step of determining a quality of an image acquired in the endoscope management acquisition step; A signal generation step of generating a predetermined signal when the quality of the acquired image is lowered to a predetermined level or less as a result of the image quality determination in the endoscope management quality determination step; Acquiring the consent of the concentrator for the endoscope cleaning after the signal generation step; And an endoscope automatic extraction step of automatically extracting the endoscope to the outside of the surgical space when acquiring the consent of the concentrator for washing the endoscope in the acquiring motion of the congestion.

In addition, the endoscopic image quality evaluation and automatic extraction method in the minimally invasive surgery according to an embodiment of the present invention, the endoscopy quality determination step is to quantify the quality of the acquired image to determine the degree of degradation of the acquired image The signal generating step may generate a signal when the quality value of the digitized image is less than or equal to a predetermined value.

In addition, in the method of endoscopic image quality evaluation and automatic extraction in minimally invasive surgery according to an embodiment of the present invention, the signal generation step may generate different signals according to a range of quality values of the digitized image.

In addition, the endoscope image quality evaluation and automatic extraction method in the minimally invasive surgery according to an embodiment of the present invention, the endoscope automatic extraction step, the endoscope using a 3D image of the surgical space in which the main body of the living body is displayed An endoscope extraction path obtaining step of obtaining an endoscope extraction movement path for extracting to the outside of the surgical space without colliding with the main body of the living body, and the endoscope extraction to extract the endoscope to the outside of the surgical space along the obtained endoscope extraction movement path It may include a step.

In addition, the endoscope image quality evaluation and automatic extraction method in the minimally invasive surgery according to an embodiment of the present invention, the automatic endoscope extraction step of the endoscope is extracted from the main body of the vital element displayed on the 3D image and the outside of the surgical space The collision prevention step further comprises a collision prevention step, the collision prevention step, the endoscope position extraction step of extracting the real-time position of the endoscope; The endoscope is extracted to the outside of the main body and the surgical space by using the position information of the main body of the main body in the 3D image and the real-time position information of the endoscope in the 3D image extracted by the endoscope position extraction unit A collision risk determination step of determining a collision risk; A risk signal generation step of generating a danger signal when it is determined in the collision risk determination step that the main body and the endoscope extracted to the outside of the surgical space have a collision risk; And when it is determined in the collision risk determining step that the main body and the endoscope extracted to the outside of the surgery space have a collision risk, the endoscope moves to prevent collision between the main body of the body and the endoscope. Anti-collision control step of controlling; may include.

In addition, the endoscope image quality evaluation and automatic extraction method in the minimally invasive surgery according to an embodiment of the present invention, the collision risk determining step is the distance from the real-time position of the main body and the endoscope extracted outside the surgical space If the calculated distance is within a predetermined range, and if the calculated distance is within a predetermined range, the dangerous signal generating step generates a dangerous signal, the collision avoidance control step is The movement of the endoscope may be controlled to prevent collision between the main body of the living body and the endoscope.

According to the endoscope image quality evaluation and automatic extraction system and the endoscopy image quality evaluation and automatic extraction method using the same in the minimally invasive surgery according to an embodiment of the present invention having the configuration as described above, The objective evaluation is possible, so that the minimum image quality can be guaranteed for safe surgery.

In addition, according to an endoscopy image quality evaluation and automatic extraction system and minimally invasive surgery in accordance with an embodiment of the present invention and the endoscope image quality determination and automatic extraction method using the same, the main components of the living body when extracting the endoscope outside the surgical space and Since the endoscope is detached automatically in a state that can prevent the collision of the human, there is an effect that can eliminate the occurrence of safety problems due to human error and skill of the nurse.

In addition, according to the endoscope image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention and the endoscope image quality determination and automatic extraction method using the same, it is possible to safely use not only rigid endoscopes but also flexible endoscopes There is an effect that the convenience of the surgeon can be enhanced and the robot surgery range can be widened.

The effects according to the present invention are not limited to the above mentioned effects, and other effects not mentioned above will be clearly understood by those skilled in the art from the claims and the description. Could be.

1 is a block diagram showing the endoscope image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention,
2 is a block diagram showing an endoscope automatic extracting unit according to an embodiment of the present invention,
3 is a block diagram showing an anti-collision unit according to an embodiment of the present invention,
4 is a block diagram illustrating a 3D image processor according to an exemplary embodiment of the present invention.
5 is a view schematically showing a state of photographing the inflated surgery space before performing the robot surgery,
FIG. 6 is a view schematically showing a 3D image of a surgery space in which main body elements are displayed.
7 is a flowchart illustrating an endoscopic image quality evaluation and automatic extraction method in minimally invasive surgery according to an embodiment of the present invention.
8 is a flowchart illustrating an anti-collision step according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

While the invention allows for various modifications and variations, specific embodiments thereof are illustrated by way of example in the drawings and will be described in detail below. However, it is not intended to be exhaustive or to limit the invention to the precise forms disclosed, but rather the invention includes all modifications, equivalents, and alternatives consistent with the spirit of the invention as defined by the claims.

In addition, in the accompanying drawings, the thickness and size may be exaggerated for clarity of the specification, so the present invention is not limited by the relative size or thickness shown in the accompanying drawings.

1 is a block diagram showing the endoscope image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention.

1, the endoscope image quality evaluation and automatic extraction system 100 in the minimally invasive surgery according to an embodiment of the present invention, the endoscope 110, endoscopy management section 120, endoscopy management quality evaluation unit 130, a signal generator 140, a motion obtaining unit 150, and an endoscope automatic extractor 160 may be included.

The endoscope 110 is inserted into the surgical space 2 inflated before the robot surgery with the robot arm 102 of the surgical robot to which the surgical tool (end-effector) for the robot surgery is attached. ) (The area where robotic surgery is done)

The endoscope management acquisition unit 120 is configured to obtain an image of the surgical site 19 taken by the endoscope 110.

The endoscope management quality evaluation unit 130 is a configuration for evaluating the quality (quality) of the image obtained by the endoscope management image acquisition unit 120.

For example, the endoscopic quality evaluation unit 130 displays the acquired image with a mean squared error (MSE), a structural similarity (SSIM), a signal to noise ratio (SNR) & a pixel signal to noise ratio (PSNR), and a frequency. Image color assessment, Image contrast assessment, Contamination in the same way as Image color variation (L * a * b *, RGV, HSV), Feature similarity index (FSIM), Gradient similarity, Knowledge based training (Deep Learning, Machine Learning) Detection can be comprehensively evaluated.

The signal generator 140 is configured to generate a predetermined signal when the quality of the acquired image is lowered below a predetermined level as a result of the image quality evaluation by the endoscopy management quality evaluation unit 130.

The signal generator 140 is configured to notify the surgeon when the quality of the image photographed by the endoscope 110 is not suitable for the progress of the robotic operation, and an auditory or / and visual method may be used. have.

In addition, the endoscopic quality assessment unit 130 may quantify the quality of the acquired image, and the signal generator 140 may generate a signal when the quality value of the digitized image is less than or equal to a predetermined value.

In this case, the signal generator 140 may generate different signals according to a range of quality values of the digitized image.

For example, the signal generator 140 may display different signals according to ranges of quality values of the digitized images, that is, cautions, warnings, and dangers, according to the degree of quality evaluation of the acquired images. May occur.

In addition, the endoscopic management image acquisition unit 120 may acquire the image of the surgical region taken by the endoscope 110 at regular intervals, that is, periodically.

In this case, the endoscopy management quality evaluation unit 130 may quantify the quality of the acquired image by comparing it with the image obtained immediately before the acquired image.

Alternatively, the endoscopy management quality evaluation unit 130 may digitize using the acquired image quality, and the digitization may be performed by matching image elements including whether foreign matter is attached, fog, brightness, and clarity.

The motion obtaining unit 150 of the collection is configured to obtain the consent of the collection for cleaning the endoscope 110 when a signal is generated in the signal generator 140.

That is, the motion obtaining unit 150 of the drawing is intended to obtain the consent of the drawing for the efficiency and safety of robotic surgery before extracting the endoscope 110 to the outside of the surgical space 2 for washing. Without consent, the endoscope 110 is not extracted to the outside of the surgical space (2).

The motion obtaining unit 150 of the drawing may be configured to obtain agreement through voice recognition of the drawing, or may be configured as a selection unit such as a button.

The endoscope automatic extraction unit 160 is configured to automatically extract the endoscope 110 to the outside of the surgical space (2) when the consent of the collector for cleaning the endoscope 110 is obtained.

On the other hand, as shown in Figure 1, the surgical space (2) in the extraction of the endoscope 110, organs (5), blood vessels including the arteries and veins that may cause damage due to collision with the endoscope 110 ), The main body 3 of the living body, such as the nerve (9).

Therefore, the endoscope automatic extractor 160 extracts the endoscope 110 in a direction of avoiding collision with the main body 3 of the living body 2 present when the endoscope 110 is automatically extracted to the outside of the surgical space 2. Needs to be.

2 is a block diagram showing an automatic endoscope extraction unit according to an embodiment of the present invention.

2, the endoscope automatic extractor 160 according to an embodiment of the present invention may include an endoscope extraction path acquisition unit 162 and an endoscope extraction unit 164.

The endoscope extraction path acquisition unit 162 uses the 3D image of the surgical space 2 on which the main biological component 3 is displayed to collide the endoscope 110 without the collision with the main biological component 3. It is a configuration to obtain an endoscope extraction movement path for extraction to the outside.

The endoscope extractor 164 is configured to extract the endoscope 110 to the outside of the surgical space 2 along the obtained endoscope extraction movement path.

In addition, the endoscope automatic extractor 160 further includes a collision preventing unit 170 that prevents the collision between the main body 3 of the living body 3 displayed on the 3D image and the endoscope 110 extracted outside the surgical space 2. It may include.

3 is a block diagram showing an anti-collision unit according to an embodiment of the present invention.

Referring to FIG. 3, the collision avoidance unit 170 according to an embodiment of the present invention includes an endoscope position extraction unit 172, a collision risk determination unit 174, a danger signal generator 175, and a collision prevention control unit 177. ) May be included.

The endoscope position extractor 172 is configured to extract the real-time position of the endoscope 110 in the 3D image of the surgical space 2 in which the main body 3 of the living body is displayed.

The endoscope position extraction unit may extract the position of the endoscope 110 extracted outside the surgical space 2 in real time using an optical tracker, a camera image, and the like.

The collision risk determination unit 174 is a location information of the main body of the living body 3 in the 3D image and the real-time location information of the endoscope 110 in the 3D image extracted by the endoscope position extraction unit 172 Using the configuration to determine whether the collision risk of the main body 3 and the endoscope 110 is extracted to the outside of the surgical space (2).

When the danger signal generator 175 determines that the collision risk determination unit 174 determines that there is a risk of collision between the main body 3 of the living body and the endoscope 110 extracted outside the surgical space 2. In other words, it generates a danger signal.

The collision avoidance control unit 177 determines that there is a risk of collision between the main body element 3 and the endoscope 110 extracted outside the surgical space 2 as a result of the determination of the collision risk determination unit 174. The endoscope automatic extraction unit 160 is controlled to prevent collision of the main body 3 of the living body and the endoscope 110.

In addition, the collision risk determination unit 174 calculates the distance from the real-time location of the main body of the body 3 and the endoscope 110 extracted to the outside of the surgical space 2, the calculated distance is a preset range It can be determined whether or not within.

In this case, when the calculated distance is within a predetermined range, the danger signal generator 175 generates a danger signal, and the collision avoidance control unit 177 may include the main body of the living body 3 and the endoscope 110. The endoscope automatic extractor 160 may be controlled to prevent a collision.

In addition, the collision risk determination unit 174 determines an expected path of the endoscope 110 extracted outside the surgical space 2 extracted by the endoscope position extraction unit 172 through trajectory estimation. In addition, it is possible to determine whether the identified predicted path is likely to collide with the main biological body 3.

On the other hand, the endoscope image quality determination and automatic extraction system 100 according to an embodiment of the present invention, the 3D image processing unit 180 for generating a 3D image of the surgical space 2 in which the main body of the body 3 is displayed. It may further include.

Figure 4 is a block diagram showing a 3D image processing unit according to an embodiment of the present invention, Figure 5 is a view showing a state of photographing the inflated surgery space before performing the robot surgery, Figure 6 is a surgery in which the main body of the living body is displayed A diagram schematically showing a 3D image of a space.

4 to 6, the 3D image processor 180 according to an embodiment of the present invention includes an image acquisition unit 182, a 3D image matching unit 183, a biometric main element extraction unit 185, and a 3D image. The generation unit 187 may be included.

The image acquisition unit 182 is configured to acquire a plurality of 2D or 3D images taken with respect to the inflated surgical space 2 before performing the robot surgery.

As shown in FIG. 5, a plurality of endoscopes 17 may be used to photograph the inflated surgical space 2 before performing the robot surgery.

The 3D image matching unit 183 is configured to match a plurality of 2D or 3D images acquired by the image obtaining unit 182 into one 3D image representing the inflated surgical space 2.

The biometric main element extractor 185 extracts the biometric main element 3 included in the matched 3D image.

The biometric main element extractor 185 may extract the biometric main element 3 after removing the noise in the image through the image processing of the matched 3D image.

Here, the image noise processing technique may include reflected light removal, image noise filtering, and imprinting. The extraction of the main body 3 of the living body may include blob counting and detection. ), Labeling, real-time tracking, and deep learning algorithms.

The 3D image generating unit 187 displays the biometric main elements 3 extracted by the biometric main element extraction unit 185 on the matched 3D image to display the surgery space 2 in which the biometric main elements 3 are displayed. ) To generate a 3D image.

6 schematically illustrates the 3D image configured as described above.

Hereinafter, an endoscope image quality evaluation and automatic extraction method using an endoscopy image quality evaluation and automatic extraction system in the minimally invasive surgery according to an embodiment of the present invention having the above-described configuration will be described in detail.

7 is a flowchart illustrating an endoscopic image quality evaluation and automatic extraction method in minimally invasive surgery according to an embodiment of the present invention.

Referring to Figure 7, the endoscope image quality evaluation and automatic extraction method according to an embodiment of the present invention (S100), endoscopy management award step (S110), endoscopy management award step (S120), endoscopy management quality determination step (S130) ), A signal generation step (S140), a motion acquisition step (S150), endoscope automatic extraction step (S160).

The endoscopy management step (S110) is a step of real-time imaging of the surgical site 19 during the robot surgery is inserted into the inflated surgery space (2) before performing the robot surgery.

The endoscopy management acquisition step (S120) is a step of acquiring an image of a surgical region photographed in the endoscopy management step (S110).

The endoscopy management quality determining step (S130) is a step of determining the quality of an image acquired in the endoscopy management obtaining step (S120).

The signal generation step (S140) is a step of generating a predetermined signal when the quality of the acquired image is lowered below a predetermined level as a result of the image quality determination in the endoscope management quality determination step (S130).

Acquisition consent (S150) of the collection is a step of acquiring the consent of the collection for the cleaning of the endoscope 110 after the signal generation step (S140).

The endoscope automatic extraction step (S160) automatically extracts the endoscope 110 to the outside of the surgical space 2 when acquiring the consent of the concentrator for washing the endoscope 110 in the acquiring motion of the congestion step S150. It's a step.

The endoscopic management quality determining step (S130) may quantify the quality of the acquired image to indicate a degree of deterioration of the quality of the acquired image, and in this case, the signal generation step (S140) may be performed. If less than a predetermined value can generate a signal.

In addition, the signal generation step (S140) may generate different signals according to the range of the quality value of the digitized image.

For example, the signal generation step (S140) may generate different signals so as to be divided into cautions, warnings, and dangers according to the degree of evaluation of the quality of the acquired image.

The endoscope automatic extraction step 160 may include an endoscope extraction path acquisition step S162 and an endoscope extraction step S164.

The endoscopic extraction path obtaining step (S162) uses the 3D image of the surgical space 2 on which the main biological component 3 is displayed to collide the endoscope 110 with the surgical main component 3 without colliding with the main biological component 3. (2) obtaining an endoscope extraction movement path for extraction to the outside;

The endoscope extraction step (S164) is a step of extracting the endoscope 110 to the outside of the surgical space 2 along the obtained endoscope extraction movement path in the endoscope extraction path acquisition step (S162).

In addition, the endoscope automatic extraction step 160 is a collision avoidance step 170 for preventing the collision of the main body 3 and the endoscope 110 is extracted to the outside of the surgical space 2 displayed on the 3D image. It may further include.

8 is a flowchart illustrating an anti-collision step according to an embodiment of the present invention.

Referring to FIG. 8, the collision avoidance step S170 may include an endoscope position extraction step S172, a collision risk determination step S174, a danger signal generation step S175, and an collision avoidance control step S177. .

The endoscope position extraction step (S172) is a step of extracting a real-time position of the endoscope 110 extracted outside the surgical space 2.

The collision risk determining step (S174) may include real-time location of the endoscope 110 in the 3D image extracted from the position information of the main body 3 of the living body and the endoscope position extracting unit S172 in the 3D image. It is a step of determining whether or not the collision risk between the main body of the living body element 3 and the endoscope 110 is extracted to the outside of the surgical space (2) using the information.

In the dangerous signal generation step (S175), as a result of the determination in the collision risk determination step (S174), it is determined that the main body element 3 and the endoscope 110 extracted to the outside of the surgical space 2 have a collision risk. If so, the step is to generate a danger signal.

The anti-collision control step (S177) is determined as the collision risk determination step (S174), it is determined that the main body element 3 and the endoscope 110 extracted to the outside of the surgical space (2) there is a risk of collision In this case, the movement of the endoscope 100 is prevented so that the main body 3 and the endoscope 110 collide with each other.

In addition, the collision risk determining step (S174) calculates the distance from the real-time position of the main body element 3 and the endoscope 110 is extracted to the outside of the surgical space 2, the calculated distance is set in advance It can be determined whether it is within the range.

In this case, when the calculated distance is within a predetermined range, the dangerous signal generating step (S175) generates a dangerous signal, and the collision prevention control step (S177) is the main body of the body 3 and the endoscope 110 The movement of the endoscope 110 may be controlled to prevent a collision of c).

As described above, the present invention is to determine the quality of the endoscope image during robotic surgery, when the endoscope lens cleaning is determined that the endoscope without colliding with the main body of the body after the confirmation of the obstruction for the efficiency and safety of surgery The present invention relates to an endoscopic image quality evaluation and automatic extraction system for minimally invasive surgery that automatically extracts the endometrium from the surgical space, and to an endoscopic image quality determination and automatic extraction method using the same. will be. Therefore, the present invention is not limited to the embodiments disclosed herein, and all forms changeable by those skilled in the art to which the present invention pertains will belong to the scope of the present invention.

2: surgery space 3: main elements of living body
19: surgical site
100: Endoscopic quality assessment and automatic extraction system for minimally invasive surgery
110: endoscope 120: endoscope management award
130: Quality Evaluation Department
140: signal generator 150: motion acquisition unit of the collection
160: endoscope automatic extraction unit
162: endoscope extraction path acquisition unit
164: endoscope extraction unit 170: collision prevention unit
172: endoscope position extraction unit
174: collision risk determination unit 175: danger signal generation unit
177: collision avoidance control unit 180: image processing unit
182: image acquisition unit 183: 3D image registration unit
185: main body extraction unit
187: 3D image generating unit

Claims (15)

An endoscope that is inserted into an inflated surgical space prior to performing robotic surgery to photograph the surgical site during robotic surgery in real time;
An endoscope management image acquisition unit for acquiring an image of a surgical site photographed by the endoscope;
An endoscope management quality evaluation unit configured to evaluate a quality of an image acquired by the endoscope management award acquirer;
A signal generator for generating a predetermined signal when the quality of the acquired image is lowered below a predetermined level as a result of the image quality evaluation by the endoscopy management quality evaluation unit;
Acquisition consent unit for acquiring the consent of the concentrator for washing the endoscope when a signal is generated in the signal generator; And
Endoscopic image quality evaluation and automatic extraction system in the minimally invasive surgery, including; an endoscope automatic extraction unit for automatically extracting the endoscope to the outside of the surgical space when acquiring consent of the endoscopic cleaning for the endoscope. The method of claim 1,
The endoscope management quality evaluation unit quantifies the quality of the acquired image,
The signal generator generates a signal when the quality value of the digitized image is less than a predetermined value, the endoscope image quality evaluation and automatic extraction system in minimally invasive surgery. The method of claim 2,
The signal generator generates different signals according to a range of quality values of the digitized image. Endoscopic image quality evaluation and automatic extraction system in minimally invasive surgery. The method of claim 2,
The endoscope management acquisition unit obtains the surgical site image taken by the endoscope at predetermined time intervals,
The endoscopic image quality evaluation unit endoscopic image quality evaluation and automatic extraction system in the minimally invasive surgery, characterized in that to quantify the quality of the acquired image compared to the image obtained immediately before the acquired image. The method of claim 2,
The endoscope management acquisition unit obtains the surgical site image taken by the endoscope at predetermined time intervals,
The endoscope management quality evaluation unit digitizes using the quality of the acquired image, the digitizing endoscope in the minimally invasive surgery, characterized in that by matching the image elements including the presence of foreign matter, fog, brightness, sharpness Image quality assessment and automatic extraction system. The method of claim 1,
The endoscope automatic extraction unit,
An endoscope extraction path acquisition unit for obtaining an endoscope extraction movement path for extracting the endoscope to the outside of the surgery space without colliding with the living body main element by using a 3D image of the surgery space in which the main body of the body is displayed;
Endoscopic image quality evaluation and automatic extraction system in the minimally invasive surgery, characterized in that it comprises an endoscope extraction unit for extracting the endoscope to the outside of the surgical space along the obtained endoscope extraction movement path. The method of claim 6,
The endoscope automatic extraction unit further includes a collision prevention unit for preventing the collision between the main body and the endoscope extracted to the outside of the surgical space displayed on the 3D image,
The anti-collision unit,
An endoscope position extraction unit for extracting a real time position of the endoscope;
The endoscope is extracted to the outside of the main body and the surgical space by using the position information of the main body of the main body in the 3D image and the real-time position information of the endoscope in the 3D image extracted by the endoscope position extraction unit A collision risk determination unit for determining a collision risk;
A risk signal generator for generating a danger signal when it is determined that the collision risk determining unit determines that the main body and the endoscope extracted outside the surgical space have a collision risk; And
As a result of the determination of the collision risk determination unit, when it is determined that the main body and the endoscope extracted outside the surgery space have a risk of collision, the collision prevention controlling the endoscope extraction unit to prevent the collision between the main body and the endoscope Endoscope image quality evaluation and automatic extraction system in the minimally invasive surgery, characterized in that it comprises a control unit. The method of claim 7, wherein
The collision risk determination unit calculates the distance from the main body of the living body and the real-time position of the endoscope extracted outside the surgical space, and determines whether the calculated distance is within a predetermined range,
When the calculated distance is within a predetermined range, the danger signal generation unit generates a danger signal, and the collision prevention control unit controls the endoscope extraction unit to prevent the collision between the main body and the endoscope. Endoscopic Image Quality Assessment and Automatic Extraction System in Minimally Invasive Surgery. The method according to any one of claims 6 to 8,
The endoscope image quality evaluation and automatic extraction system further includes a 3D image processing unit for generating a 3D image of the surgical space in which the main body of the living body is displayed,
The 3D image processing unit,
An image acquisition unit for acquiring a plurality of 2D or 3D images photographed for the inflated surgical space before performing a robotic operation;
A 3D image registration unit for matching a plurality of 2D or 3D images acquired by the image acquisition unit into one 3D image representing the inflated surgical space;
A biological main element extracting unit extracting a biological main element included in the matched 3D image; And
And a 3D image generating unit generating a 3D image of the inflated surgical space in which the biometric main elements are displayed on the matched 3D image by displaying the biometric main elements extracted by the biometric main element extracting unit. Endoscopic Image Quality Assessment and Automatic Extraction System in Minimally Invasive Surgery. An endoscopic management photographing step of inserting into an inflated surgical space before performing a robotic operation to photograph a surgical part during a robotic operation in real time;
An endoscope management acquisition step of acquiring an image of a surgical site photographed in the endoscopy management step;
An endoscope management quality determining step of determining a quality of an image acquired in the endoscope management acquisition step;
A signal generation step of generating a predetermined signal when the quality of the acquired image is lowered to a predetermined level or less as a result of the image quality determination in the endoscope management quality determination step;
Acquiring the consent of the concentrator for acquiring the consent of the concentrator for the endoscope cleaning after the signaling step; And
Endoscopic image quality determination and automatic in the minimally invasive surgery, including; the endoscope automatic extraction step of automatically extracting the endoscope to the outside of the surgical space when obtaining the consent of the endoscopic cleaning for the endoscopy in the motion acquisition step Extraction method. The method of claim 10,
In the step of determining the quality of the endoscopy management, the quality of the acquired image is quantified to indicate a degree of deterioration of the quality of the acquired image.
The signal generation step is an endoscopy image quality determination and automatic extraction method in the minimally invasive surgery, characterized in that for generating a signal when the quality value of the digitized image is less than a predetermined value. The method of claim 11,
The signal generation step is an endoscope image quality judgment and automatic extraction method for minimally invasive surgery, characterized in that for generating different signals according to the range of the quality value of the quantized image. The method of claim 11,
The endoscope automatic extraction step,
An endoscope extraction path acquisition step of acquiring an endoscope extraction movement path for extracting the endoscope to the outside of the surgery space without colliding with the living body main element by using a 3D image of the surgery space in which the main body of the body is displayed;
Endoscopic image quality evaluation and automatic extraction method in the minimally invasive surgery comprising the step of extracting the endoscope to the outside of the surgical space along the obtained endoscope extraction movement path. The method of claim 13,
The endoscope automatic extraction step further includes a collision prevention step of preventing a collision between the main body and the endoscope extracted to the outside of the surgical space displayed on the 3D image,
The anti-collision step,
An endoscope position extraction step of extracting a real time position of the endoscope;
The endoscope is extracted to the outside of the main body and the surgical space by using the position information of the main body of the main body in the 3D image and the real-time position information of the endoscope in the 3D image extracted by the endoscope position extraction unit A collision risk determination step of determining whether a collision risk exists;
A risk signal generation step of generating a danger signal when it is determined that the main body and the endoscope extracted outside the surgical space have a collision risk as a result of the determination at the collision risk determination step; And
As a result of the determination at the collision risk determining step, when the main body and the endoscope extracted to the outside of the surgical space are determined to have a collision risk, the movement of the endoscope is controlled to prevent the collision between the main body and the endoscope. Endoscopic image quality evaluation and automatic extraction method in the minimally invasive surgery comprising a; anti-collision control step. The method of claim 14,
The collision risk determining step is to calculate the distance from the real-time location of the main body and the endoscope to be extracted to the outside of the surgical space, and determine whether the calculated distance is within a predetermined range,
When the calculated distance is within a predetermined range, the dangerous signal generating step generates a dangerous signal, and the anti-collision control step controls the movement of the endoscope to prevent collision between the main body of the living body and the endoscope. Endoscopic image quality evaluation and automatic extraction in minimally invasive surgery.

Images