KR20210157532A - Image Display System and Method For The Same For Direct Projection On Surgical Or Operating Skin Portion For Reducing Sense Separating Phenomenon Generated In Minimally Invasive Surgery Or Operation - Google Patents

Abstract

The present invention relates to an image display system and an image display method for direct projection onto a skin of a surgical or surgical procedure site to reduce sensory separation phenomena that occurs during minimally invasive surgery and, more specifically, to an image display system and an image display method which do not match image devices for photographing and displaying a surgical procedure situation inside a body during minimally invasive surgery with a location and a direction in which a surgical procedure is performed, thereby preventing sensory separation phenomena from occurring.

Description

Image Display System and Method For The Same For Direct Projection On Surgical Or Operating Skin Portion For Reducing Sense Separating Phenomenon Generated In Minimally Invasive Surgery Or Operation}

The present invention relates to an image display system and an image display method for direct projection onto the skin of a surgical or surgical site for reducing sensory separation that occurs during minimally invasive surgery, and more particularly, to a surgical condition inside the body during minimally invasive surgery. The present invention relates to an image display system and an image display method for preventing a phenomenon in which sensory separation occurs because imaging devices that are photographed and displayed do not match the position and direction in which a procedure is performed.

Minimally invasive surgery refers to a surgery that minimizes the incision during surgery to minimize surgical wounds. In the case of surgery using monitors or robots, the medical staff does not directly operate on the patient. Often times. 1 is a photograph showing an example of minimally invasive surgery.

On the other hand, in order to know the location of a surgical tool used during surgery or a procedure, the operator or the operator can directly grasp the location of the surgical tool, but a separate measuring device is required for a surgery or procedure that requires a high degree of precision. , There is a need for a method capable of reducing the risk of surgery or procedure, and reducing the pain and tissue damage of the patient through this.

In the case of conventional minimally invasive surgery, the image devices that record and display the surgical situation inside the body do not match the location and direction of the operation, resulting in sensory separation and increasing side effects. There is an increasing demand for improvement of technology that can be operated and operated.

Korean Patent Registration Publication No. 10-1700514 (Title of the invention: Robot system for minimally invasive surgery) Korean Patent Registration Publication No. 10-1684781 (Title of the invention: X-ray imaging device for minimally invasive surgery) Korea Patent Registration Publication No. 10-2085588 (Title of the invention: surgical tool position tracking system)

Confirmation of the exact treatment site and timeout, J. Korean Med. Assoc. 2015 February; 58(2): 110-115

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image display system for direct projection onto the skin of a surgical or surgical site for reducing sensory separation that occurs during minimally invasive surgery.

It is an object of the present invention to provide an image display method for direct projection onto the skin of a surgical site or a surgical site for reducing sensory separation that occurs during minimally invasive surgery.

A video display system according to an embodiment of the present invention,

a surgical tool tracking module for tracking the position of a surgical tool inserted into the body during surgery;

a doctor's motion tracking module for tracking the position of the doctor's arm holding the surgical tool during surgery;

a matrix scan module for scanning the patient's surgical site;

a multi-channel output module for projecting a camera image on a patient's surgical site during surgery;

a display module that can check the surgical site; and

a control module for controlling the surgical tool tracking module, the doctor's motion tracking module, the matrix scan module, the multi-channel output module, and the display module;

A video display system according to an embodiment of the present invention,

The surgical tool tracking module tracks the position of the surgical tool and the angle of the surgical tool, and the angle of the surgical tool is a z-axis in a direction perpendicular to the ground and an x-axis and a y-axis in a direction parallel to the ground. When defining, an angle α inclined with respect to the x-axis with respect to the longitudinal direction of the surgical tool, an angle β inclined with respect to the y-axis with respect to the longitudinal direction of the surgical tool, with respect to the longitudinal direction of the surgical tool Data obtained by calculating the angle γ of the inclination with respect to the z-axis may be transmitted to the control module.

A video display system according to an embodiment of the present invention,

The doctor's motion tracking module tracks the position and angle of the arm operated by the doctor, and the angle of the operator's arm is a z-axis in a direction perpendicular to the ground, an x-axis and a y-axis in a direction parallel to the ground. When defined as , the angle α' inclined with respect to the x-axis with respect to the longitudinal direction of the doctor's arm, the angle β' inclined with respect to the y-axis with respect to the length direction of the doctor's arm, the doctor's Transmits the data calculated by calculating the angle γ' with respect to the z-axis with respect to the longitudinal direction of the arm to the control module,

In addition, data obtained by calculating a rotation angle with respect to the length direction of the arm of the doctor by tracking the area of the arm operated by the doctor may be transmitted to the control module.

A video display system according to an embodiment of the present invention,

The matrix scan module for scanning the surgical site of the patient may manage the scan data in the form of a matrix on the skin of the surgical site and transmit the operation progress to the control module.

A video display system according to an embodiment of the present invention,

The multi-channel output module for projecting a camera image to the surgical site of the patient during the operation and the display module for confirming the surgical site may operate in conjunction with a correction module for correcting the distortion of the actual image.

A video display system according to an embodiment of the present invention,

The data obtained by the surgical tool tracking module and the data obtained by the doctor's motion tracking module are transmitted to the control module, and the transmitted data is combined with the virtualized data in the correction module to minimize the shading of the surgical site in the display module can do.

A video display system according to an embodiment of the present invention,

It further comprises an auxiliary video screen module,

The auxiliary image screen module shows the surgical site as an auxiliary when a malfunction of the display module or an additional screen is required, the auxiliary image screen module is a translucent head-up display, and the auxiliary image screen includes the situation in front and Information necessary for surgery and the area to be treated can be indicated.

An image display method according to an embodiment of the present invention,

tracking the position of the surgical tool inserted into the body during surgery by the surgical tool tracking module;

tracking the position of the doctor's arm holding the surgical tool during surgery by the doctor's motion tracking module;

dividing the patient's surgical site into matrix sections by the matrix scan module and scanning;

Projecting a camera image to the surgical site of the patient during surgery by a multi-channel output module so that the doctor can know the surgical site;

A step of confirming the surgical site by the display module; and

controlling the surgical tool tracking module, the doctor's motion tracking module, the matrix scan module, the multi-channel output module, and the display module by the control module; may include.

An image display system according to an embodiment of the present invention,

a surgical tool tracking module for tracking the position of a surgical tool inserted into the body during a procedure;

an operator's motion tracking module for tracking the position of the doctor's arm holding the surgical tool during the operation;

a matrix scan module for scanning the patient's treatment area;

a multi-channel output module for projecting a camera image to the treatment site of the patient during the procedure;

a display module that can confirm the treatment site; and

It may include; a control module for controlling the procedure tool tracking module, the operator's motion tracking module, the matrix scan module, the multi-channel output module, and the display module.

An image display method according to the present invention,

tracking the position of the surgical tool inserted into the body during the treatment by the treatment tool tracking module;

tracking the position of the doctor's arm holding the surgical tool during the operation by the operator's motion tracking module;

dividing the treatment area of the patient into matrix sections by the matrix scan module and scanning;

Projecting a camera image to the treatment site of the patient during the treatment by the multi-channel output module so that the operator can know the treatment site;

A step of confirming the treatment site by the display module; and

It may include; controlling the surgical tool tracking module, the operator's motion tracking module, the matrix scan module, the multi-channel output module, and the display module by the control module.

According to the image display system of the present invention, by providing a surgical tool tracking module, a doctor's motion tracking module, a matrix scan module, a multi-channel output module, a display module, and a control module, it is possible to reduce the sensory separation phenomenon that occurs during minimally invasive surgery. have.

1 is a photograph showing an example of minimally invasive surgery.
Figure 2 (a) shows a photograph for the analysis of the surgical use environment, Figure 2 (b) shows a photograph of the surgical site for the operation environment analysis.
3 is a photograph showing an example of a surgical site image projection.
4 and 5 are schematic diagrams for showing an image display system for direct projection on the skin of the surgical site to reduce the sensory separation phenomenon that occurs during minimally invasive surgery of the present invention.
6 to 10 are schematic diagrams showing an image display system of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, but the present invention is not limited or limited by the embodiments.

In the following drawings, the same reference numerals refer to the same components, and the size of each component in the drawings may be exaggerated for clarity and convenience of description. Meanwhile, the embodiments described below are merely exemplary, and various modifications are possible from these embodiments. Hereinafter, what is described as "upper" or "upper" may include not only directly on in contact but also on non-contacting. Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. The singular expression includes the plural expression unless the context clearly dictates otherwise. Also, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation.

The image display system of the present invention needs to secure a theoretical background and mathematical modeling in order to reduce the sensory separation phenomenon that occurs during minimally invasive surgery. In addition, it is necessary to precede the analysis of the usability environment of the surgical or surgical situation with the medical staff, and to correct the distortion of the image projected by the projection based on this. FIG. 2 shows a picture of a surgical site as an example for analyzing the surgical use environment, and FIG. 3 is a picture showing an example of projecting an image of the surgical site. Here, distortion and shading should be minimized when displaying an image directly on the skin in a surgical or surgical situation, such as image projection by projection, multi-channel image synthesis, and 3D object-space recognition. Here, 3D object-space recognition can be achieved through tracking and modeling.

4 is a schematic diagram illustrating an image display system for direct projection onto the skin of a surgical site for reducing sensory separation that occurs during minimally invasive surgery of the present invention.

That is, in the image display system of the present invention, the captain doctor can perform the operation by the image transmitted by the image projector to the patient's surgical site, and the staff doctor can use the image projector on the patient's surgical site from the opposite side of the captain doctor According to the transmitted image, the surgeon can assist the captain to proceed with the operation. Here, the first sensor for tracking the movement of the arm of the captain doctor and the staff doctor, and the second sensor for tracking the movement of the surgical tool used by the captain doctor and the staff doctor to proceed with the operation can be provided

Here, a plurality of first sensors for tracking the movements of the arms of the captain doctors and the staff doctors are provided to track the movements of the arms of the captain doctors and the staff doctors, and the captain doctors and the staff doctors perform the surgery A plurality of second sensors for tracking the movement of the surgical tool used to do this are provided to track the movement of the surgical tool used by the captain doctor and the staff doctor. Providing a plurality of sensors is preferable in terms of three-dimensional scanning.

The image display system of the present invention can provide an improved effect on the technology for a high-resolution image fluoroscopy device and technique that projects directly on the skin for surgery or a procedure. That is, it provides an improved technique for correcting distortion of an image that occurs when projecting directly to the skin.

6, the image display system of the present invention includes a surgical tool tracking module for tracking the position of a surgical tool inserted into the body during surgery; a doctor's motion tracking module for tracking the position of the doctor's arm holding the surgical tool during surgery; a matrix scan module for scanning the patient's surgical site; a multi-channel output module for projecting a camera image on a patient's surgical site during surgery; a display module that can check the surgical site; and a control module for controlling the surgical tool tracking module, the doctor's motion tracking module, the matrix scan module, the multi-channel output module, and the display module. It may include a correction module.

As shown in FIG. 7 , data such as the position of the surgical tool can be transmitted by the surgical tool tracking module, and data such as the position of the doctor's arm can be transmitted by the doctor's motion tracking module.

5 shows a conceptual diagram of a surgical tool tracking module and a doctor's motion tracking module.

Here, the surgical tool tracking module tracks the position of the surgical tool and the angle of the surgical tool, and the angle of the surgical tool is a z-axis in a direction perpendicular to the ground, an x-axis in a direction parallel to the ground, and a y- When defined as an axis, an angle α inclined with respect to the x-axis with respect to the longitudinal direction of the surgical tool, an angle β inclined with respect to the y-axis with respect to the longitudinal direction of the surgical tool, the longitudinal direction of the surgical tool The data obtained by calculating the inclination angle γ with respect to the z-axis may be transmitted to the control module.

In addition, the motion tracking module of the doctor tracks the position and angle of the arm operated by the doctor, and the angle of the operator's arm is a z-axis in a direction perpendicular to the ground, an x-axis in a direction parallel to the ground, and y When defined as the -axis, the angle α' inclined with respect to the x-axis with respect to the longitudinal direction of the arm of the doctor, the angle β' inclined with respect to the y-axis with respect to the longitudinal direction of the arm of the doctor, said Data obtained by calculating the angle γ' inclined with respect to the z-axis with respect to the longitudinal direction of the doctor's arm may be transmitted to the control module.

In addition, data obtained by calculating the rotation angle θ with respect to the center line of the arm of the doctor by tracking the area of the arm operated by the doctor may be transmitted to the control module.

In the surgical tool tracking module and the doctor's motion tracking module, a motion or motion can be grasped with numerical data using a sensor, but if the sensor is an optical sensor, it falls within the scope of the present invention. For example, a sensor that detects ultraviolet rays, infrared rays, or the like, and image scanning through a camera image may be used. In addition, one or a plurality of first optical marks for scanning precision are attached to the surgical tool to track the position of the surgical tool, and one or more for tracking the position of the doctor's arm holding the surgical tool during surgery For scanning precision, a plurality of second optical marks may be attached to the doctor's arm to increase the sensing efficiency of a surgical tool or movement of the doctor's arm.

As shown in FIG. 8 , in the image display system of the present invention, the matrix scan module for scanning the surgical site of the patient manages the scan data in the form of a matrix on the skin of the surgical site to transmit the surgical progress to the control module. have.

9, the multi-channel output module for projecting a camera image to the surgical site of the patient during the operation and the display module for confirming the surgical site operate in conjunction with the correction module for correcting the distortion of the actual image can do.

In this way, the data obtained by the surgical tool tracking module and the data obtained by the doctor's motion tracking module are transmitted to the control module, and the transmitted data is combined with the virtualized data in the correction module to display the shadow of the surgical site in the display module. It can be controlled to be minimized, and accordingly, it becomes possible to reduce the sensory separation phenomenon that occurs during minimally invasive surgery.

As shown in Fig. 10, the imaging system of the present invention further includes an auxiliary image screen module, wherein the auxiliary image screen module shows the surgical site as an auxiliary when a malfunction of the display module or an additional screen is required, and the auxiliary image screen The module is a translucent head-up display, and the auxiliary image screen may indicate the situation in the front, information necessary for surgery, and a treatment area.

The present invention includes the steps of tracking the position of a surgical tool inserted into the body during surgery by a surgical tool tracking module; tracking the position of the doctor's arm holding the surgical tool during surgery by the doctor's motion tracking module; dividing the patient's surgical site into matrix sections by the matrix scan module and scanning; Projecting a camera image to the surgical site of the patient during surgery by a multi-channel output module so that the doctor can know the surgical site; A step of confirming the surgical site by the display module; and controlling the surgical tool tracking module, the doctor's motion tracking module, the matrix scan module, the multi-channel output module, and the display module by the control module; It is possible to provide an image display method for direct projection on the skin of the surgical site for

Here, the step of tracking the position of the surgical tool inserted into the body during surgery by the surgical tool tracking module tracks the position of the surgical tool and the angle of the surgical tool, and the angle of the surgical tool is a direction perpendicular to the ground. When the z-axis and the direction parallel to the ground are defined as the x-axis and the y-axis, the angle α inclined with respect to the x-axis with respect to the longitudinal direction of the surgical tool is y with respect to the longitudinal direction of the surgical tool. - It may include the step of transmitting to the control module the data obtained by calculating the angle β inclined with respect to the axis and the angle γ inclined with respect to the z-axis with respect to the longitudinal direction of the surgical tool.

The step of tracking the position of the arm of the doctor holding the surgical tool during surgery by the motion tracking module of the doctor tracks the position and angle of the arm operated by the doctor, and the angle of the operator's arm is perpendicular to the ground. When the direction is defined as the z-axis and the direction parallel to the ground is defined as the x-axis and the y-axis, the angle α' inclined with respect to the x-axis with respect to the longitudinal direction of the doctor's arm is Transmitting, to the control module, data obtained by calculating the angle β' of the inclination with respect to the y-axis with respect to the longitudinal direction and the angle γ' of the angle of inclination with respect to the z-axis with respect to the longitudinal direction of the arm of the doctor can Furthermore, the method may include tracking the area of the arm operated by the doctor and transmitting data obtained by calculating the rotation angle with respect to the centerline direction of the arm of the doctor to the control module.

The scanning by dividing the surgical site of the patient into matrix sections by the matrix scan module may include managing scan data in the form of a matrix on the skin of the surgical site and transmitting the surgical progress to the control module.

Projecting a camera image to the surgical site of the patient during surgery by the multi-channel output module so that the doctor can know the surgical site includes a multi-channel output module for projecting the camera image to the surgical site of the patient during the surgery and the surgery The display module capable of confirming the part may include a step of operating in conjunction with a correction module for correcting the distortion of the actual image.

In addition, it may include the step of transmitting the data obtained by the surgical tool tracking module and the data obtained by the doctor's motion tracking module to the control module. In addition, by combining the transmitted data with virtual data in the correction module, the control module may include the step of controlling the shading of the surgical site to be minimized and transmitting it to the display module.

In addition, the auxiliary image screen module prevents malfunction of the display module and shows the surgical site as an auxiliary when an additional screen is required, and the auxiliary image screen module is a translucent head-up display, and the auxiliary image screen has It may include a step of indicating the situation and information necessary for the operation and the operation site in the front.

In addition, the above-mentioned bar shows an image display system for direct projection to the skin of the surgical site to reduce the sensory separation phenomenon occurring during minimally invasive surgery, but directly onto the skin of the surgical site to reduce the sensory separation phenomenon occurring during minimally invasive surgery. The image display system and method for projecting can be represented in the same way.

As described above, although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited to the above embodiments, and various modifications and variations from these descriptions are provided by those skilled in the art to which the present invention pertains. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the following claims as well as the claims and equivalents.

Claims (10)

a surgical tool tracking module for tracking the position of a surgical tool inserted into the body during surgery;
a doctor's motion tracking module for tracking the position of the doctor's arm holding the surgical tool during surgery;
a matrix scan module for scanning the patient's surgical site;
a multi-channel output module for projecting a camera image on a patient's surgical site during surgery;
a display module that can check the surgical site; and
Surgery for reducing sensory separation occurring during minimally invasive surgery, characterized in that it includes; a control module for controlling the surgical tool tracking module, the doctor's motion tracking module, the matrix scan module, the multi-channel output module, and the display module An image display system for direct projection onto the local skin. The method according to claim 1,
The surgical tool tracking module tracks the position of the surgical tool and the angle of the surgical tool, and the angle of the surgical tool is a z-axis in a direction perpendicular to the ground and an x-axis and a y-axis in a direction parallel to the ground. When defining, an angle α inclined with respect to the x-axis with respect to the longitudinal direction of the surgical tool, an angle β inclined with respect to the y-axis with respect to the longitudinal direction of the surgical tool, with respect to the longitudinal direction of the surgical tool An image display system for direct projection on the skin of the surgical site to reduce sensory separation that occurs during minimally invasive surgery, characterized in that the data calculated by calculating γ for the angle γ with respect to the z-axis is transmitted to the control module. The method according to claim 1,
The doctor's motion tracking module tracks the position and angle of the arm operated by the doctor, and the angle of the operator's arm is a z-axis in a direction perpendicular to the ground, an x-axis and a y-axis in a direction parallel to the ground. When defined as , the angle α' inclined with respect to the x-axis with respect to the longitudinal direction of the doctor's arm, the angle β' inclined with respect to the y-axis with respect to the length direction of the doctor's arm, the doctor's Transmits the data calculated by calculating the angle γ' with respect to the z-axis with respect to the longitudinal direction of the arm to the control module,
In addition, an operation for reducing sensory separation occurring during minimally invasive surgery, characterized in that by tracking the area of the arm operated by the doctor and transmitting the data calculated by calculating the rotation angle with respect to the centerline direction of the arm of the doctor to the control module An image display system for direct projection onto the local skin. The method according to claim 1,
The matrix scan module for scanning the surgical site of the patient manages the scan data in the form of a matrix on the skin of the surgical site to transmit the surgical progress to the control module. An image display system for direct projection onto the skin of the surgical site for The method according to claim 1,
Minimally invasive, characterized in that the multi-channel output module for projecting a camera image to the surgical site of the patient during the operation and the display module for confirming the surgical site operate in conjunction with a correction module for correcting the distortion of the actual image An image display system for direct projection on the skin of the surgical site to reduce sensory separation that occurs during surgery. 6. The method of claim 5,
The data obtained by the surgical tool tracking module and the data obtained by the doctor's motion tracking module are transmitted to the control module, and the transmitted data is combined with the virtualized data in the correction module to minimize the shading of the surgical site in the control module An image display system for direct projection on the skin of the surgical site to reduce the sensory separation phenomenon that occurs during minimally invasive surgery, characterized in that it is transmitted to the display module. The method according to claim 1,
It further comprises an auxiliary video screen module,
The auxiliary image screen module shows the surgical site as an auxiliary when an additional screen is required to prevent malfunction of the display module, and the auxiliary image screen module is a translucent head-up display, and the auxiliary image screen displays the situation in the front. and an image display system for direct projection on the skin of the surgical site to reduce sensory separation that occurs during minimally invasive surgery, characterized in that it displays information necessary for surgery and the surgical site. tracking the position of the surgical tool inserted into the body during surgery by the surgical tool tracking module;
tracking the position of the doctor's arm holding the surgical tool during surgery by the doctor's motion tracking module;
dividing the patient's surgical site into matrix sections by the matrix scan module and scanning;
Projecting a camera image to the surgical site of the patient during surgery by a multi-channel output module so that the doctor can know the surgical site;
A step of confirming the surgical site by the display module; and
Controlling the surgical tool tracking module, the doctor's motion tracking module, the matrix scan module, the multi-channel output module, and the display module by the control module; An image display method for direct projection on the skin of the surgical site. a surgical tool tracking module for tracking the position of a surgical tool inserted into the body during a procedure;
an operator's motion tracking module for tracking the position of the doctor's arm holding the surgical tool during the operation;
a matrix scan module for scanning the patient's treatment area;
a multi-channel output module for projecting a camera image to the treatment site of the patient during the procedure;
a display module that can confirm the treatment site; and
A procedure for reducing sensory separation occurring during minimally invasive surgery, comprising a control module for controlling the surgical tool tracking module, the operator's motion tracking module, the matrix scan module, the multi-channel output module, and the display module An image display system for direct projection onto the local skin. tracking the position of the surgical tool inserted into the body during the treatment by the treatment tool tracking module;
tracking the position of the doctor's arm holding the surgical tool during the operation by the operator's motion tracking module;
dividing the treatment area of the patient into matrix sections by the matrix scan module and scanning;
Projecting a camera image to the treatment site of the patient during the treatment by the multi-channel output module so that the operator can know the treatment site;
A step of confirming the treatment site by the display module; and
Controlling the surgical tool tracking module, the operator's motion tracking module, the matrix scan module, the multi-channel output module, and the display module by the control module; An image display method for direct projection onto the skin of the treatment site for

Images