JP2020134710A - Surgical operation training device - Google Patents

Abstract

To achieve a more superior location grasping property with respect to a monitor depth direction.SOLUTION: A surgical operation training device 10 synthesizes and display operation training images shot by an endoscope camera on a monitor for a teaching material image, and operates training-purposed operation gadgets so as to overlap on a movement of images of the operation gadget in the teaching material image. The surgical operation training device 10 comprises: a probing-purpose encounter type force sensing presentation instrument 6 that has a simulation body 60 providing a dimple surface having an elastic layer 601 on a surface, and a motor 63 causing the simulation body 60 to move in a three-dimensional space; and control units (7 and 73). The control unit is configured to obtain a space coordinate position of a contact position with a biological surface of the probing in the teaching material image, and a position on the dimple surface of the simulation body 60 corresponding to angle information on the biological surface as a surface coordinate position; and cause the simulation body 60 to move before replaying an image of the probing so as to match the surface coordinate position to the space coordinate position.SELECTED DRAWING: Figure 5

Description

The present invention relates to a surgical training device that supports training in endoscopic surgery using an endoscopic camera.

Laparoscopic surgery, which is a surgical operation performed by inserting an endoscopic camera and forceps into the abdominal cavity, is less invasive than open surgery and therefore less burdensome to patients, and is adopted by many medical institutions today. It has been popularized. However, since the forceps insertion hole (trocar) acts like a fulcrum of the "lever", a unique operation is required, and since the operation is performed while looking inside the abdominal cavity through the monitor, the direction and scale are changed (hand-). Surgery is difficult because it requires unique skills such as adaptation to eye coordination). Therefore, it is desired that a skilled doctor be on the side of the training of medical students, but there are not enough skilled doctors who have the ability to guide laparoscopic surgery. Robot-assisted surgery is also available as a method for performing laparoscopic surgery, but at present it is not significantly superior to laparoscopic surgery and will continue to be adopted as a major surgical method in many medical institutions. It is thought that it will be done. Further, a laparoscopic surgery training system using a visual field sharing method has been proposed for solving the problem of a shortage of skilled doctors (for example, Patent Documents 1 and 2).

The follow-up training system that applies the field-of-view sharing method can obtain real-time feedback of errors by superimposing the movements of the expert and their own movements in the shared field of view, so that their movements are always performed by the expert. It can be superimposed on the movement. Therefore, out of the four steps of "seeing, remembering, demonstrating, and modifying", which are the procedures of the usual training method, it is not necessary to "remember", and "seeing, demonstrating, and modifying" can be performed at the same time. it can. With the development of virtual reality technology in recent years, a laparoscopic-following training system that applies a field-of-view sharing method has been proposed as a simulator in the medical field.

International release W02015 / 097825 JP 2016-148765

However, the training methods described in Patent Documents 1 and 2 relate to the acquisition of basic ligation / suture skills, and are used to comprehensively acquire the skills necessary for surgery, including followability in the depth direction. It does not present a method or device.

Further, when the above-mentioned visual field sharing method is adopted, it is desirable to reproduce the surgical environment as a teaching material, but it is not easy to reproduce a complicated abdominal cavity. Furthermore, since laparoscopic surgery is performed by looking at the monitor, it is difficult to grasp the depth, and in actual surgery, it is thought that the force-tactile sensation (probing) that touches the organ supplements the depth information. In other words, in the experience-based laparoscopic surgery training device that efficiently learns skills by superimposing the practitioner's forceps on the actual surgery video of the expert, the surgery is performed while looking at the surgical field through the monitor, so it is on the monitor. There is a problem that the accuracy of the followability in the depth direction of the monitor is lower than the followability in the monitor surface direction. In actual surgery, it is thought that the depth direction of the monitor is acquired using both visual and tactile sensations, so in order to acquire the same level of skill as an expert, a laparoscopic-following training system can also be used. It is necessary for the trainee to grasp the space in the abdominal cavity, supplement the depth information, and learn the space grasp by the sense of touch as in the actual surgery.

As a method of tactile presentation, there are a method of creating and reproducing the same model as in the abdominal cavity, and a method such as a virtual reality simulator that reproduces all in a virtual environment. However, it is costly and technically difficult to make as many intraperitoneal models as necessary for training. Since the size of one virtual reality simulator is large and the cost is high, it is difficult to prepare a sufficient number of virtual reality simulators. Since there is a shortage of skilled doctors and prompt training of skilled doctors is an urgent issue, it is thought that multiple trainable devices will be required, so it is not possible to adopt a method that reproduces all in a virtual environment. , It is difficult to realize from the cost aspect.

The present invention has been made in view of the above, and provides a surgical training device that enables high position grasping in the depth direction of a monitor by reproducing probing with an encounter-type force-tactile presenter.

The surgical training device according to the present invention synthesizes a practice image being photographed by a practice camera on a monitor with a teaching material image of a living body performed by using a surgical instrument, which has been previously photographed by an endoscopic camera. In a surgical training device in which a surgical instrument for practice is operated so as to overlap with the movement of the image of the surgical instrument in the teaching material image, a simulated body having a concave surface having an elastic layer on the surface and the simulated body are displayed. It includes an encounter-type force-tactile presenter for probing, which has a drive unit for moving in a three-dimensional space, and a control unit. The control unit sets the position on the three-dimensional space corresponding to the contact position with the biological surface by the probing in the teaching material image as the spatial coordinate position, and the concave surface of the simulated body corresponding to the angle of the biological surface at the contact position. The upper position is set as the surface coordinate position, and the simulated body is moved before the reproduction of the probing image so as to match the surface coordinate position with the spatial coordinate position.

According to the present invention, it is possible to express the space 5 degrees of freedom excluding the roll rotation while controlling the space 3 degrees of freedom. Further, since the roll rotation does not contribute to shape recognition in the case of point contact, all the necessary degrees of freedom in space can be expressed. This presentation method enables contact with an elastic object, which was not considered in the conventional method, and expression of space 5 degrees of freedom by translation control of space 3 degrees of freedom, allowing the illusion of object shape in a virtual reality environment to be experienced. Is possible. In laparoscopic-following training, it is an illusion that a surgical instrument such as forceps is touching a visceral wall that does not actually exist during training. Furthermore, by obtaining the tactile sensation of touching the internal organ wall, it is possible to grasp the spatial position of the monitor depth with higher accuracy and acquire the skill of the skilled doctor as well as the skilled doctor.

Further, by making the concave surface of the simulated body a curved surface having a predetermined curvature, various contact angles with the living body at the time of probing can be reproduced only by movement control.

Further, if the concave surface of the simulated body has a semi-elliptical shape, all angles in the circumferential direction can be realized.

Further, by adopting a parallel link mechanism as the drive unit, a simple yet highly operable mechanism is configured.

Further, the driving unit is braked so as to keep the simulated body stationary during the probing period, so that careless movement during probing is suppressed.

Further, after the probing period ends, the driving unit moves the simulated body to a predetermined evacuation position in a direction away from the surgical instrument in the three-dimensional space. According to this configuration, the inconvenience that the surgical instrument carelessly comes into contact with the simulated body during the operation of the surgical instrument other than the probing period is prevented.

In addition, by making the surface of the simulated body black, it is difficult to visually recognize the simulated body from the practice image captured by the practice camera by a simple method, and the image of the internal organs such as the surgical target part in the teaching material image. Can be effectively or more reliably presented to the practitioner.

Further, the present invention includes a contact detection unit that detects contact between the surgical instrument for practice and the simulated body. According to this configuration, the success or failure of the probing operation can be confirmed.

According to the present invention, by reproducing the probing with the encounter type force tactile presenter, it is possible to grasp the position higher in the depth direction of the monitor.

It is a schematic block diagram which shows one Embodiment of the surgical training apparatus which concerns on this invention. It is a figure which shows one Embodiment of the encounter type force tactile presenter, (a) is the schematic block diagram of the side view of the encounter type force tactile presenter, (b) is the simulated body which presents the encounter type force tactile sense to probing. It is a figure explaining the coordinate system of the simulated body reference. It is a figure explaining the coordinate relation of the encounter type force tactile sense presentation by a simulated body corresponding to the position of the probing in the teaching material image imaged by an endoscopic camera. It is a schematic block diagram of the side view of the support structure of a practice camera. It is a block diagram which shows one Embodiment of the surgical training apparatus which concerns on this invention. It is a figure which shows an example of the memory map which stores the probing history. In the video diagram showing an example of the relationship between the movement of the practice camera and the monitor image, (a) is a state in which the practice camera has a shallow approach, and (b) is a state in which the practice camera has entered deeper. It is a flowchart which shows an example of the procedure of practice support processing. It is a flowchart which shows an example of the procedure of practice evaluation processing. It is a figure which shows the relationship between the presence or absence of the encounter type force tactile presentation and the position error, (a) is the case where there is an encounter type force tactile presentation, and (b) is the case where there is no encounter type force tactile presentation. It is a figure which shows the error amount of each axis about a probing time, (a) is the case of X axis, (b) is the case of Y axis, (c) is the case of Z axis.

First, the background and outline of the present invention will be described. A follow-up surgical training device that applies a field-sharing technique captures surgical instruments operated by the practitioner, such as forceps, with a camera attached to the tip of the robot, and the video is being played back on the monitor. The training is performed by superimposing it on the (teaching material video) and the practitioner chasing the forceps of a skilled doctor. At this time, the position of the camera that reflects the practitioner's forceps is measured in advance, and it is controlled so that it moves in the same manner as the position and posture at each time, which is imaged at the time of surgery and measured in advance. The forceps can always be superimposed on the screen as if they were inserted into the abdominal cavity from the same position as the surgery performed by a skilled doctor. As a result, the practitioner can always follow the movement of the forceps of a skilled doctor, and can acquire skills such as smooth movement. When the practice camera adopts a robot with three-dimensional freedom of position, that is, in a mode in which the rotation around the optical axis cannot be reproduced, the orientation of the practitioner's forceps image taken in the practice image matches the teaching material image ( The images were rotated and combined in the drawing program so as to be (angle).

The surgical training device can advance learning in the form of re-experiencing the surgery by superimposing the forceps of the practitioner on the surgical image of the skilled doctor and following it, but since the surgical image is two-dimensional, the depth Since the information is judged only by the size of the forceps on the image, the accuracy of following is lacking as compared with the direction of the monitor surface. In addition, before adopting the feedback method for tactile sensation, it can be inferred from the re-experience of tactile information obtained by a skilled doctor directly touching the internal organs with an electronic knife (an example of forceps) in the model image. It was difficult to obtain depth information. Therefore, we tried to reproduce the position of internal organs using an encounter-type force-tactile presenter. The original encounter-type force-tactile presenter measures the user's position and makes a simulated body that looks like a visceral wall stand by at the target position and posture before the user touches an object in the virtual environment. Is. Then, various shapes can be expressed by moving the simulated body according to the movement of the user during contact. When expressing an encounter with the visceral wall of forceps in laparoscopic surgery, the contact point must be soft. On the other hand, if the simulated body moves during contact with the soft body, an unexpected traction force may be applied to the user. Therefore, it is desirable to keep the simulated body stationary during contact with the forceps.

In order to improve the follow-up of the forceps in the depth direction of the monitor surface, the encounter with the visceral wall was realized with respect to the movement of the forceps in the depth direction of the monitor surface. That is, it was decided to use the probing motion as a scene for presenting the tactile sensation to the forceps. Probing is an operation of pressing forceps against a visceral wall to grasp the state of the visceral wall, and is considered to be an important operation in laparoscopic surgery. In addition, this probing refers to the work in the screen depth direction or the direction having the depth component.

In surgical training, it is easier to follow the monitor surface direction than in the depth direction, and in the probing scene, the part of the entire internal organ wall through which the tip of the forceps of the practitioner passes (presses) has a certain radius. It can be predicted from the preliminary experiment that it fits within the circle.

The surgical training device does not measure the position of the forceps in contact with the visceral wall for probing, and at the time of each probing operation, the simulated body is moved and controlled to stand by at the position of each visceral wall.

FIG. 1 is a schematic configuration diagram showing an embodiment of the surgical training device 10 according to the present invention. The surgical training device 10 includes a table-shaped base 2 having legs that match the height in the surgical position, with major components disposed on the surface of the table. The surgical training device 10 is arranged in the vicinity of the practice instrument section 3 and the practice instrument section 3 provided with a member for reproducing the practice space and the surgical state, and movably supports the practice camera 40 for imaging the practice space. It is configured to include an imaging unit 4, a monitor 5 that synthesizes and displays teaching material images and images captured by a practice camera, and an encounter-type force-tactile presenter 6 that movably supports the simulated body 60.

In the training equipment section 3, a T-shaped support member 31 is erected on one end side of the table surface, in this example, on the left portion, and a trocar is functionally simulated on an arm portion extending to the left and right at a predetermined height. Cylindrical annular bodies 32, 32 are attached at a predetermined distance. A trocar is an annular member provided in a circular hole opened in the abdomen of a surgical patient, and is used to insert and remove surgical instruments such as forceps (including an electronic knife) and an endoscopic camera into the abdominal cavity. In FIG. 1, it is assumed that the electronic knife 33 and the forceps 34 (hereinafter, referred to as forceps 33 and 34 for convenience) are inserted. The space near the tips of the forceps 33 and 34 is a practice space assuming a surgical space. The forceps 33 corresponds to an electronic scalpel that supplies a high-frequency current to the tip of a long body to heat-cut the affected part, and grasps the position and state of the part or its surrounding part before incision, excision, or the like. It is also used as a probing for confirmation. In addition to the actual forceps 33 and 34, a simulated instrument for practice may be adopted.

The imaging unit 4 is provided with a practice camera 40 corresponding to the endoscopic camera and a light source 401 for illumination that illuminates the imaging region at the tip. The main body housing portion 4A (see FIG. 4) of the imaging unit 4 is provided with a drive mechanism portion described in detail in FIG. The practice camera 40 captures the movement of the surgical instruments of the forceps 33 and 34 during practice, and enables the practitioner to observe the practice space and the practice situation via the monitor 5.

The monitor 5 is provided with a display screen set to a predetermined height so as to face the front of the practitioner practicing in the practice equipment unit 3. On the monitor 5, the teaching material image and the practice image captured by the practice camera 40 are combined and displayed. As the composite display method, various display forms can be adopted as long as it is a visual field sharing method, and a time division method or a translucent composite method is preferable. The space of the teaching material image displayed on the monitor 5 has an X-axis in the horizontal direction, a Y-axis in the vertical direction, and a Z-axis in the depth direction. The monitor 5 may be a head-mounted HMD (Head Mount Display) as well as a stationary type.

The encounter-type force-tactile presenter 6 is provided with a simulated body 60 at the tip, which simulates the internal organs of a living body. The main body housing portion 6A (see FIG. 2) of the encounter-type force-tactile presenter 6 is provided with a drive mechanism portion described in detail in FIG. The simulated body 60 is driven so as to appear and disappear with respect to the practice space of the practice equipment unit 3.

2A and 2B are views showing one embodiment of the encounter type force tactile presenter 6, in which FIG. 2A is a schematic configuration diagram of a side view of the encounter type force tactile presenter 6, and FIG. 2B is an encounter type force with respect to probing. It is a figure explaining the coordinate system of the simulated body 60 which presents a tactile sense and the simulated body reference.

In FIG. 2A, the encounter-type force-tactile presenter 6 is arranged above the base 2, and in the present embodiment, a known parallel link mechanism is adopted as the mechanism 62. The encounter-type force-tactile presenter 6 includes a disk-shaped base portion 61 supported by the main body housing portion 6A, a mechanism portion 62 supported on the side surface of the base portion 61, and a motor as an actuator for moving the mechanism portion 62. 63 and a simulated body 60 attached to the tip of the mechanical portion 62 via an attachment member 64 are provided.

The mechanical portion 62 has a drive link 621 rotatably connected to the side surface of the base portion 61 around a base point, a passive link 623 connected to the drive link 621 via a spherical joint 622, and a spherical joint to the passive link 623. It includes a movable plate 625 connected via a 624. The motor 63 may be an actuator having a cylinder structure that expands and contracts the length dimension of the mechanism portion 62 connected to the base portion 61. The mechanism unit 62 and the motor 63 each have a configuration for three axes. The simulated body 60 has a concave surface on the front surface side, for example, a semi-elliptical spherical concave surface, and is connected to the mounting member 64 on the outer surface side of the rear portion which is the top of the ellipse. The concave surface is preferably a semi-elliptical sphere, a paraboloid or a hyperboloid, or a curved surface having a curvature, that is, a curved surface in which the surface angle changes continuously.

The simulated body 60 is made of, for example, a resin material or a metallic plate material, and the concave surface thereof reproduces a situation close to the softness and elasticity of the internal organs of the living body, that is, a similar feeling of contact through a surgical instrument. Therefore, an elastic layer 601 to which an elastic resin material or gel-like material is applied or adhered is formed. In addition, an adhesive material may be included. Further, by adjusting the weight of the simulated body 60 or the braking force (rigidity) of the simulated body 60 by the motor 63, it is possible to approach the situation closer to the internal organ wall.

Further, since the simulated body 60 is located in the practice space, enters the angle of view of the practice camera 40 and is imaged, it is combined with the teaching material image of the actual surgery and displayed on the monitor 5 as it is. Therefore, in order for the practitioner to practice properly, the simulated body 60 cannot be visually recognized on the monitor, while the forceps 33 and 34 to be operated are subjected to visibility improving treatment such as increasing light reflection on the surface. It needs to be clearly visible. Therefore, the surface of the simulated body 60 is painted with, for example, a black matte lacquer by spraying or the like, while a reflective material is applied to the tip side of the forceps, etc., and the brightness, contrast, and white balance are set in the imaging settings of the practice camera 40. It is desirable that adjustments such as are made. As a result, the video signal captured by the practice camera 40 is superimposed on the monitor 5 without processing by the image synthesis software, and the video signal is superimposed on the built-in image and the forceps image of the teaching material video in an environment where the delay of the video is minimized. , The image of only the forceps 33 and 34 for practice can be visually displayed.

The mounting member 64, that is, the simulated body 60, can be set to an arbitrary position and orientation in space by driving the motor 63 of each axis. As shown in the present embodiment, in the embodiment in which the concave surface of the simulated body 60 is formed axisymmetrically, all the orientations can be set as described later.

In the method using the concave simulated body 60, it is possible to present an arbitrary shape, that is, an orientation (angle of the surface) only by controlling the position of a small device. Since forceps are used in the surgical training device 10, when the convex side of the semi-elliptical sphere is adopted as the simulated shape of the internal organ wall, the forceps 33 may not come into contact with each other depending on the approach angle. That is, when the simulated shape of the internal organ wall becomes larger from the center of the ellipse to the outside in the direction of the approach axis of the forceps 33 than the inclination of the tangent line of the contact point of the convex surface, the contact fails. On the other hand, when the simulated shape of the internal organ wall is a concave surface, the tangential direction at all points that can be contacted and the approach axis direction of the forceps 33 always intersect, so that there is no failure condition in the convex surface type. By making the shape a concave surface of a semi-elliptical sphere like the simulated body 60, the inclination of the contact point can be continuously changed, and the contact object (forceps 33) can be touched only by controlling the position of the simulated body 60. Since it is possible to present an arbitrary angle, even if the contact object and the simulated body 60 are misaligned, the tilt error is small. Further, the smaller the curvature of the semi-elliptical sphere (strictly speaking, the contact point), the smaller the difference between the contact point and its vicinity, and the reality is less likely to be impaired. The curvature and size of the simulated body 60 can be designed according to the environment in which this method is used.

Further, by making the simulated body 60 a semi-elliptical sphere, since the concave surface has a continuous inclination, the attitude control becomes unnecessary, and the space 5 degrees of freedom can be expressed only by the position control. Therefore, by presenting the encounter-type force-tactile sensation by a semi-elliptical sphere, it is possible to express a space having 5 degrees of freedom by moving a point having an arbitrary inclination with 3 degrees of freedom in space. The method presented using this semi-elliptical sphere can suppress the degree of freedom of control of the simulated body 60 to 3 degrees of freedom in space, and gives a person the shape of an elastic object that could not be obtained by the conventional method. It is thought that the learner's monitor depth will be learned by laparoscopic-following training. As described above, the method according to the present embodiment can express the degrees of freedom of parallel movement and rotation required for shape recognition only by parallel movement, that is, the space 5 degrees of freedom can be expressed by controlling 3 degrees of freedom. Is.

When the spatial coordinate system of the simulated body 60 is defined as shown in FIG. 2 (b), if the angles (θ, φ) (0 ≦ θ ≦ π, 0 ≦ φ ≦ π) to be contacted are determined, the simulated body The spatial coordinates (x, y, z) of the concave surface of 60 are uniquely determined by the following equation (1).

p (x, y, z) = (r (1-sinθ sinφ), rcosθ cosφ, rcosθ) ・ ・ (1)
Further, when designing the encounter type force tactile presenter 6 and the simulated body 60, it is only necessary to input the vector from the mounting member 64 to the origin O as an offset portion, so that there are three degrees of freedom in space even in actual application. You only need to set.

The contact angle (θ, φ) is, in other words, the direction vector of the returning force. That is, the elastic force returns a force in the normal direction of the contact point, and a force is generated as a frictional force in the shear direction of the contact point.

FIG. 3 is a diagram illustrating a coordinate relationship of encounter-type force-tactile presentation by a simulated body corresponding to a probing position in a teaching material image captured by an endoscopic camera. More specifically, in FIG. 3, the probing position Qp in the teaching material image Lb captured by the endoscopic camera is measured or manually specified and acquired, and the captured image at one or a plurality of positions of the endoscopic camera is obtained. , And the real space position P (X, Y, Z) of the probing and the angle (α, β) of the surface Or which is the posture from the real space position corresponding to one or more positions of the endoscopic camera and its posture. To get. Next, the surface coordinates S (x, y, z) of the simulated body 60, which is the (matching) angle (θ, φ) corresponding to the surface angle (α, β), and the simulated body 60 are moved to move the surface. An encounter-type force-tactile presentation method is shown in which the position of the coordinate S is adjusted to the real space position P (X, Y, Z) of the probing (that is, the movement is controlled). By associating the angle of the probing position (α, β) with the surface angle (θ, φ) of the simulated body 60, the feeling during probing, for example, the elasticity in the plane direction caused by pressing against the elastic layer 601. Deformation and stress can be reproduced more realistically.

FIG. 4 is a schematic configuration diagram of a side view of the support structure of the practice camera 40. The image pickup unit 4 includes a disk-shaped base portion 41 supported by the main body housing portion 4A, a mechanism portion 42 supported on the side surface of the base portion 41, a motor 43 as an actuator for moving the mechanism portion 42, and a mechanism portion. A practice camera 40 or the like attached to the tip of 42 via an attachment member 44 is provided. The mechanism unit 42 has a drive link 421 rotatably connected to the base portion 41, a passive link 423 connected to the drive link 421 via a spherical joint 422, and a spherical joint 424 to the passive link 423. It includes a movable plate 425 connected via a movable plate 425.

The practice camera 40 obtains a driving amount from the motion history information acquired in the teaching material creation by using inverse kinematics, and supplies the driving amount to the motor 43 of each axis to create teaching materials such as actual surgery. It is possible to reproduce the movement history of the endoscopic camera at that time. In the above, if the length dimensions of the practice camera 40 and the endoscope camera are the same, the calculated drive amount can be applied as it is, while if the length dimensions are different, the drive amount is driven according to the difference. The amount may be corrected.

FIG. 5 is a block diagram showing an embodiment of the surgical training device 10 according to the present invention. The surgical training device 10 includes a control unit 7 composed of a computer. The control unit 7 is connected to the storage unit 70, and is also connected to the image pickup unit 4, the monitor 5, and the encounter-type force-tactile presenter 6. The teaching material creation unit 8 preferably includes a computer and creates teaching material images and related information from surgical images as teaching materials. The created teaching material video and the like are stored in the storage unit 70. Details will be described later.

As will be described later, the storage unit 70 stores information about the teaching material and a control program for executing surgical training. The storage unit 70 includes a teaching material video storage unit 701, a camera movement history storage unit 702 for teaching materials, and a probing history storage unit 703 for storing the information created by the teaching material creation unit 8.

The teaching material image storage unit 701 stores the teaching material image captured by the endoscopic camera. The teaching material camera movement history storage unit 702 stores the movement history (spatial coordinates, etc.) for making the position and orientation of the practice camera 40 correspond to the position and orientation of the teaching material camera. In the mode of controlling only the movement of the practice camera 40, the image is rotated in the opposite direction and guided to the monitor 5 in response to the change in orientation. The probing history storage unit 703 has real space coordinates P (X, Y, Z) corresponding to the probing position corresponding to the order of probing (elapsed time from the start of playback of the teaching material video, etc.) occurring in the teaching material video. And the surface angle (α, β), and the surface coordinates S (x, y, z) and the surface angle (α, β) on the simulated body 60 are stored in the data table format as shown in FIG. To do. The data to be stored may include at least the time t, the actual coordinates P, and the surface coordinates S on the simulated body 60.

When the teaching material creation unit 8 is configured to include a computer, the teaching material creation unit 8 functions as a teaching material information creation unit 81 that creates teaching material information from the teaching material video 82 by executing a creation processing program. Further, as an operation unit for instructing the creation of teaching material information as needed, a keyboard or the like, or a touch panel for displaying various function buttons on the screen may be provided. The teaching material information stored in the storage unit 70 is created by the teaching material information creating unit 81. It is preferable that the teaching material information creation unit 81 uses a known Visual SLAM (Simultaneous Localization and Mapping) process in addition to the mode in which the manual process is partially used. According to this process, it is a technology that simultaneously estimates the environment, that is, the internal organs and other three-dimensional information in the living body and the position and orientation of the teaching material camera from the surgical images taken by the teaching material camera. Then, by utilizing the fact that the real space coordinates and posture of the teaching material camera are known and applying the known coordinate position and posture information, the position and surface angle in the three-dimensional space such as internal organs can be obtained. Is possible.

By executing the control program, the control unit 7 executes an image creation unit 71, a camera position drive control unit 72, an encounter type force tactile presenter drive control unit 73, a probing detection unit 74, a motion information acquisition unit 75, and a practice evaluation. Functions as part 76.

The video creation unit 71 reads the teaching material video from the teaching material video storage unit 701 in the time direction, synthesizes it with the practice video captured by the practice camera 40, and outputs it to the monitor 5 via a video RAM (not shown). .. As a result, the practitioner can practice the chase operation so as to superimpose the forceps 33 and 34 of the monitor 5 on the corresponding forceps 133 and 134 (see FIG. 7) in the teaching material image. The video composition may be a visual field composition method in which the two images are superimposed in addition to the mode in which the two images are displayed alternately in a time division manner. Both images may be processed as they are or in an identifiable display manner.

The camera position drive control unit 72 outputs an instruction signal to the motor 43 to move the practice camera 40 to the same position or the like according to the information of the teaching material camera movement history storage unit 702. For the rotation information of the teaching material camera, the video creation unit 71 performs rotation processing of the practice video.

The encounter-type force-tactile presenter drive control unit 73 clocks the time from the start of practice with an internal timer or the like, and while referring to the table of FIG. 6, immediately before the time when the probing event occurs, FIG. An instruction to move the simulated body 60 to the set position in advance from the table is output to the motor 63. As a result, the simulated body 60 can stand by at the planned probing position, and the encounter-type force-tactile presentation can be realized. Therefore, when the practitioner properly performs the probing operation, the practitioner receives a simulated reaction force from the internal organ wall via the forceps 33, whereby the practitioner can experience a sense of depth and the depth direction is higher. It is possible to feel with accuracy.

Further, the encounter-type force-tactile presenter drive control unit 73 sets a predetermined time width with the probing time point indicated by the teaching material in between, and during that time, the simulated body 60 is stationary to wait and accept the operation from the practitioner. I try to do it. Then, when that time elapses, the forceps 33 are retracted to a position where the forceps 33 cannot reach, for example, to the depth side so that the forceps 33 does not inadvertently hit the simulated body 60 during the period when there is no probing. .. The retracted position may not be fixed and may be set with respect to the next probing position. Further, by providing a stationary period, the simulated body 60 moves during the probing operation so that the reaction force from the simulated body 60 does not change unnaturally.

In the probing detection unit 74, when the motor 63 is, for example, a servomotor or the like, when the simulated body 60 receives the pressing force of the probing operation, a current for maintaining the position flows in the drive signal supply circuit system of the motor 63 by the force. Proper probing is detected by detecting this current. Alternatively, the probing detection unit 74 may determine the success or failure of the probing by detecting the spatial coordinates of the tip of the forceps 33 from the practice image and comparing it with the probing coordinate positions.

The motion information acquisition unit 75 continuously detects the spatial coordinates of the tips of the forceps 33 and 34 from the position and orientation information of the practice camera 40 and the practice image. In addition to the method of analyzing from images, the motion information acquisition unit 75 is known to attach an optical or magnetic signal source to the tips of forceps 33 and 34 and measure the position of the source three-dimensionally from the surroundings. The sensor may be used, or a sensor having a 3-axis accelerometer attached to the tips of the forceps 33 and 34 may be used.

The practice evaluation unit 76 calculates, for example, a numerical value the superposition state of the forceps 133 and 134 in the teaching material image and the corresponding forceps 33 and 34 in the practice image from the superimposition (chasing) situation with the teaching material image, which will be described later. Evaluate the practice results as you do. The practice evaluation unit 76 can include the success or failure of probing.

FIG. 7 shows an example of a composite image during practice. FIG. 7A shows a case where the practice camera 40 is on the front side, and FIG. 7B shows a case where the practice camera 40 is inserted on the back side. FIG. 7 shows a surgical example of the sigmoid colon as an example of surgery. In FIG. 7A, the practice camera 40 is positioned at a shallow position where almost the entire colon image B1 can be seen, and in FIG. 7B, the practice camera 40 is moved further back, that is, the rectal images B2 and S. It is close to the site with the character colon image B3. In this way, as the practice camera 40 moves, the image size and orientation of the surgical site change, and the forceps 133, 134, 33, 34, which are surgical instruments on the screen, also change in image size, display range, and line of sight ( The appearance has changed, including the orientation).

As a result of the position of the practice camera 40 being reproduced by the motor 43 in the movement history of the endoscope camera at the time of creating the teaching material, the image is displayed on the monitor 5 with the same viewpoint and line of sight as at the time of creating the teaching material. In the figure, forceps 33 and 34 are images of forceps operated by the practitioner. Even if the appearance of the forceps 133 and 134 on the teaching material side changes significantly with the movement of the practice camera 40, the viewpoint and line of sight of the practice camera 40 are matched in order to reproduce the movement history of the forceps 133 and 134. It comes to match the appearance, and as a result, the training of the operation superimposed on the movement of the forceps 133 and 134 on the teaching material side becomes significant. In the above, the surface condition and position of the colon are confirmed by operating forceps 133 to perform probing on the biological surface of the sigmoid colon, which is the surgical site, before incision or excision.

FIG. 8 is a flowchart showing an example of the procedure of the practice support process. First, when the reproduction of the teaching material image is started (step S1), the teaching material image is read out in the time direction, the practice image from the practice camera 40 is read out, both images are combined, and output to the monitor 5. (Step S3).

Next, the probing information at the next timing is read in advance from the data table shown in FIG. 6 (step S5), and the simulated body 60 is moved to the set position based on the read information to be in the standby state (step). S7). Subsequently, the process of confirming the presence or absence of the probing operation is performed by using the method described above (step S9), and the probing evaluation is performed according to the success or failure (step S11). Next, with reference to the data table, the presence or absence of the next probing is determined (step S13), and if there is the next probing, the process returns to step S5 and the same process is repeated, while if there is no next probing, , End this process. In addition, step S9 and step S11 may be adopted as needed.

FIG. 9 is a flowchart showing an example of the procedure of the practice evaluation process. In the present embodiment, the practice evaluation is performed by interrupting at predetermined time intervals, for example, every second. First, it is determined whether or not it is at the time of interruption (evaluation time) (step S21), and if it is not at the time of interruption, the flow is exited. Reading is performed (step S23). Next, the tip positions of the forceps 33 and 34 for practice are detected (step S25). Then, the difference between the tip positions of both forceps 133 and 33 and 134 and 34 is calculated (step S27). Then, the practice result is evaluated based on the calculated difference (step S29), and the result is returned. As the evaluation method, a method based on the integrated value of the difference, a method based on the increase / decrease status of the difference, and other general evaluation methods can be adopted.

10 and 11 are charts showing the results of an "experiment" regarding the significance of the monitor surface in the depth direction.

The "experiment" is divided into important tasks from the surgical images of actual human sigmoid colectomy, and a series of segments are continuously followed in each task. In this experiment, the work of "inner approach of sigmoid mesentery, confirmation of gonadal arteries and veins" was adopted as a teaching material in the surgical video. This scene included a lot of probing work (11 times in about 1 minute). Since the practitioner can judge only by the size of the forceps on which the depth direction of the monitor is superimposed, it can be expected that the depth information can be obtained not only visually but also by touch by attaching the encounter type force tactile presenter.

"Experimental conditions"
In the experiment, comparison was made under the conditions of using and not using the encounter type force tactile presenter. Before the experiment, the experimenter gave a lecture on the system explanation and forceps operation to the experiment participants, and then the experimenter gave a demonstration only once. In each trial, with or without the encounter-type force-tactile presenter, a still image of the first frame of the teaching material image is first displayed on the monitor, and the subject superimposes as close as possible to the superimposition teaching displayed in the upper right of the screen. , When it was judged that it could be superimposed, the experimenter was asked to declare it, and the video was played after the declaration. This is to prevent the subject from losing sight of his forceps at the start of the video. As a strategy for superimposing the forceps that serve as a model during training and their own forceps, the size of the forceps tips was instructed to be the same on the screen.

Both conditions were tried 16 times, with a 1-minute break between each trial. At that time, he was instructed to take his hand off the forceps so that he could not learn hand-eye coordination outside the experiment. The participants in the experiment were men in their twenties in engineering who had no medical expertise and no prior knowledge of laparoscopes, and there were a total of 6 people, 3 for each condition.

"Evaluation criteria"
In the experiment, the coordinates of the tip of the forceps operated by each subject with the right hand were measured. For the tip position of the right hand forceps, the error from the model position was calculated as RMSE, and the comparison was made with and without encounter-type force tactile presentation. Here, the position serving as a model is the position of the tip of the right forceps measured by superimposing forceps at each time by preparing a still image every second for the moving image used in the experiment. In addition, in this experiment, in order to verify whether the correct depth information could be learned by the encounter-type force-tactile presenter, the position at the time of probing was particularly verified.

"Experimental result"
First, the amount of position error is shown in FIGS. 10A and 10B in the three directions of the screen up / down / left / right and the depth at each time of the one-minute moving image. The horizontal axis is the number of trials, and the vertical axis is the amount of error. The left-right direction of the monitor is the X-axis, the up-down direction is the Y-axis, and the depth direction is the Z-axis.

It seems that both methods with and without tactile sensation can follow the X and Y axes because the amount of error is small. On the other hand, regarding the Z-axis, the group without tactile sensation is compared with the group with tactile sensation as shown in (s1 / Z-axis), (s2 / Z-axis), and (s3 / Z-axis) in FIG. 10 (b). The amount of error is large. This indicates that learning in the monitor depth direction is more difficult than in the monitor plane direction, and indicates that the monitor depth information is supplemented by the sense of touch.

Next, FIG. 11 shows a graph comparing the amount of position error in each axial direction between the group of the training advance and the proposed system for one of the 11 probing points (second). In FIG. 10, the amount of error with respect to all the times was confirmed, but the error with respect to the scene of the technique for the purpose of mastery was obtained. For ± 0.5 seconds from the moment the forceps touched the visceral wall, the amount of error and variation from the position at the moment of touching were calculated.

In FIGS. 11A, 11B, and 11C, the horizontal axis represents the number of trials and the vertical axis represents the amount of error. Error bars indicate standard deviation. From FIGS. 11A, 11B, and 11C, there is little error amount and variation in the X and Y axis directions regardless of the presence or absence of tactile sensation, but there is tactile sensation in both the error amount and variation when there is no tactile sense in the Z axis direction. It turns out that it is larger than the case.

"Discussion"
From FIG. 10, it was found that learning was progressing at the beginning of the training in terms of followability in the vertical and horizontal directions of the screen in both conditions with and without the encounter-type force-tactile presenter. This means that the learning of hand-eye coordination was established early in the horizontal and vertical directions of the monitor, and it was shown that it was not hindered by the addition of the simulated body.

In addition, regarding the screen depth direction, a large difference appeared between the conditions with and without the encounter-type force-tactile presenter. That is, the group without the encounter-type force-tactile sense has a large error amount in the depth direction and the position tends to converge, whereas the group with the encounter-type force-tactile sense presenter has a small error amount and the position also converges. I found a tendency to do it. That is, it is considered that the practitioner can acquire the correct depth information by presenting the virtual position of the internal organs by tactile sensation.

Next, it was compared whether the position at the time of pushing (probing) of the forceps, which is required to learn the correct depth information, could be learned correctly. Regarding the second of the 11 probing scenes that appear in the video used in the experiment, the error amount of the position of the forceps tip from the time when the probing starts to the time when the probing ends is shown in each axial direction in FIG. 11A (a). b) Shown in (c). The solid line in the figure is the group with the encounter type force tactile presenter, and the broken line (c) is the group without the encounter type force tactile presenter. FIGS. 11 (a) and 11 (b) show the error amounts in the vertical and horizontal directions of the screen, respectively, and it was found that the probing was followed at similar positions in both groups. FIG. 11C is a plot of the screen depth direction, and the group without the encounter-type force-tactile presenter has a larger variation in the error amount among the subjects than the group with the encounter-type force-tactile presenter. I understand. This means that when following the action of pushing the forceps into the back of the screen, the difficulty of grasping the position of the internal organs touched by the forceps differs greatly depending on the presence or absence of tactile sensation, and if there is no tactile feedback, the positions differ between the subjects. It shows that learning progresses with. That is, it seems difficult to obtain accurate depth information simply by superimposing the tips of the forceps on the screen.

"Summary"
In this experiment, in order to improve the difficulty of obtaining information on the depth direction of the screen in the follow-up laparoscopic surgery training system using the visual field sharing method, a system that reproduces the position of internal organs was constructed using an encounter-type force tactile presenter. did. In the constructed system, it was confirmed that the amount of error of the subjects and the variation among the subjects in the reproducibility of the screen depth direction were reduced when the forceps were followed. From this, it was found that the depth direction can be learned at a more correct position. This means that the depth information that was missing on the 2D monitor is stored, and it is expected that the practitioner's surgical error will be reduced because the 3D positional relationship of the internal organs can be correctly grasped.

In the present embodiment, the parallel link mechanism is adopted as the mechanism units 42 and 62 to realize the movement in the three-axis direction, but the movement is not limited to this, and the mode in which the three-axis slide rail is assembled or the three-axis type A mode equipped with the gimbal mechanism of the above may be adopted.

Further, in the present embodiment, the acquired teaching material video is stored in the storage unit 70, but instead of this, the teaching material video and the teaching material-related information are surgically trained from the external storage unit in real time by using the communication unit. It is also possible to transfer to the device 10 and practice.

Further, in the present embodiment, the operation of the sigmoid colon has been described as an example having a surgical process, but in addition, a series of operations having a surgical process using an endoscope, or general training for a series of other surgical modes. Applicable to.

Further, in the present embodiment, the semi-elliptical sphere has been described, but here, in addition to the meaning of exactly half of the elliptical sphere, a substantially half aspect may be included.

10 Surgical training device 3 Practice instrument part 33, 34 Forceps 4 Imaging part 40 Practice camera 5 Monitor 6 Encounter type force tactile presenter 43, 63 Motor 60 Simulated body 601 Elastic layer 7 Control unit 73 Encounter type force tactile presenter Drive control Department (control unit)
70 Memory

Claims (8)

The practice video taken by the practice camera is combined and displayed on the monitor with the teaching material video of the treatment performed on the living body using the surgical instrument, which was previously taken by the endoscopic camera, and displayed in the teaching material video. In a surgical training device that operates a surgical instrument for practice so as to overlap the movement of the image of the surgical instrument
An encounter-type force-tactile presenter for probing, which has a simulated body having a concave surface having an elastic layer on its surface, and a driving unit for moving the simulated body in a three-dimensional space.
The position on the three-dimensional space corresponding to the contact position with the biological surface by the probing in the teaching material image is set as the spatial coordinate position, and the position on the concave surface of the simulated body corresponding to the angle of the biological surface at the contact position is set. A surgical training device including a control unit that is set as a surface coordinate position and moves the simulated body before playing back the probing image so as to match the surface coordinate position with the spatial coordinate position. The surgical training device according to claim 1, wherein the simulated body is a curved surface in which the concave surface has a predetermined curvature. The surgical training device according to claim 2, wherein the simulated body has a semi-elliptical concave surface. The surgical training device according to any one of claims 1 to 3, wherein the drive unit is a parallel link mechanism. The surgical training device according to any one of claims 1 to 4, wherein the driving unit is a stationary body during a probing period. The surgical training device according to any one of claims 1 to 5, wherein the driving unit moves the simulated body to a retracted position in a direction away from the surgical instrument in a three-dimensional space after the probing period ends. The surgical training device according to any one of claims 1 to 6, wherein the simulated body has a black surface. The surgical training device according to any one of claims 1 to 7, further comprising a contact detection unit that detects contact between the surgical instrument for practice and the simulated body.