JP6014450B2 - Motion learning support device - Google Patents

Description

  The present invention relates to a motion learning support apparatus that learns through a series of motion teaching material images including a plurality of motion elements connected in the time direction, such as laparoscopic surgery training.

  In recent years, for example, laparoscopic surgery that does not require incision in organ surgery, for example, has a merit that the burden on the patient is light and quick recovery is effective, especially for elderly people and children who have insufficient physical strength Popularization is desired. However, since laparoscopic surgery requires advanced techniques, the number of specialists is not necessarily sufficient, and training of skilled workers is an important issue. Conventionally, a training method using a dry box has been proposed as a training method therefor. However, this training method only simulates surgery (simulation). When a skilled person teaches a skill to a non-experienced person, the monitor screen is usually pointed out with a pointing or verbal instruction. It was enough to repeat trial and error by dry box training without fully understanding the knack. For example, Patent Literature 1 describes a surgical training simulator that synthesizes and displays both student and teacher images for a surgical simulation.

  Further, as a higher-order system, a computer simulation and a forceps-type tactile sense presentation interface for an endoscope are realized, and for example, a medical simulator LAP Mentor (Non-patent Document 1) manufactured by Symbionics is cited. In the remote operation system da Vinci (Non-Patent Document 2), an attempt is made to monitor the situation during the operation, convert it into data, and apply it to simulation and training.

JP 2005-525598 A

http://www.tri-med.co.jp/ http://www.kyudai2geka.com/contents/davinci.html

  However, Patent Document 1 only describes that both videos of students and teachers are synthesized and displayed. Also, from Non-Patent Documents 1 and 2, from the viewpoint of acquiring skills on what can be done and how well when training a trainer, it is possible to use verbal or linguistic instructions from an educator who observes the trainer. Since it is mainly based on feedback and trial and error learning by the trainee, it takes time to learn and there is a risk that incorrect habits may be attached.

  The present invention provides a motion learning support device that can acquire a correct motion in a short time and effectively by supporting step-by-step learning for learning each motion element and learning for tracking a series of motions. This is a proposal.

  A motion learning support device according to the present invention is a motion learning support device that learns through a series of motion teaching material videos including a plurality of motion elements connected in the time direction, the monitor displaying the video, the teaching material video, An imaging unit that captures the movement of the learner at the same shooting position, and a motion element video corresponding to the motion element of the teaching material video are individually displayed on the monitor, and the display of the motion element video is followed by the imaging First learning support processing means for displaying the captured image of the image on the monitor, and second learning support processing for combining and displaying the teaching material image of the series of movements and the captured image of the image capturing unit on the monitor Means.

  According to this invention, the first learning support processing means individually displays the motion element video corresponding to the motion element of the teaching material video on the monitor, and then displays the captured video of the imaging unit on the monitor. Accordingly, the learner can visually recognize the motion element video and can practice the motion element displayed immediately before while observing his / her own video displayed on the monitor. And it becomes possible to understand each procedure by completing each procedure learning. Next, the learning material video of a series of movements and the imaging video of the imaging unit are combined and displayed on the monitor, so that the learner shows his / her movements displayed on the monitor while referring to the teaching material video of the movements. You can practice to match the movement of the teaching material video. Therefore, it is possible to learn the correct motion in a short time and effectively by supporting the procedure learning for learning each motion element and the learning for following a series of motions in stages.

  According to a second aspect of the present invention, in the motion learning support apparatus according to the first aspect, the second learning support processing unit is permitted to be executed after the execution of the first learning support processing unit. It is characterized by. According to this configuration, since the second learning support process is performed after the first learning support process, first, learning is performed on the motion element, and then a series of movements is learned. Is provided.

  According to a third aspect of the present invention, in the motion learning support device according to the first or second aspect of the present invention, the movement learning support device further includes a motion element image creating means for extracting each motion element image from the teaching material image including the series of movements. To do. According to this configuration, each motion element is automatically created, which is effective.

  According to a fourth aspect of the present invention, in the motion learning support device according to any one of the first to third aspects, the first learning support processing means includes a final motion element image corresponding to the captured image of the imaging unit. A frame image is displayed together on the monitor. According to this configuration, since the learner understands the final state of the movement element being learned, the learner can move smoothly.

  According to a fifth aspect of the present invention, in the movement learning support device according to any one of the first to fourth aspects, the learner has an operation unit that can be operated, and the first learning support processing means includes one movement. After the element video is displayed on the monitor, the captured video of the imaging unit is displayed on the monitor, and when an operation to the operation unit is received, the next movement element video in the time direction is displayed on the monitor. It is a thing to do. According to this configuration, it is possible to shift to learning of the next motion element when it is determined that one motion element is mastered, and effective learning is provided.

  A sixth aspect of the present invention is the motion learning support device according to any one of the first to fifth aspects, wherein the second learning support processing means includes the series of motion instructional images, the captured image of the imaging unit, Are alternately displayed on the monitor. According to this configuration, by adopting a so-called time-division display method, an apparatus that exhibits exercise inductivity and improves learning efficiency is provided.

  The invention according to claim 7 is the motion learning support apparatus according to any one of claims 1 to 6, wherein the teaching material image of the series of movements is an endoscopic camera using a ligature suture using forceps during laparoscopic surgery. It is the video image | photographed in. According to this configuration, there is provided an apparatus in which the correct skill of a ligature suture treatment using forceps is effectively improved in a short time.

  ADVANTAGE OF THE INVENTION According to this invention, the movement learning assistance apparatus which can acquire a series of model movements correctly in a short time effectively can be provided.

It is a schematic block diagram which shows one Embodiment of the movement learning assistance apparatus which concerns on this invention. It is a circuit block diagram which shows one Embodiment of the movement learning assistance apparatus which concerns on this invention. It is a figure which shows the example of the frame image | video of a procedure learning, (A) is a frame image which picks up a needle | hook, (B) is a frame image | video which inserts a needle | hook, (C) is a figure of the frame image | video which shows the state which winds the suture thread. . It is the schematic for demonstrating the time division display of a learning material image | video (A) and a learner's live image | video (B). It is a flowchart which shows one Embodiment of a procedure learning teaching material preparation process. It is a flowchart which shows one Embodiment of a procedure learning teaching material display mode process. It is a flowchart which shows one Embodiment of a follow-up learning teaching material display mode process. It is a flowchart which shows one Embodiment of the time division switching display process in a follow-up learning teaching material display mode process. The chart for explaining the effect, (A) is a chart showing the transition of the time required for a series of operations performed by each experimenter according to the progress of learning, (B) is the comparison experimenter and the learning end It is a graph which shows the difference of the time required for each series of operation | movement by each experimenter after. The chart for explaining the effect, (A) is a chart showing the transition of the number of errors (movement errors) caused during each series of operations performed by each experimenter according to the progress of learning, and (B) is a comparison. It is a graph which shows the difference of the average of the error number which arose during each series of operation | movement by an experimenter and each experimenter after completion | finish of learning.

  FIG. 1 is a schematic configuration diagram showing an embodiment of a motion learning support apparatus according to the present invention. The movement learning support apparatus 1 can be applied to an apparatus that learns various movements. In the following, the motion learning support device 1 will be described as a device applied to support for learning (practice) of ligature suture motion using forceps in laparoscopic surgery.

  In FIG. 1, the motion learning support device 1 includes a practice table device 10 and an information processing device 20. The practice table device 10 includes a dry box 11 that simulates a patient's torso, a base 12 on which the dry box is placed at a predetermined position above, and a monitor 13 that displays an image. The dry box 11 has a cylindrical body 111 that is horizontally oriented and has a flat bottom surface. The housing 111 has an upper surface portion 112 formed in a convex curved surface so as to simulate the abdomen of a human body. The upper surface portion 112 is made of an opaque or translucent member, and a plurality of circular holes are formed at appropriate positions. A film material 113 such as a resin material is affixed to the circular hole, and a cross-shaped notch simulating a cylindrical trocar for inserting and removing forceps and an endoscope camera, which will be described later, is formed in the center of the circular hole. ing. In addition, the aspect which directly arrange | positions the structure which simulated the trocar may be sufficient.

  The forceps 114 is inserted from above through the cross-shaped cut of the membrane material 113. As is well known, for example, in the case of ligation, the forceps 114 has a pair of finger insertion portions on the hand side and a pair of openable / closable action portions (a pair of clamping members) on the other side. They are connected by a pipe portion having a predetermined length. The pipe part is internally provided with a mechanism for transmitting the operation force at the finger insertion part to the action part. By such a force transmission mechanism, a pair of action portions at the front end is opened and closed in conjunction with an opening / closing operation from the finger insertion portion by the user, so that the object can be clamped or opened. The forceps 114 can open a working portion so that a later-described needle and suture thread can be introduced between the pair of working portions, and then the pair of working portions are closed to sandwich the needle and suture thread. It can be done.

  A dummy affected part such as a resin material (not shown) is arranged inside the dry box 11. It can be assumed that the dummy affected part is made of, for example, a rectangular parallelepiped shape, and a dummy incision part is formed on the upper surface thereof so as to be sewn. Further, in the present embodiment, the membrane material 113 is used to insert and remove only the forceps 114, and the endoscope camera 115 is fixed at an appropriate position of the dry box 11. The endoscopic camera 115 presents the treatment space to the learner so that the treatment space can be observed. The distal end action of the forceps 114 is set so that the dummy affected area is in the center of the field of view and sutures the dummy affected area. The movement of the part is imaged.

  The base 12 is provided with a foot pedal as an example of the operation unit 121 at an appropriate position, for example, at the bottom. As is well known, the operation unit 121 includes a swingable structure and a switch that is turned on by a stepping operation. In addition, the practice base apparatus 10 is provided with a motion detection unit 116 as necessary. The motion detection unit 116 includes a magnetic generator 1161 and a magnetic sensor 1162. The magnetic generator 1161 is a magnetic signal generation source, and is fixed at an appropriate position on the base 12. On the other hand, the magnetic sensor 1162 is attached to a predetermined portion of the forceps 114 so as to be directed in three axial directions. By detecting magnetic signals generated from the magnetic generator 1161 corresponding to the three axes, the three-dimensional position and orientation are detected. The position and orientation (orientation) of the action part at the distal end of the forceps 114 are described later based on the detection information by the motion detector 116 using the attachment position and direction to the forceps 114 and the known shape and size of the forceps 114. The motion information acquisition unit 215 calculates the relative information. Instead of the magnetic motion detector 116, a three-dimensional acceleration sensor may be employed.

  The monitor 13 is attached to the base 12 in this embodiment. The monitor 13 is disposed at a position that is easy for the learner to see during movement learning using the dry box 11, preferably at a position corresponding to the rear portion of the base 12 and the viewpoint height of the learner.

  The information processing apparatus 20 inputs / outputs information to / from the practice base apparatus 10, and uses the input information and other information to create a motion learning teaching material and send it to the monitor 13 as will be described later. To do.

  FIG. 2 is a circuit configuration diagram showing an embodiment of the motion learning support apparatus according to the present invention. In FIG. 2, the practice base apparatus 10 is provided with a microphone 1151 and a speaker 131. The microphone 1151 captures tips and guidance for movements of model performers as voice information at the time of creating learning materials. As will be described later, the speaker 131 is for reproducing the audio information acquired by the microphone 1151 during learning.

  The information processing apparatus 20 includes a control unit 21 configured by a CPU (Central Processing Unit), a ROM (Read Only Memory) 22 and a RAM Random Access Memory (ROM) 23 connected to the control unit 21, an operation unit 24, and as necessary. A communication unit 25 is provided. The ROM 22 stores necessary processing programs and information necessary for executing the programs in advance, and the RAM 23 executes information processing and temporarily stores processing information. The RAM 23 also stores a follow-up learning teaching material storage unit 231 that stores a series of model motion videos, and a procedure teaching material video that is divided from the series of model motion videos into video for each motion element as described later. And a procedural learning material storage unit 232.

  The operation unit 24 includes a touch panel, a mouse, a keyboard, and the like, and performs operation instructions necessary for creating learning materials and executing learning support processing. In the communication unit 25, when two motion learning support devices are connected, a model person is displayed on one side, and a learner on the other side is synthesized and displayed as a live video (in this embodiment, time division display method). In addition, they are used to exchange video information with each other.

  The information processing apparatus 20 is configured such that the procedure learning video creation unit 211, the procedure learning video display processing unit 212, the follow-up learning video display processing unit 213, the display form setting unit 214, the motion information are executed by executing the processing program in the ROM 22 on the CPU. It functions as the acquisition processing unit 215 and the communication processing unit 216 employed as necessary.

  A model operation of ligation stitching by a model person is performed in advance on the practice table device 10, and the state of this model operation is acquired by the endoscope camera 115 and the microphone 1151 and stored in the following learning material storage unit 231 of the RAM 23. Alternatively, it may be stored in the follow-up learning teaching material storage unit 231 via the teaching material capturing unit 30 from an external recording medium in which a video of a ligature stitching procedure (including sound if necessary) is stored in advance. Further, the model may adopt an image of an actual operation.

  The procedure learning video creating unit 211 creates a procedure learning video (teaching material) from a video of a ligation suture treatment by a model person. The procedure learning image creation unit 211 reproduces an image of a ligation suture treatment by a model person, and when a division condition such as the movement of the forceps 114 stops or the voice guide is interrupted during the reproduction, The video portion up to that time is divided (cut out), and a process number is assigned with a serial number, for example. The divided motion image for each time width is sequentially written in the procedure learning material storage unit 232 in correspondence with the procedure number. The stop of the movement of the forceps 114 during reproduction may be determined by picking up the image of the forceps 114 in the image and judging from the movement of the image, but from the three-dimensional information obtained by the movement detection unit 116. Alternatively, the movement of the distal end working part of the forceps 114 acquired by the movement information acquisition unit 215 described later may be determined using three-dimensional movement information (position, movement speed).

  A series of movements of the forceps 114 (which may include movements of the needle and suture thread) referred to in the ligation and suturing procedure includes a plurality of movement elements (each corresponding to each procedure) in the time direction. Yes. For example, (1) the movement of holding the needle with forceps (procedure number 1), (2) the movement of inserting the needle (procedure number 2), (3) the movement of pulling the suture thread to form the letter C (procedure number) 3), (4) Two-turn movement (procedure number 4), (5) Ligation movement (procedure number 5).

  FIG. 3 is a reference diagram showing an example of a frame image for procedure learning. (A) is a frame image for picking up a needle corresponding to procedure number 1, and (B) is a frame image for inserting a needle corresponding to procedure number 2. (C) is a picture of a frame image showing a state in which a suture thread corresponding to procedure number 3 is wound. In FIG. 3, an image captured by the endoscope camera 115 is displayed on the monitor screen 13 a, a dummy affected part image 1171 is displayed on the entire screen, and an incision part image 1181 is displayed substantially at the center. . Also, in the monitor screen 13a, a forceps image 1141, a needle image 1191 held between the distal ends of the forceps 114, and a suture thread image 1192 connected to one end of the needle image 1191 are displayed.

  The procedure learning video display processing unit 212 receives a procedure learning instruction and shifts to the procedure learning video display mode, thereby reading the video of each motion element from the procedure learning material storage unit 232 according to the assigned procedure number and monitoring 13. To display. Thereby, the learner understands each procedure, how to move the motion element of each procedure, and points. The procedure learning video display processing unit 212 switches the monitor 13 to the endoscope camera 115 side when the reproduction of the video of each motion element is completed. That is, the captured image of the endoscope camera 13 is displayed on the monitor 13 instead of the read image from the procedure learning material storage unit 232. Thereby, the learner can imitate the movement element observed immediately before.

  In addition, the procedure learning video display processing unit 212 displays the final frame video of the motion element video corresponding to the monitor 13 as a still video when reproduction of the video of each motion element is completed. Thereby, it becomes easy to grasp the final trace position of the forceps 114 in each movement element. It is preferable that the element writing display picks up and displays only the forceps image of the model person. The forceps image can be picked up, for example, in a state where light is applied to the forceps using illumination light at the time of shooting the model image, and a threshold value in the luminance direction is set in the final frame image. Alternatively, it is also possible to extract a forceps image with a color difference by attaching a predetermined color to the forceps 114. In the present embodiment, the final frame image is displayed in a blinking manner to facilitate identification from the captured image of the endoscope camera 115. The blinking display can be realized, for example, by displaying the final frame image on the monitor at predetermined time intervals.

  When the procedure learning video display processing unit 212 receives a signal from the operation unit 121 that is a foot pedal, if the procedure number is in the middle, the monitor 13 is switched to display the motion element image of the next procedure number as the procedure learning teaching material storage unit 232. Is read out and displayed on the monitor 13. On the other hand, if it is the final procedure number, a repeat process is performed to return to the first procedure number. When the operation on the operation unit 121 is performed in another manner, for example, if the operation is performed twice or another pressing operation using the length of the pressing time, it is treated as an instruction to end the procedure learning video display mode. You can also. Alternatively, another operation member may be provided.

  The follow-up learning video display processing unit 213 receives a follow-up learning instruction and shifts to the follow-up learning video display mode, thereby monitoring a series of motion images in the follow-up learning teaching material storage unit 231 and an image captured by the endoscope camera 115. 13 is a composite display, here is a time-division display alternately.

  FIG. 4 schematically shows a part of this display mode. (A) shows each frame of a series of motion images in the follow-up learning material storage unit 231, and (B) shows an image captured by the endoscope camera 115. Each frame. On the monitor 13, for example, the video is rewritten at a frame period of 16.7 ms (1/60 Hz). In this embodiment, FIG. 4 shows that (A) video continuous in the time direction is alternately switched for a display time corresponding to the number of A frames, and (B) video continuous in the time direction is switched for a display time corresponding to the number of B frames. Is displayed. Accordingly, the display period (one cycle) is a time corresponding to the number of (A + B) frames, and the ratio of the other (learner / exemplary) is (B / A).

  The display form setting unit 214 can change and set at least one of the display cycle and the self-other ratio in the follow-up learning video display mode via the operation unit 24.

  The follow-up learning video display processing unit 213 adopts a time-division display method as a video synthesis method in this embodiment, but a superimposition method for superimposing both videos and a parallel display method for writing them in parallel depending on the application target. Is conceivable. By the way, when applied to training for laparoscopic surgery, in the field of view presentation, there may be a case where the follow-up characteristic at the time of follow-up of forceps overlapping in a complicated manner with a small displacement is deteriorated. In this case, since correspondence between a large number of feature points is taken, it is considered that complicated switching of the attention between the learner's video and the model's video deteriorates the tracking characteristic. Therefore, as a method of following the attention without changing, it is considered that the time division method of alternately switching and presenting the video is preferable. As a result, it is expected that an illusion phenomenon in which the learner maintains the same feeling as the model and also causes a sense of fusion. That is, in the time-division method, when performing coordinated physical exercise, the voluntary nature of self-motion is not lost by alternately switching the video of the same viewpoint of the self-video and the teaching material video on the monitor 13, In addition, there is induction of exercise that naturally aligns the exercise with the opponent. When tracking the movement of teaching material images, the movement parts of the two displayed sequentially in the field of view merge to create an illusion that they are one's own movement part. Is possible. Here, the sense of fusion refers to the impression of voluntary and involuntary fusion that moves as expected, even though it is a movement site on its own side. In other words, it brings about a subjective feeling that it is not visible in the motion part of the teaching material video, and cannot be seen in anything other than its own motion part. As a result, it is considered that the self-side achieves the multipoint association and the execution of the tracking under the consciousness that the tracking error can be tracked in a state where the tracking error is not clearly recognized or cannot be recognized.

  In the prior application (Japanese Patent Application No. 2012-97328), the applicant of the present application has a period for producing the above-described effect when the self-motion video imaged from the same viewpoint and the video image to be tracked are displayed on the monitor by the time division method. Is approximately 2 Hz to 4 Hz in terms of frequency, and the self-other ratio is preferably 1: 1 to 1: 3. Under this condition, the tracking accuracy can be made higher by the occurrence of the fusion feeling.

  The motion information acquisition unit 215 relatively calculates the position and orientation of the distal end working unit of the forceps 114 based on the measurement information from the motion detection unit 116 described above. In addition, the movement of the learner or forceps, for example, by attaching a pressure sensor that applies finger pressure to the finger insertion part at the proximal end, measure the amount of force when the finger moves, or measure muscles at appropriate locations such as the learner's arm. It is preferable to attach an electric sensor (such as an electrode plate) and measure the myoelectric potential generated during exercise so that the smoothness and smoothness of movement can be evaluated. The movement information acquisition unit 215 further compares the measurement result of the model person's movement with the measurement result of the learner's movement, thereby obtaining a difference, for example, and can be used for the evaluation.

  FIG. 5 is a flowchart illustrating an embodiment of the procedure learning material creation process. First, reproduction of the original teaching material performed by the model is started (step S1). Next, it is determined whether or not at least one of the motion speed 0 and the sound interruption in the motion detection information is satisfied as a division condition during reproduction (step S3). If the division conditions are met, the video from the beginning to the current time, or if the division conditions are met in the first and subsequent steps, the video from the previous division location to the current time is cut out, and the procedure number is assigned sequentially. It is stored in the learning material storage unit 232 (step S5). The video may be actually cut out and stored individually, but the division part information (storage address) may be recorded in association with the procedure number, and the same division processing may be performed by address management. Next, it is determined whether or not the reproduction is finished (step S7). If completed, this flow ends.

  On the other hand, if there is no division condition in step S3, the process proceeds to step S7, and on the condition that the reproduction is not finished, the process returns to step S3 and the same cutout process is repeated.

  FIG. 6 is a flowchart illustrating an embodiment of the procedural learning material display mode process. First, i indicating the procedure number is set to i = 1 (step S21). Next, the teaching material video corresponding to the procedure number i is read from the procedure learning material storage unit 232 and displayed (reproduced) on the monitor 13 (step S23). When the teaching material video of the procedure number i is read out, it is determined whether or not the reading of the teaching material video is finished (reproduction is finished) (step S25). If the reading of the teaching material video has not been completed (if playback is in progress), the process returns to step S23. On the other hand, if the reading of the teaching material video is completed, the monitor 13 is switched to the endoscope camera 115 side, and the last frame video of the teaching material video of the procedure number i is blinked and displayed together. (Step S27).

  In this state, the learner can practice the movement corresponding to the procedure number i while observing the monitor 13. Subsequently, a transition instruction operation to the teaching material video in the next procedure, that is, a standby state for the operation by the operation unit 121 is entered (step S29). If there is an instruction to move to the teaching material video of the next procedure (Yes in step S29), the procedure number i is incremented to i + 1 (step S31), and then the procedure number i> I, that is, whether the last procedure number I has been exceeded. It is determined whether or not (step S33). If the procedure number i> I is not satisfied, the process returns to step S23, and the teaching material video of the next procedure number is guided to the monitor 13.

  On the other hand, if the procedure number i> I, the procedure learning is completed up to the teaching material video of the last procedure number, and it is determined again whether or not the procedure learning is performed, that is, the presence or absence of the repeat instruction operation (step). ST35). Here, if there is a repeat instruction operation, the procedure returns to step S21 and the procedure learning is started from the teaching material video of the first procedure number. On the other hand, if there is no repeat instruction operation (or if there is an operation instruction to end the procedure learning), the procedure shifts from the procedure learning to the follow-up learning teaching material display mode for the follow-up learning (step S37).

  FIG. 7 is a flowchart showing an embodiment of the follow-up learning material display mode process. In this flowchart, the time division switching display process is executed (step S41). This time-division switching display process can be executed as many times as desired by accepting an instruction operation to the operation unit 121 according to the intention of the learner, and can be ended (step S43).

  FIG. 8 is a flowchart showing an embodiment of the time-division switching display process in the follow-up learning material display mode process. First, a teaching material video is read from the follow-up learning teaching material storage unit 231 and output to the monitor 13, and the number of frames is counted (step S51). Next, when display is performed for the number of A frames (Yes in step S53), reading of the teaching material video is interrupted when the number of A frames is read (step S55).

  Subsequently, the monitor 13 is switched to the endoscope camera 115 side, the captured image of the endoscope camera 115 is displayed on the monitor 13, and the frame number counting process is performed (step S57). Next, when the display is performed for the number of B frames (Yes in step S59), the monitor 13 is switched to the teaching material video side, and the teaching material video is sent to the monitor 13 from the address location just interrupted in the follow-up learning teaching material storage unit 231. The number of frames is output and the number of frames is counted (step S61). Subsequently, it is determined whether or not the reading of the teaching material video is finished (step S63). If the reading of the teaching material video is not finished, the process returns to step S53, and similarly, the teaching material video of the number of A frames and the number of B frames are returned. The process of alternately displaying the video of the endoscope camera on the monitor 13 is repeated. On the other hand, if the reading of the teaching material video has been completed, this flow ends.

  In the time-division switching display process, the A frame is read from the following learning material storage unit 231 (Yes in step S53), but frame images are continuously read from the following learning material storage unit 231. In addition, while the captured image of the endoscope camera 115 is displayed, it may not be guided to the monitor 13 or may be displayed without being displayed. By doing in this way, the change of the time direction of the frame image from the following learning material storage unit 231 corresponds to the real time.

  Next, FIGS. 9 and 10 are tables for explaining the effects. FIG. 9A is a chart showing the transition of the time required for a series of operations performed by each experimenter according to the progress of learning. FIG. 9B is a comparative experimenter and each experimenter after completion of learning. It is a graph which shows the difference of the time required for each series of operation | movement by these. FIG. 10 (A) is a chart showing the transition of the number of errors (motion errors) caused during each series of operations performed by each experimenter according to the progress of learning, and FIG. It is a graph which shows the difference of the average of the number of errors which occurred during each series of operation | movement by each experimenter after completion | finish of learning.

  FIG. 9 and FIG. 10 show experimental results when procedural learning is performed for 30 minutes for five trainees A, B, C, D, and E, and then follow-up learning (forceps movement learning) is performed for 30 minutes. Show. In the follow-up learning, the follow-up learning and the self practice were performed alternately, and the time required for the self practice was 26 minutes. First, in FIG. 9 (A), the horizontal axis represents the time elapsed during learning and shows each time point of 0 minutes, 15 minutes, 30 minutes, 45 minutes, and 60 minutes, and the vertical axis represents the ligation performed at each time point. The required time (seconds) required for a series of movement operations for this purpose is shown. As shown in FIG. 9 (A), the time required on average about 120 seconds before the start of learning shows a tendency to gradually decrease as it progresses to 15 minutes, 30 minutes, and 45 minutes. It can be seen that after learning, almost everyone has fallen to about 40 seconds. This confirms that this learning method is effective for everyone.

  In FIG. 9B, the horizontal axis uses only the conventional learning platform device 10 regardless of whether the system is used, that is, the trainees A to E who have learned for 60 minutes as described above, and the system is not used, that is, this learning method. The self-taught doctors G, H and I who practiced for a total of 7 hours (no ligature suture operation results) are shown, and the vertical axis is required for a series of movements for ligation after completion of learning by each resident Indicates time (seconds). When the average time required for the skilled doctor is about 35 seconds, the average of the trainees A to E is about 40 seconds and there is little variation. On the other hand, for self-taught doctors G, H, and I, the average is about 50 seconds, and a slight difference from the average of the trainees A to E is recognized. In addition, there is a problem that the variation (individual difference) among each person is large.

  FIG. 10A shows the transition of the number of errors in a series of movements for ligation before learning, after 30 minutes (that is, when only procedural learning is completed) and after 60 minutes for each of the residents A to E. Is shown.

  The error checklist is a list of important points for ligature stitching, and is as follows. Here, 19 are listed. The error contents at the time of follow-up learning are marked with * to distinguish them from those at the time of procedure learning.

Procedure number 1 Contents: Needle gripping (changing)
(01) Is the angle first considered so that the needle can be easily grasped? (02) Whether the needle is held at a right angle (or a slightly obtuse angle than the right angle) (03) 1/3 to 1/4 of the needle (04 *) How to grip the needle smoothly (Is there any error)
Procedure No. 2 Contents: Needle insertion (05 *) Is the relationship between the needle insertion position and the needle insertion position approximately equal? (06) Is the needle pointed at the target point and the needle tip at the target point? (Does the needle tip come out from a horizontal position with the insertion position?)
(07) Is the needle pulled out so as not to damage the tissue? (08) Whether the needle is pulled by holding the forceps in order and twisting it clockwise. Procedure number 3 Contents: Pull the thread, C shape Formation (09 *) Is the thread being manipulated smoothly? (10 *) Is the left hand grasping the thread without error and can be pulled at once? (11) Is the short tail too long (1.5cm- About 2.0cm)
Procedure No. 4 Contents: Surgical ligation (2 turns)
(12) Can the thread held with the right hand be raised? (13) Is the left forceps curved upward? (14 *) Is the right hand and left hand moving together and wound (15) Is the wrapping operation near the suture site? Procedure number 5 Description: Ligation (pull the thread on the side with the needle with the left forceps)
(16) Check if the right forceps is away from the left forceps when grasping the short tail thread with the left forceps (17 *) Can the short tail be grasped without error (18 *) After grabbing the left forceps (19) Can it be tightened with an appropriate force without loosening?

  As shown in FIG. 10 (A), the trainee A had a total of 13 at the time of procedural learning and 4 at the time of follow-up learning before learning, but 13 at the time of procedural learning after 30 minutes. The number is greatly reduced to 1 at the time of procedure learning after minutes. Before learning, Resident B had 9 during procedural learning and 4 during follow-up learning, but a total of 13 during procedural learning after 30 minutes, 1 during follow-up learning, and after 60 minutes It is greatly reduced to zero. Before learning, Resident C had 12 in total, 8 during procedural learning and 4 during follow-up learning, but 4 during 30-minute procedural learning and 3 during follow-up learning. Reduced to 2 enemies and 2 level enemies during procedure learning. Before learning, Resident D was 10 at the time of procedural learning and 3 at the time of follow-up learning, but it was 3 at the time of procedural learning after 30 minutes and 3 at the time of follow-up learning. The number is reduced to 2 for step learning and 1 for follow-up learning. Before learning, Resident E had 13 in total, 10 during procedural learning and 3 during follow-up learning, but 2 during procedural learning after 30 minutes and 2 during follow-up learning. Reduced to 3 level enemies during procedural learning and 1 during follow-up learning.

Thus, it can be seen that the error is greatly reduced at the stage after 30 minutes, and then gradually reduced after 60 minutes.

  Further, as shown in FIG. 10B, the average number of errors caused during each series of operations by each experimenter (residents A to E) after the end of learning using the present system is 19 On the other hand, the average number of errors that occurred during each series of operations by the comparative experimenters (self-taught doctors GI) who did not use this system was 13 out of 19, and there was a large difference. It can be seen that the learning method using this system has a remarkable learning effect.

  Since the motion learning device 1 has a structure realized by incorporating the information processing device 20 in the conventional device, the motion learning device 1 is compared with the conventional method that requires a robot-type actuator sensor and a simulation system that requires advanced calculation processing. Thus, it can be introduced at a very low cost.

  The present invention may adopt the following aspects.

(1) In the present embodiment, the example of the ligature suture treatment has been described. However, the present invention is not limited to this, and the present invention is also applicable to a specialized medical treatment performed with an endoscope camera. In addition, teleoperation using a surgical robot while remotely observing the affected area image via a monitor is also specialized, and since the number of doctors that can be handled is small, it can be applied to various medical procedures in such remote surgery. is there.

(2) The application range of the present invention is not limited to the medical field, and can be applied to other fields. For example, there are various operation elements according to the field where skill or skill is required, for example, the production process of a product (work), and a series of operations are completed through these operation elements to complete a product etc. A field is envisaged. In addition, the present invention can also be applied to teaching materials for cooking training in which a specific cooking is performed on each ingredient and finally a target dish is completed.

(3) In this embodiment, the monitor 13 is arranged in front of the learner. As the monitor, a video see-through head mounted display (VST-HMD) as a device that realizes visual guidance of body movement is used. A mode to be adopted may be used.

(4) Further, when the video of the endoscope camera 115 is guided to the monitor 13, a delay process for setting a predetermined time lag is performed by the information processing apparatus 20, so that a sense of force can be given to the learner, and a more realistic feeling can be provided. A certain learning can be expected. Further, by interposing a digital filter that performs differential processing on the video, the edge of the video can be enhanced, and video recognition is improved.

(5) In the present embodiment, each time of the time-division display has been described by setting the number of frames. However, a mode in which time measuring means is provided instead of counting the number of frames and time division switching is performed may be used.

(6) In the present embodiment, switching from procedural learning to follow-up learning is performed by an instruction to the operation unit 121. Instead of this, follow-up learning is automatically performed after a lapse of a certain time from the end of procedural learning. You may make it transfer.

DESCRIPTION OF SYMBOLS 1 Movement learning assistance apparatus 10 Practice stand apparatus 11 Dry box 114 Forceps 115 Endoscope camera (imaging part)
121 Operation Unit 13 Monitor 20 Information Processing Device 21 Control Unit 211 Procedure Learning Video Creation Unit (Motion Element Video Creation Unit)
212 Procedure learning video display processing unit (first learning support processing means)
213 Follow-up learning video display processing unit (second learning support processing means)
23 RAM
231 Follow-up learning material storage unit 232 Procedure learning material storage unit

Claims (7)

A motion learning support device that learns through a series of motion learning videos including a plurality of motion elements connected in the time direction,
A monitor that displays video,
An imaging unit that images the learner's movement at the same shooting position as the teaching material video;
First learning support processing means for individually displaying the motion element video corresponding to the motion element of the teaching material video on the monitor and displaying the captured video of the imaging unit on the monitor following the display of the motion element video When,
A motion learning support apparatus comprising: a second learning support processing unit configured to synthesize and display the series of motion instructional videos and the captured video of the imaging unit on the monitor. The motion learning support apparatus according to claim 1, wherein the second learning support processing unit is allowed to execute after the first learning support processing unit is executed. The motion learning support apparatus according to claim 1, further comprising a motion element video creation unit that extracts each motion element video from the teaching material video including the series of motions. The said 1st learning assistance processing means displays the last frame image | video of the motion element image | video corresponding to the image pick-up image of the said image pick-up part together on the said monitor, The display in any one of Claims 1-3 characterized by the above-mentioned. Motion learning support device. Has an operation unit that can be operated by the learner
The first learning support processing unit displays one motion element image on the monitor, displays the imaged image of the imaging unit on the monitor, and then accepts an operation to the operation unit. The movement learning support apparatus according to claim 1, wherein the movement learning image is shifted to display on the monitor of the next movement element image. The movement according to claim 1, wherein the second learning support processing unit alternately displays the teaching material video of the series of motions and the captured video of the imaging unit on the monitor. Learning support device. The motion learning support according to claim 1, wherein the teaching material video of the series of motions is a video obtained by photographing a ligature suture using forceps during laparoscopic surgery with an endoscope camera. apparatus.